Overview
The Microsoft Azure Storage Client Library for C++ is a library built on top of
the C++ REST SDK that lets you access Azure Storage from your C++ apps. You need
to install it via NuGet. While installing it, I could not locate it using the NuGet
packages UI in Visual Studio 2013. Instead I had to install it via the Package Manager
Console. I assume it's because the library is pre-release. The latest version is
0.3.0 (preview) dated May 16 2014. So when you install it, you need to add the
-pre
parameter.
install-package wastorage -pre
This will add the REST SDK and other dependencies like the wastorage.redist
to your project.
Accessing Azure tables
To test if it works, I created a table using a quickly put together
C# project and then accessed it from a C++ console app. I just used the Azure storage
emulator. Here are the includes you need to have in your project.
#include "../packages/wastorage.0.3.0-preview/build/native/include/was/storage_account.h"
#include "../packages/wastorage.0.3.0-preview/build/native/include/was/table.h"
You can also add this using namespace
declaration to save some keystrokes.
using namespace azure::storage;
Here's the code snippet, and I've added comments to make it easier to understand.
auto storage_account = cloud_storage_account::parse(U("UseDevelopmentStorage=true"));
auto table_client = storage_account.create_cloud_table_client();
auto table = table_client.get_table_reference(U("Clients"));
bool created = table.create_if_not_exists();
table_query query;
auto results = table.execute_query(query);
for (auto item : results)
{
auto properties = item.properties();
for (auto property : properties)
{
ucout << property.first << U(" = ") << property.second.str() << U("\t");
}
ucout << endl;
}
Example output:
Id = 100 Name = John Brown Phone = 777-1234
Id = 101 Name = Mary Jane Phone = 777-5678
Now this was basically as easy as using C#. The one difference, and it's a non-trivial
one, is that with C# you can map the returned table_entity
to a .NET
type. So I can write something like this:
TableQuery<client>query = new TableQuery<client>();
foreach (var item in table.ExecuteQuery(query))
{
Console.WriteLine("{0} - {1}", item.Name, item.Phone);
}
Here, Client
is an entity object.
class Client : TableEntity
{
public Client(int id)
{
this.Id = id;
this.PartitionKey = id.ToString();
this.RowKey = id.ToString();
}
public Client()
{
}
public string Name { get; set; }
public string Phone { get; set; }
public int Id { get; set; }
}
I am guessing here, but it's most likely done via reflection. With C++, you'd need
to write plumbing code to map the returned data to your C++ objects. Not a big deal
really, and more a matter of coding convenience. Probably more performant to do
it the C++ way (that's an unverified personal thought).
Inserting data
The following code snippet shows how you can insert data into Azure tables.
void InsertTableData(string_t key, int id, string_t name, string_t phone)
{
auto storage_account = cloud_storage_account::parse(
U("UseDevelopmentStorage=true"));
auto table_client = storage_account.create_cloud_table_client();
auto table = table_client.get_table_reference(
U("Clients"));
bool created = table.create_if_not_exists();
table_entity entity(partitionKey, key);
auto& properties = entity.properties();
properties.reserve(3);
properties[U("Name")] = entity_property(name);
properties[U("Phone")] = entity_property(phone);
properties[U("Id")] = entity_property(id);
auto operation = table_operation::insert_entity(entity);
auto result = table.execute(operation);
}
Azure table data is really just about properties. So what's involved is creating
a new table entity, setting up some properties (named keys with values), and then
an insert_entity
operation is executed against the table.
Updating data
Here's how you'd update table data.
void UpdateTableData(string_t key, int id, string_t name, string_t phone)
{
auto storage_account = cloud_storage_account::parse( U("UseDevelopmentStorage=true"));
auto table_client = storage_account.create_cloud_table_client();
auto table = table_client.get_table_reference( U("Clients"));
bool created = table.create_if_not_exists();
auto operation = table_operation::retrieve_entity( partitionKey, key);
auto result = table.execute(operation);
auto entity = result.entity();
auto& properties = entity.properties();
properties[U("Name")] = entity_property(name);
properties[U("Phone")] = entity_property(phone);
properties[U("Id")] = entity_property(id);
auto operationUpdate = table_operation::replace_entity(entity);
result = table.execute(operationUpdate);
}
This code is very similar. The retrieve_entity
operation is used
to access the entity we need to update. The property values are updated, and then
a replace_entity
operation is executed.
Deleting data
void DeleteTableData(string_t key)
{
auto storage_account = cloud_storage_account::parse( U("UseDevelopmentStorage=true"));
auto table_client = storage_account.create_cloud_table_client();
auto table = table_client.get_table_reference( U("Clients"));
bool created = table.create_if_not_exists();
auto operation = table_operation::retrieve_entity( partitionKey, key);
auto result = table.execute(operation);
auto entity = result.entity();
auto operationUpdate = table_operation::delete_entity(entity);
result = table.execute(operationUpdate);
}
This is very similar to the update, except that a delete_entity
operation is executed.
Querying table data
While not as powerful as SQL, Azure tables do allow
you to do minimal querying. With the native SDK, you'd do this using the
table_query
object's set_filter_string
function. Here's a modified ReadTableData
method from the previous example.
void ReadTableData(string_t filter)
{
auto storage_account = cloud_storage_account::parse(
U("UseDevelopmentStorage=true"));
auto table_client = storage_account.create_cloud_table_client();
auto table = table_client.get_table_reference(U("Clients"));
bool created = table.create_if_not_exists();
table_query query;
query.set_filter_string(filter);
auto results = table.execute_query(query);
for (auto item : results)
{
auto properties = item.properties();
for (auto property : properties)
{
ucout << property.first << U(" = ")
<< property.second.str() << U("\t");
}
ucout << endl;
}
}
Here's an example query that gets all rows within a range of RowKey
values.
void ReadTableData(string_t rowKeyStart, string_t rowKeyEnd)
{
auto filter = table_query::combine_filter_conditions(
table_query::generate_filter_condition(
U("RowKey"),
query_comparison_operator::greater_than_or_equal,
rowKeyStart),
query_logical_operator::and,
table_query::generate_filter_condition(
U("RowKey"),
query_comparison_operator::less_than_or_equal,
rowKeyEnd));
ReadTableData(filter);
}
The combine_filter_conditions
function is used to create a query string. The query_comparison_operator
class allows you to set comparison operators and the query_logical_operator
class lets you set logical operators. In case you are wondering, that gets converted
to the following string.
(RowKey ge '100') and (RowKey le '104')
Here's a similar method, that queries against the Name
column.
void ReadTableDataStartingWith(string_t letter1, string_t letter2)
{
auto filter = table_query::combine_filter_conditions(
table_query::generate_filter_condition(
U("Name"), query_comparison_operator::greater_than_or_equal, letter1),
query_logical_operator::and,
table_query::generate_filter_condition(
U("Name"), query_comparison_operator::less_than, letter2));
ReadTableData(filter);
}
The
generated query filter looks like this.
(Name ge 'D') and (Name lt 'E')
You can call it as follows :
ReadTableDataStartingWith(U("D"), U("E"));
That'd return all rows with names starting with a D. Querying against partition
and row keys would be a faster approach. Also, for repeated querying against a set
of data, you may want to fetch a larger subset of the data and then query it in
memory using standard STL.
Accessing Azure blob storage
Blob storage is for storing large amounts of semi-structured
or unstructured data such as images, videos, documents etc. The blob service lets
you create named containers that can then contain one or more named blobs which
can be publicly (optional) accessed via an URI. You need to add this header
include.
#include "../packages/wastorage.0.3.0-preview/build/native/include/was/blob.h"
Creating blobs
void CreateTextBlobs()
{
auto storage_account = cloud_storage_account::parse(
U("UseDevelopmentStorage=true"));
auto blob_client = storage_account.create_cloud_blob_client();
auto container = blob_client.get_container_reference(
U("textdata"));
bool created = container.create_if_not_exists();
blob_container_permissions permissions;
permissions.set_public_access(
blob_container_public_access_type::container);
container.upload_permissions(permissions);
auto text_blob1 = container.get_block_blob_reference(
U("texts/text1"));
text_blob1.upload_text(U("This is some text - modified"));
auto text_blob2 = container.get_block_blob_reference(
U("texts/text2"));
text_blob2.upload_from_file(U("./stdafx.h"));
}
The classes/methods used are very similar to that used for table storage. Notice
the use of blob_container_permissions
to set the public access
level for the container. It's off by default, and you can optionally set it to blob (clients can read blob data) or container (clients
can list blobs in the container and read blob data). You can simulate directories
in the blob names. In the above example, both text1 and text2 are under the texts
directory. This just affects the URI and are not physical directories.
Listing blobs
void ListTextBlobs()
{
auto storage_account = cloud_storage_account::parse(
U("UseDevelopmentStorage=true"));
auto blob_client = storage_account.create_cloud_blob_client();
auto container = blob_client.get_container_reference(
U("textdata"));
bool created = container.create_if_not_exists();
continuation_token token;
auto result = container.list_blobs_segmented(token);
for (auto dir : result.directories())
{
ucout << U("Directory: ") << dir.uri().path() << endl;
ucout << endl;
continuation_token dir_token;
auto resultInner = dir.list_blobs_segmented(dir_token);
for (auto item : resultInner.blobs())
{
ucout << item.name() << endl;
ucout << item.uri().path() << endl;
ucout << item.properties().content_type() << endl;
ucout << endl;
}
}
}
To list the blobs in a container, we need to use a continuation_token
object. The container object supports a list_blobs_segmented
method
which takes this token. For each directory returned, we can then call list_blobs_segmented
using a separate continuation_token
object. Once we iterate through
the blobs in the directory, we can access properties like the name, URI, content
type etc. Here's a sample output from calling the above function.
Directory: /devstoreaccount1/textdata/texts/
texts/text1
/devstoreaccount1/textdata/texts/text1
text/plain; charset=utf-8
texts/text2
/devstoreaccount1/textdata/texts/text2
application/octet-stream
Notice how the text data we uploaded is of type text/plain whereas the file we uploaded
is of type application/octet-stream. If you paste the first blob's URI in a browser,
the text is directly returned. Whereas in the second case, the file is offered for
download - obviously this is browser specific behavior based on the content type.
Downloading blob content
The SDK makes it really trivial to extract
blob data. For text content you can directly get the text, and for non-text content
you can either download to a file or to a stream. The code snippet below shows how
it's done.
void DownloadBlobData()
{
auto storage_account = cloud_storage_account::parse(U("UseDevelopmentStorage=true"));
auto blob_client = storage_account.create_cloud_blob_client();
auto container = blob_client.get_container_reference(U("textdata"));
bool created = container.create_if_not_exists();
auto text_blob1 = container.get_block_blob_reference(U("texts/text1"));
auto text = text_blob1.download_text();
ucout << text << endl;
auto text_blob2 = container.get_block_blob_reference(U("texts/text2"));
text_blob2.download_to_file(U("d:\\tmp\\blobdata.txt"));
stringstreambuf buffer;
concurrency::streams::ostream output(buffer);
text_blob2.download_to_stream(output);
cout << buffer.collection() << endl;
}
Using Azure queues
Azure queues are used to store a large number of items (referred
to as messages). Each message can go up to 64 Kb. A typical use for queues is intra-app
communication, for example your website might add messages to the queue for a background
service to process later on. The C++ Azure storage SDK maintains its consistent
API when it comes to queues as well (so if you know how to use blobs and tables,
you are good). You need the following header include.
#include "../packages/wastorage.0.3.0-preview/build/native/include/was/queue.h"
Here's a code snippet that shows how to insert messages into a queue.
void InsertQueueMessages()
{
auto storage_account = cloud_storage_account::parse(U("UseDevelopmentStorage=true"));
auto queue_client = storage_account.create_cloud_queue_client();
auto queue = queue_client.get_queue_reference(U("sample"));
bool created = queue.create_if_not_exists();
queue.add_message(cloud_queue_message(U("queue message 01")));
queue.add_message(cloud_queue_message(U("queue message 02")));
queue.add_message(cloud_queue_message(U("queue message 03")));
}
Like any decent queue, you can either peek or dequeue a message.
Here's an example snippet that shows how to peek into a queue.
void PeekQueueMessage()
{
auto storage_account = cloud_storage_account::parse(U("UseDevelopmentStorage=true"));
auto queue_client = storage_account.create_cloud_queue_client();
auto queue = queue_client.get_queue_reference(U("sample"));
bool created = queue.create_if_not_exists();
queue.download_attributes();
int count = queue.approximate_message_count();
ucout << U("Approx count = ") << count << endl;
auto messages = queue.peek_messages(2);
for (auto message : messages)
{
ucout << message.content_as_string() << endl;
}
}
The count method has 'approximate' in the name because
the count returned is not refreshed until the next call to download_attributes
.
Here's a code snippet that shows how to dequeue items from a queue.
void DeQueueMessages()
{
auto storage_account = cloud_storage_account::parse(
U("UseDevelopmentStorage=true"));
auto queue_client = storage_account
.create_cloud_queue_client();
auto queue = queue_client.get_queue_reference(
U("sample"));
bool created = queue.create_if_not_exists();
queue.download_attributes();
int count = queue.approximate_message_count();
ucout << U("Approx count = ") << count << endl;
for (size_t i = 0; i < count; i++)
{
auto message = queue.get_message();
queue.delete_message(message);
ucout << message.content_as_string() << endl;
}
queue.download_attributes();
count = queue.approximate_message_count();
ucout << U("Approx count = ") << count << endl;
}
Notice how I call delete_message
after calling get_message
.
This is because get_message
only hides the item from other callers
for a default time out (30 seconds) after which the message is back. This allows
you to rollback changes in case something goes wrong. So you'd get a message, process
it, and when you are done processing it, you can call delete_message
.
This assumes you do your processing in under 30 seconds. If you want to increase
(or decrease) that time out, you can do that too.
The default invisibility
timeout of 30 seconds may not suffice on occasions. For those situations, you can
change the timeout so that you get more time to do something with your message prior
to deleting it.
void DeQueueMessagesWithTimeOut(std::chrono::seconds timeout)
{
auto storage_account = cloud_storage_account::parse(
U("UseDevelopmentStorage=true"));
auto queue_client = storage_account.create_cloud_queue_client();
auto queue = queue_client.get_queue_reference(
U("sample"));
bool created = queue.create_if_not_exists();
queue.download_attributes();
int count = queue.approximate_message_count();
ucout << U("Approx count = ") << count << endl;
queue_request_options options;
operation_context op_context;
auto messages = queue.get_messages(
count, timeout, options, op_context);
for (auto message : messages)
{
queue.delete_message(message);
ucout << message.content_as_string() << endl;
}
queue.download_attributes();
count = queue.approximate_message_count();
ucout << U("Approx count = ") << count << endl;
}
You can also update an existing message. The code snippet below shows how that can
be done.
void UpdateQueueMessage()
{
auto storage_account = cloud_storage_account::parse(
U("UseDevelopmentStorage=true"));
auto queue_client = storage_account.create_cloud_queue_client();
auto queue = queue_client.get_queue_reference(
U("sample"));
bool created = queue.create_if_not_exists();
auto message = queue.get_message();
message.set_content(U("Updated content"));
queue.update_message(
message, std::chrono::seconds(0), true);
}
One thing to be aware of is that the position of the message in the queue is lost.
The updated content goes back to the back of the queue. This shouldn't matter though
for most practical scenarios as the order of items in the queue is not expected
to be depended on by consumers. This might be a little different to how you used
queues in your programming classes, where it was a guaranteed FIFO structure. But
remember, those queues did not allow in-place edits either.
That's it. As always, your feedback is most welcome, including (and
especially) critical ones. Please check out my blog for related write-ups on similar topics, specially relating to the use of C++ with Microsoft platforms and libraries.
History
- Article published - June 17th 2014