Click here to Skip to main content
65,938 articles
CodeProject is changing. Read more.
Articles / Languages / C#

Windows Azure Storage Extensions

4.97/5 (12 votes)
11 Aug 2013CPOL6 min read 56K   453  
.NET library aimed for managing and querying entities from Windows Azure Storage. Contains LINQ to Azure Table provider.

Introduction

Windows Azure Storage Extensions is a .NET library aimed for managing and querying entities from Windows Azure Storage. It's built on top of the Windows Azure SDK v2.0, provides asynchronous interfaces (Task-based Asynchronous Pattern) and LINQ to Azure Table queries via TableSet context by using POCO entities. 

Table of Contents

Windows Azure Storage Extensions

Background

Currently Windows Azure SDK v1.x with LINQ to Azure Table provider marked as obsolete. Its successor Windows Azure SDK v2.0 was written from scratch and has few breaking changes which includes new querying mechanism. It's implementation has following gaps.

Querying by using TableQuery 

Now if we don't want to use legacy WCF Data Services we should construct our queries to Azure Tables by using ugly code like that:

C#
string filter = TableQuery.GenerateFilterCondition(
    "RowKey", QueryComparisons.GreaterThanOrEqual, "5");
TableQuery<TableEntity> query = new TableQuery<TableEntity>().Where(filter).Take(5);
var entities = myTable.ExecuteQuery(query);

Requirements to derive from TableEntity in POCO objects

For using POCO entities we should use TableEntity base class: 

C#
public class SampleEntity : TableEntity
{
    public int SampleProperty { get; set; }  
}

Except that it grant us ITableEntity interface with PartitionKey, RowKey, Timestamp, and ETag properties it uses slow Reflection for entities serialization / deserialization. 

Low Abstraction Level

For me as a software engineer Azure Storage Library v2.0 looks too low level abstraction of Azure Tables REST API. For instance when I want to use DTO entities I must remember that PartitionKey is a user identifier, actually it's not meaningful and we should use some kind of entity mappers. Querying also is a too complicated task.

Windows Azure Storage Extensions

Windows Azure Storage Extensions library offering following features. 

POCO 

Entity's properties and fields should be marked by one or both of PartitionKey and RowKey attributes for defining composite table key. Also can be used Timestamp, ETag, Property and Ignore attributes.

AttributeDescriptionConstraint 
PartitionKeyDefines a partition key property.Only for string properties.
RowKey Defines a row key property.Only for string properties.
ETagDefines an etag property.Only for string properties.
TimestampUsed for receiving table entity timestamp.Only for DateTime or DateTimeOffset properties.
PropertyUsed for defining custom table property name via Name property.
IgnoreUsed for skipping properties from serialization / deserialization.

Table Entities Management

Generic TableSet context provides both synchronous & asynchronous (TAP) methods for managing entities:

  • Synchronous: Add, AddOrUpdate, Update, and Remove
  • Asynchronous: AddAsync, AddOrUpdateAsync, UpdateAsync, and RemoveAsync.

To avoid restrictions of group operations in Azure Storage all entities sorted by partition keys and merged into groups by 100 entities. Execution of requests with such batch operations can be configured via TableSet's ExecutionMode property. Allowed values:

  • Sequential
  • Parallel

Default ExecutionMode is Sequential.

LINQ to Azure Tables 

TableSet context implements IQueryable interface for using LINQ Expressions. Provider supports next synchronous LINQ methods:

  • First
  • FirstOrDefault
  • Single
  • SingleOrDefault
  • Take
  • Where

To utilize filtering capabilities of string properties it supports following methods: 

Also you can use Contains method. In this case query statement for each collection's item will be joined by using OData or operator.

Note: For creating a custom queries you should take a look at next article Mixing LINQ Providers and LINQ to Objects.

Asynchronous LINQ Queries

In addition TableSet can be used for asynchronous queries powered by LINQ extensions (TAP) in EF 6 Async style. Available methods:

  • FirstAsync
  • FirstOrDefaultAsync
  • SingleAsync
  • SingleOrDefaultAsync
  • TakeAsync
  • ToListAsync

LINQ Projections

LINQ Projections supported with a limitation - projection class should be a reference type.

Task-based Extension Methods

Library contains TAP-based extensions for a following Windows Azure SDK classes:

  • CloudBlobClient
  • CloudBlobContainer
  • CloudTableClient
  • CloudTable

To use it just add Async postfix to synchronous method name like that:

C#
blobs = cloudBlobContainer.ListBlobs();
blobs = await cloudBlobContainer.ListBlobsAsync();

Task Cancellation

All of TAP-based methods accepts optional CancellationToken parameter for Task Cancellation.

Download

Via NuGet

To install library by using Windows Azure Storage Extensions nuget package execute following command:

PM> Install-Package WindowsAzure.StorageExtensions 

Via Git

To get the source code of the library via git just type:

git clone git://github.com/dtretyakov/WindowsAzure.git
cd ./WindowsAzure 

Example

You can download SPA example powered by ASP.NET MVC4 & Web API. It uses Azure Table Storage as a persistence, so before you run that example start Azure Storage Emulator or configure proper connection string in the Web.config.

Example App

One of the interesting things in this example is a Get method in the IssuesController. At runtime SPA sends an OData queries to WebAPI IssuesController which by leveraging Queryable attribute transforms into LINQ queries to the TableSet context which in turn translates LINQ expressions into OData filters for Azure Storage Table service. 

Code Samples

Defining a new class:

C#
public sealed class Country
{
    [PartitionKey]
    public string Continent { get; set; }
    [RowKey]
    public string Name { get; set; }
    public long Population { get; set; }
    public double Area { get; set; }
    public DateTime Formed { get; set; }
}

Creating a new table context:

C#
var storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
var tableClient = storageAccount.CreateCloudTableClient();

var countryTable = new TableSet<Country>(tableClient);

Adding a new entities:

C#
var resultSync = countryTable.Add(country);
var resultAsync = await countryTable.AddAsync(country);

Updating an entities:

C#
resultSync.Area += 333333;
resultSync = countryTable.Update(resultSync);
 
resultAsync.Population *= 2;
resultAsync = await countryTable.UpdateAsync(resultAsync);

Removing an entities: 

C#
countryTable.Remove(resultSync);
await countryTable.RemoveAsync(resultAsync);

Querying entities:

C#
var query = countryTable.Where(
    p => p.Formed > new DateTime(1950, 1, 1) &&
        (p.PresidentsCount < 10 || p.Population < 10000000 && p.IsExists));
 
var result = query.ToList();
result = await query.ToListAsync();

or using string filtering methods:

C#
var countryStartingWithF = await countryTable.FirstAsync(
    p => p.Name.CompareTo("F") >= 0 && p.Name.CompareTo("G") < 0);

Using LINQ projections:

C#
var projection = from country in countryTable
    where country.Area > 400000
    select new { country.Continent, country.Name };

var entities = projection.ToList();
entities = await projection.ToListAsync();

Using Contains method in the LINQ query:

C#
var countryNames = new List<string> { "Germany", "Finland" };
var countries = countryTable.Where(p => countryNames.Contains(p.Name)).ToList();

Points of Interest

Source code of the library are available at the github.com/dtretyakov/windowsazure.  Core components of the Windows Azure Storage Extensions are following.

Table Context

ITableSet

TableSet<T> class is an abstraction over Azure Storage Table. It implements IQueryable<T> and ITableSet<T> interfaces. Internals in actions it performs single and batch requests to the Azure Storage Tables by using CloudTable class and uses POCO object serialization by leveraging TableEntityConverter<T>.

Table Entity Converter

TableEntityConverter<T> implements ITableEntityConverter<T> interface. It uses compiled Expression Trees for fastest access to POCO fields and properties. Internally it converts POCO objects into DynamicTableEntity and back. Performance comparison results in ms of TableEntityConverter for POCO, TableEntity and new operator are following:

Conversion10M50M
DynamicTableEntity to Object850143,690
DynamicTableEntity to POCO17,82689,907
DynamicTableEntity to TableEntity54,158276,421
Object to DynamicTableEntity10,69351,485
POCO to DynamicTableEntity15,52878,368
TableEntity to DynamicTableEntity41,471218,935

Tests can be found in: EntityConverterTests.cs

Table Query Provider

TableQueryProvider<T> implements IQueryProvider and IAsyncQueryProvider interfaces. It provides translation of LINQ Expressions into Azure Storage Table OData filters and able to execute it synchronously and asynchronously.

Task-based Extension Methods 

Windows Azure SDK v2.0 provides us APM-based interfaces and ICancellableAsyncResult for operation cancellation. .NET 4.0 and higher uses other TAP-based asynchronous programming model. Thus Windows Azure Storage Extensions provides APM to TAP wrappers for most of  Windows Azure SDK classes. Library internally uses these extensions for executing asynchronous requests to Azure Storage Tables.

Feedback 

Your suggestions and comments are very welcome at the GitHub project page

History

  • 0.1.0 - December 16, 2012: Alpha version.
  • 0.7.0 - April 12, 2013: First stable version.
  • 0.7.2 - April 22, 2013: Added LINQ projections, entities partitioning.
  • 0.7.3 - April 27, 2013: Serialization performance improvements, ITableSet's methods provides IEnumerable again.
  • 0.7.4 - May 10, 2013: Added Contains method; performance optimization of the LINQ to OData translator.
  • 0.7.6 - May 20, 2013: Performance optimizations and code stabilization.
  • 0.7.7 - August 10, 2013: Code stabilization. 

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)