Introduction
Windows Azure Storage Extensions is a .NET library aimed for managing and querying entities from Windows Azure Storage. It's built on top of the Windows Azure SDK v2.0, provides asynchronous interfaces (Task-based Asynchronous Pattern) and LINQ to Azure Table queries via TableSet context by using POCO entities.
Table of Contents
Background
Currently Windows Azure SDK v1.x with LINQ to Azure Table provider marked as obsolete. Its successor Windows Azure SDK v2.0 was written from scratch and has few breaking changes which includes new querying mechanism. It's implementation has following gaps.
Querying by using TableQuery
Now if we don't want to use legacy WCF Data Services we should construct our queries to Azure Tables by using ugly code like that:
string filter = TableQuery.GenerateFilterCondition(
"RowKey", QueryComparisons.GreaterThanOrEqual, "5");
TableQuery<TableEntity> query = new TableQuery<TableEntity>().Where(filter).Take(5);
var entities = myTable.ExecuteQuery(query);
Requirements to derive from TableEntity in POCO objects
For using POCO entities we should use TableEntity
base class:
public class SampleEntity : TableEntity
{
public int SampleProperty { get; set; }
}
Except that it grant us ITableEntity
interface with PartitionKey
, RowKey
, Timestamp
, and ETag
properties it uses slow Reflection for entities serialization / deserialization.
Low Abstraction Level
For me as a software engineer Azure Storage Library v2.0 looks too low level abstraction of Azure Tables REST API. For instance when I want to use DTO entities I must remember that PartitionKey
is a user identifier, actually it's not meaningful and we should use some kind of entity mappers. Querying also is a too complicated task.
Windows Azure Storage Extensions
Windows Azure Storage Extensions library offering following features.
POCO
Entity's properties and fields should be marked by one or both of PartitionKey and RowKey attributes for defining composite table key. Also can be used Timestamp, ETag, Property and Ignore attributes.
Attribute | Description | Constraint |
---|
PartitionKey | Defines a partition key property. | Only for string properties. |
RowKey | Defines a row key property. | Only for string properties. |
ETag | Defines an etag property. | Only for string properties. |
Timestamp | Used for receiving table entity timestamp. | Only for DateTime or DateTimeOffset properties. |
Property | Used for defining custom table property name via Name property. | |
Ignore | Used for skipping properties from serialization / deserialization. | |
Table Entities Management
Generic TableSet
context provides both synchronous & asynchronous (TAP) methods for managing entities:
- Synchronous:
Add
, AddOrUpdate
, Update
, and Remove
. - Asynchronous:
AddAsync
, AddOrUpdateAsync
, UpdateAsync
, and RemoveAsync
.
To avoid restrictions of group operations in Azure Storage all entities sorted by partition keys and merged into groups by 100 entities. Execution of requests with such batch operations can be configured via TableSet
's ExecutionMode
property. Allowed values:
Default ExecutionMode
is Sequential
.
LINQ to Azure Tables
TableSet
context implements IQueryable interface for using LINQ Expressions. Provider supports next synchronous LINQ methods:
- First
- FirstOrDefault
- Single
- SingleOrDefault
- Take
- Where
To utilize filtering capabilities of string properties it supports following methods:
Also you can use Contains method. In this case query statement for each collection's item will be joined by using OData or
operator.
Note: For creating a custom queries you should take a look at next article Mixing LINQ Providers and LINQ to Objects.
Asynchronous LINQ Queries
In addition TableSet
can be used for asynchronous queries powered by LINQ extensions (TAP) in EF 6 Async style. Available methods:
FirstAsync
FirstOrDefaultAsync
SingleAsync
SingleOrDefaultAsync
TakeAsync
ToListAsync
LINQ Projections
LINQ Projections supported with a limitation - projection class should be a reference type.
Task-based Extension Methods
Library contains TAP-based extensions for a following Windows Azure SDK classes:
CloudBlobClient
CloudBlobContainer
CloudTableClient
CloudTable
To use it just add Async postfix to synchronous method name like that:
blobs = cloudBlobContainer.ListBlobs();
blobs = await cloudBlobContainer.ListBlobsAsync();
Task Cancellation
All of TAP-based methods accepts optional CancellationToken
parameter for Task Cancellation.
Download
Via NuGet
To install library by using Windows Azure Storage Extensions nuget package execute following command:
PM> Install-Package WindowsAzure.StorageExtensions
Via Git
To get the source code of the library via git just type:
git clone git://github.com/dtretyakov/WindowsAzure.git
cd ./WindowsAzure
Example
You can download SPA example powered by ASP.NET MVC4 & Web API. It uses Azure Table Storage as a persistence, so before you run that example start Azure Storage Emulator or configure proper connection string in the Web.config.
One of the interesting things in this example is a Get method in the IssuesController
. At runtime SPA sends an OData queries to WebAPI IssuesController
which by leveraging Queryable
attribute transforms into LINQ queries to the TableSet
context which in turn translates LINQ expressions into OData filters for Azure Storage Table service.
Code Samples
Defining a new class:
public sealed class Country
{
[PartitionKey]
public string Continent { get; set; }
[RowKey]
public string Name { get; set; }
public long Population { get; set; }
public double Area { get; set; }
public DateTime Formed { get; set; }
}
Creating a new table context:
var storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
var tableClient = storageAccount.CreateCloudTableClient();
var countryTable = new TableSet<Country>(tableClient);
Adding a new entities:
var resultSync = countryTable.Add(country);
var resultAsync = await countryTable.AddAsync(country);
Updating an entities:
resultSync.Area += 333333;
resultSync = countryTable.Update(resultSync);
resultAsync.Population *= 2;
resultAsync = await countryTable.UpdateAsync(resultAsync);
Removing an entities:
countryTable.Remove(resultSync);
await countryTable.RemoveAsync(resultAsync);
Querying entities:
var query = countryTable.Where(
p => p.Formed > new DateTime(1950, 1, 1) &&
(p.PresidentsCount < 10 || p.Population < 10000000 && p.IsExists));
var result = query.ToList();
result = await query.ToListAsync();
or using string filtering methods:
var countryStartingWithF = await countryTable.FirstAsync(
p => p.Name.CompareTo("F") >= 0 && p.Name.CompareTo("G") < 0);
Using LINQ projections:
var projection = from country in countryTable
where country.Area > 400000
select new { country.Continent, country.Name };
var entities = projection.ToList();
entities = await projection.ToListAsync();
Using Contains method in the LINQ query:
var countryNames = new List<string> { "Germany", "Finland" };
var countries = countryTable.Where(p => countryNames.Contains(p.Name)).ToList();
Points of Interest
Source code of the library are available at the github.com/dtretyakov/windowsazure. Core components of the Windows Azure Storage Extensions are following.
Table Context
TableSet<T>
class is an abstraction over Azure Storage Table. It implements IQueryable<T>
and ITableSet<T>
interfaces. Internals in actions it performs single and batch requests to the Azure Storage Tables by using CloudTable
class and uses POCO object serialization by leveraging TableEntityConverter<T>
.
Table Entity Converter
TableEntityConverter<T>
implements ITableEntityConverter<T>
interface. It uses compiled Expression Trees for fastest access to POCO fields and properties. Internally it converts POCO objects into DynamicTableEntity
and back. Performance comparison results in ms of TableEntityConverter
for POCO, TableEntity
and new
operator are following:
Conversion | 10M | 50M |
---|
DynamicTableEntity to Object | 8501 | 43,690 |
DynamicTableEntity to POCO | 17,826 | 89,907 |
DynamicTableEntity to TableEntity | 54,158 | 276,421 |
Object to DynamicTableEntity | 10,693 | 51,485 |
POCO to DynamicTableEntity | 15,528 | 78,368 |
TableEntity to DynamicTableEntity | 41,471 | 218,935 |
Tests can be found in: EntityConverterTests.cs
Table Query Provider
TableQueryProvider<T>
implements IQueryProvider
and IAsyncQueryProvider
interfaces. It provides translation of LINQ Expressions into Azure Storage Table OData filters and able to execute it synchronously and asynchronously.
Task-based Extension Methods
Windows Azure SDK v2.0 provides us APM-based interfaces and ICancellableAsyncResult
for operation cancellation. .NET 4.0 and higher uses other TAP-based asynchronous programming model. Thus Windows Azure Storage Extensions provides APM to TAP wrappers for most of Windows Azure SDK classes. Library internally uses these extensions for executing asynchronous requests to Azure Storage Tables.
Feedback
Your suggestions and comments are very welcome at the GitHub project page.
History
- 0.1.0 - December 16, 2012: Alpha version.
- 0.7.0 - April 12, 2013: First stable version.
- 0.7.2 - April 22, 2013: Added LINQ projections, entities partitioning.
- 0.7.3 - April 27, 2013: Serialization performance improvements,
ITableSet
's methods provides IEnumerable
again. - 0.7.4 - May 10, 2013: Added Contains method; performance optimization of the LINQ to OData translator.
- 0.7.6 - May 20, 2013: Performance optimizations and code stabilization.
- 0.7.7 - August 10, 2013: Code stabilization.