CQRS - Hosting an Event Sourcing System on Microsoft Orleans

Introduction

I have written, in the past, about the Command Query Responsibility Segregation (CQRS) architecture and how the data storage/processing idea of Event Sourcing fits well within it. However, anyone choosing this route for their applications also has to make a decision as to where and how it is hosted.

In a purely monolithic application, my recommendation would be to bake it into the code in a very similar fashion to the way MVC and MVVM architectures are baked in to their host applications but once you go to microservices or even serverless functions for your system, you will need to look at hosting the CQRS/ES system external to these.

One candidate to do this on is Microsoft Orleans, which is a framework that aims massively to simplify the creation of fault tolerant, asynchronous distributed systems by abstracting away the complexities of persistence and thread synchronization that arise in such systems. It does this by using the actor model (albeit calling the actors "Grains") and by restricting the operations you can perform on them so as to make them thread safe.

Background

If you are new to the CQRS architecture and event sourcing, I recommend reading the above articles or if you have 45 minutes to spare, there is also this YouTube video.

Prerequisites

• The Orleans Tools for Visual Studio plug in was used to create the project
• Visual Studio 2015 or 2017 (professional or higher if you want to use the plug-in)
• A CQRS model code generated from the CQRS designer

Design Decisions

The first question is should each projection be available independently or should the aggregate grain contain all of its projections?

For example - in the above CQRS domain, should the Running Balance projection be a grain in its own right, or should it be an aspect of the Bank Account grain?

One of the criticisms of CQRS/ES is the need to fully rehydrate the aggregate in order to use any aspect of it so for the purposes of this article, I will have each projection be its own grain. However, this design decision does depend on your business model so you should consider either approach.

Creating the Orleans "grains"

The first step is to create a library that wraps the business classes (from the CQRS designer) in the Orleans interfaces so that they can be hosted by it. Using the Orelans Tools plug in create a new interfaces project:

Then, within that project, add a reference to the .EventSourcing project created by the code generation from your CQRS modelling domain.

Wrapping the Aggregates

Each aggregate in your CQRS domain has to be wrapped with the grain interface that uses the same data type for the unique identifier - for example, the "Account" aggregate in the bank example has a string (account number) that uniquely identifiers it so we wrap it in a class that also inherits from IGrainWithStringKey.

Then, in the interface, you need to define a task that handles each of the event types for the aggregate, including a parameter for the sequence number (to prevent an event being processed twice):

/// <summary>
/// Grain interface IAccountGrain for the Account aggregate
/// </summary>
public interface IAccountGrain :
    IGrainWithStringKey ,
    Accounts.Account.IAccount
{
    /// <summary>
    /// The account was opened
    /// </summary>
    Task HandleOpenedEvent(int eventSequence,
           Accounts.Account.eventDefinition.IOpened eventData);

    /// <summary>
    /// The account was closed
    /// </summary>
    Task HandleClosedEvent(int eventSequence,
           Accounts.Account.eventDefinition.IClosed eventData);

    /// <summary>
    /// Money was paid into the account
    /// </summary>
    Task HandleMoneyDepositedEvent(int eventSequence,
            Accounts.Account.eventDefinition.IMoney_Deposited eventData);

    /// <summary>
    /// Money was withdrawn from this account
    /// </summary>
    Task HandleMoneyWithdrawnEvent(int eventSequence,
            Accounts.Account.eventDefinition.IMoney_Withdrawn eventData);
}

As we are implementing the projections separate to the aggregate class, we would also need a grain interface for them with only those event types that the projection cares about:

public interface IRunningBalanceGrain
    : IGrainWithStringKey ,
    Accounts.Account.projection.IRunning_Balance
{

    /// <summary>
    /// Money was paid into the account
    /// </summary>
    Task HandleMoneyDepositedEvent(int eventSequence,
           Accounts.Account.eventDefinition.IMoney_Deposited eventData);

    /// <summary>
    /// Money was withdrawn from this account
    /// </summary>
    Task HandleMoneyWithdrawnEvent(int eventSequence,
           Accounts.Account.eventDefinition.IMoney_Withdrawn eventData);

    /// <summary>
    /// The date/time of the last transaction against this account
    /// </summary>
    Task<DateTime> GetLastTransactionDate();

    /// <summary>
    /// The current running balance for the account
    /// </summary>
    Task<decimal> GetBalance();
}

We also add tasks that can be used to retrieve the current state of the projection. The business class created by the CQRS designer will perform the actual projection logic but we need this additional set of functions so as to allow it to be queried over the Orleans infrastructure.

As all Orleans grain classes (the concrete implementations of the above interfaces) must inherit from the base class Grain and multiple-inheritance is not possible, we connect the CQRS designer derived classes into their Orleans grain wrapper using a private instance of the class:

   public class AccountGrain :
       Grain, IAccountGrain
   {
       /// <summary>
       /// Private link to the CQRS-DSL created account instance
       /// </summary>
       private Accounts.Account.Account _account;

       public string GetAggregateIdentifier()
       {
           if (null != _account )
           {
               return _account.GetAggregateIdentifier();
           }
           else
           {
               throw new NullReferenceException("Account instance not initialised");
           }
       }

// - - 8< - - - - - - -
   }

This also applies to the concrete implementation of the projection wrapper:

    public class RunningBalanceGrain
        : Grain, IRunningBalanceGrain
    {
        /// <summary>
        /// Private link to the CQRS-DSL created running balance projection instance
        /// </summary>
        private Accounts.Account.projection.Running_Balance _runningBalance;

        /// <summary>
        /// The date/time of the last balance affecting transaction
        /// </summary>
        public DateTime Last_Transaction_Date
        {
            get
            {
                if (null != _runningBalance )
                {
                    return _runningBalance.Last_Transaction_Date;
                }
                else
                {
                    throw new NullReferenceException
                    ("Running balance projection instance not initialised");
                }
            }
        }
 // - - - 8< - - - - - - -
}

This does require a bit of extra code to wire-up the CQRS class into the Orleans wrapper but it does still maintain the separation between business logic (which comes from the CQRS designer class) and implementation logic (which is provided by Orleans).

In order to synchronize the CQRS-DSL provided business class with the grain it is being hosted in, we need to instantiate it when the grain is created. In Orleans, you can override the OnActivateAsync method to do this:

public class AccountGrain :
    Grain, IAccountGrain
{
    /// <summary>
    /// Private link to the CQRS-DSL created account instance
    /// </summary>
    private Account _account;
    private void InitialiseAccount()
    {
        if (null == _account )
        {
            _account = new Account(this.GetPrimaryKeyString ());
        }
    }

    public override Task OnActivateAsync()
    {
        InitialiseAccount();
        return base.OnActivateAsync();
    }

 // - - 8< - - - - - -
}

Creating a Silo

Instances of these grains (entities) need to be hosted by a silo, which is effectively a virtual machine environment for that grain. This, combined with the way that Orleans allows for cluster management means that you can rapidly spin up a truly distributed CQRS/ES application.

Points of Interest

• This is by no means the definitive way to do CQRS on Microsoft Orleans - I would also recommend looking at the OrleansAkka project - especially if using F#
• There are a number of storage providers you can use to persist the grains between uses including the various Azure cloud storage options.

History

• 20^th May, 2017 - Initial version (I may add identifier groups)