Introduction
A good start to learn about Service Fabric (SF) is here.
This article is the third article in a series of three and describes how to build an application with loose couplings using NServiceBus(NSB) and to host it in Azure Service Fabric. The first two articles are:
Article 1 describes how NSB could (but maybe shouldn’t) be used as a communication link between a client and a server. The problem with this solution is that it is very monolithic, i.e., strong couplings between client and server. As you probably know, loose couplings should be strived for when building software applications.
Article 2 describes how to port a Web API application to SF.
Using the Code
When the solution is downloaded, you need to update the connection strings with your Azure SQL Server and your Azure Storage Account. It will require some restarts of the application before the Azure Storage Account is initialized. Regarding SQL Server, EntityFramework will create the database and populate the tables for you. To allow the application to access your Azure SQL Server, you must update the Azure SQL Server Firewall with the IP address you use. There will be an exception where you can read which IP address you are using. Use it to update the Azure SQL Server firewall settings. The NSB license file is included in the application and valid until 2018-08-17.
Below is the code which has to be customized. It is located in project Shared file Helpers.cs.
public static string GetSqlConnection()
{
return "Server=tcp:sireusdbserver.database.windows.net,1433;
Initial Catalog=companyvehicles;Persist Security Info=False;User ID=xxx;Password=yyy;
MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;";
}
public static string GetAspNetDbConnection()
{
return "Server=tcp:sireusdbserver.database.windows.net,1433;
Initial Catalog=aspnetdb;Persist Security Info=False;User ID=xxx;Password=yyy;
MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;";
}
public static string GetStorageConnection()
{
return "DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=yyy;
EndpointSuffix=core.windows.net";
}
The databases aspnetdb and companyvehicles will be created automatically by EntityFramework if they do not already exist.
You can also deploy the application to Azure. Then in project Client file appsettings.json, the URL needs to be changed accordingly. Initially, appsettings.json looks like this and the WebAPI is located on localhost:83:
{
"ConnectionStrings": {
"ApiAddress": "http://localhost:83"
},
"Logging": {
"IncludeScopes": false,
"LogLevel": {
"Default": "Warning"
}
}
}
Background
This application, CompanyVehiclesFabric, has evolved during the first and second article and gone from a .NET Core Web API with a SQLite database to an application hosted in Azure Service Fabric based on Web API, NServiceBus, SQL Server, Vertical Slice Architecture, Azure Storage Queues and Azure Storage Presence.
There are many obstacles to overcome when learning SF and NSB. Some comes from misleading or missing documentation, other from bugs in samples or frameworks and some are due to incompatibility between .NET Core, NSB and SF.
Working with Azure Service Fabric
SF Pricing
SF is expensive. The smallest configuration with three virtual machines costs more than $500 per month. To minimize cost, I recommend programmers to create a SF cluster from Visual Studio, use it to test and develop and when finished, delete it from Azure. On the next working day, you can recreate the cluster when needed. This might be a difficult way to work but it will keep the cost for SF virtual machines under control.
SF Party Cluster
Another alternative for cost control is to use SF Party Cluster. The instructions for using party cluster can be improved, to say the least, but here are some tips for understanding the instruction. First, you need a github account. Then go to:
After having logged in via github, you arrive at a page where you are told to download a certificate. There, you read “Download the certificate using the PFX link”. But where is that PFX link? It’s on the previous page! After having downloaded the certificate, you must install it on your computer. The instruction says:
On Windows machine, install the PFX in CurrentUser\My certificate store.
However, a much easier way is to right click on the certificate and select “Install PFX”. Now you can use the Party Cluster for one hour, then you have to do the same procedure once again.
SF Stateless and Stateful Service
Company Vehicles Fabric application is a stateless application. In a stateful application, data is stored in Reliable Collections spread out over a number of partitions in the application. I prefer to use a traditional database, e.g. SQL Server, for storing data instead of storing data in reliable collections. The stateful applications samples I have seen appear to me as incredibly complicated regarding how data is stored and retrieved. Stateful applications can also be used as data cache together with an external database.
Vertical Slice Architecture
Vertical Slice Architecture is a way to achieve Command Query Responsibility Segregation, CQRS and thus looser couplings between application entities. Company Vehicles Fabric application implements CQRS by letting the WebAPI query the database directly via GET commands and do all PUT/POST/DELETE commands via NSB. Vertical Slice Architecture is described on page vertical slice architecture. In articles 1 and 2, there were two tables in the database, Cars and Companies where one Company can have many Cars. In order to convert this architecture to Vertical Slice Architecture, all updateable properties will get both read and write tables. This is illustrated with the following Car object which looks like this:
public class Car
{
public Guid CarId { get; set; }
public Guid CompanyId { get; set; }
public string CreationTime { get; set; }
public string VIN { get; set; }
public string RegNr { get; set; }
public bool Online { get; set; }
public bool Locked { get; set; }
public long LockedTimeStamp { get; set; }
public int Speed{ get; set; }
public int QueueLength { get; set; }
}
The first 4 properties are “readonly” and set during object creation. Properties “Online” and “Speed” can be updated by the application and are visible in the GUI. Property “Locked” is a flag set when a user is about to edit or delete a Car object. Locked is used to prevent other users from changing the object while the first user is editing it. To implement vertical slice architecture, the updateable properties (Online, Speed and Locked) gets their own read and write tables. The properties Address and Name in Company object can be updated and therefore gets read and write tables in database as shown in the picture below:
To really understand how this works, please refer to the code.
Company Vehicles Fabric Solution
The solution consists of the Company Vehicles Fabric application and the Client, please refer to the picture below:
To deploy the solution locally to your SF development cluster, right click CompanyVehiclesFabric and chose Publish. After deployment, right click on project Client and select Debug > Start new instance. The client is a .NET Core MVC web application communicating with the WebAPI in the Service Fabric cluster via http. Inside the cluster, the WebAPI communicates with Server via NSB and also directly with the database. The Server is connected to the SQL Server Database via the connection string and to WebAPI via NSB as shown below:
Graphical User Interface
This is the main with view showing a list of all companies with all cars. When the user has logged in, three buttons are available. The buttons can be used for:
- Showing a summary of all cars plus the length of the NSB message queue
- Arbitrarily updating speed and online on/off for all cars
- Reinitialize the database
Button "Automation Overdrive"
Opens a new window and starts sending speed and online/offline messages to the cars, one message per second. Use this button to see how the number of messages in queue to each car increases as more messages per second are sent.
Button "Show Message Queues"
This button opens a new window where the number of messages in queue to each car are shown. The code looks like this:
static async Task<Dictionary<string, int>> GetQueueLenghtForEachCar()
{
Dictionary<string, int> queueLengthPerCar = new Dictionary<string, int>();
CloudStorageAccount storageAccount = QueueStorage.Common.CreateStorageAccountFromConnectionString(Helpers.GetStorageConnection());
CloudQueueClient cloudQueueClient = storageAccount.CreateCloudQueueClient();
CloudQueue queue = cloudQueueClient.GetQueueReference(Helpers.ServerEndpoint);
IEnumerable<CloudQueueMessage> peekedMessages = await queue.PeekMessagesAsync(32);
if (peekedMessages != null)
{
foreach (var msg in peekedMessages.ToList())
{
var carId = GetCarIdFromMessage(msg);
if (CarIdAlreadyInQueue(queueLengthPerCar, carId))
{
queueLengthPerCar[carId]++;
}
else
{
queueLengthPerCar.Add(carId, 1);
}
}
}
return queueLengthPerCar;
}
To really understand how this works, please refer to the code.
Editing a Car
When a user wants to update speed or online status of a car, the car object is locked in the database. The lock order is sent via NSB and arrives at the database with a short delay. When the lock is stored in the database, the Save button is enabled and it is possible to save the changes. Before the lock is effectuated, the Save button is disabled. Directly after the Save has been effectuated, the database is updated but this will not be shown in the overview directly due to the delay in NSB. Instead, the text Update below header Pending is seen as shown below. If the editing takes longer than 40 seconds, the lock will be removed and editing will be disabled. The text Timeout will then be shown in the overview below header Pending.
NSB Transport and Persistence
In Company Vehicles Fabric, two types of transport queues are configured. The first transport queue is used for sending command messages and the second is a “priority” queue used for event messages with higher priority. In the application, there are altogether 15 command messages and 2 event messages.
Transport Queue for Priority Messages Implementing IEvent
The messages ClearDatabase and UpdateCarLockedStatus are event messages, i.e., implementing IEvent. The publish/subscribe pattern is used for the priority queue. Therefore, we need a transport and persistence that can handle multicast, is compatible with Net Core and can be run on Service Fabric. These requirements are fulfilled by AzureStoragePersistence and AzureStorageQueueTransport.
The event messages use the publish/subscribe pattern. However, in this application, there is only one subscriber listening for the published events.
Configuring AzureStoragePersistence for NSB Publish/Subscribe is a science of its own. In the Company Vehicles Fabric application persistence for the priority queue is configured like this:
var endpointConfiguration.UsePersistence<AzureStoragePersistence, StorageType.Subscriptions>()
.ConnectionString(Helpers.GetStorageConnection());
endpointConfiguration.UsePersistence<AzureStoragePersistence, StorageType.Timeouts>()
.ConnectionString(Helpers.GetStorageConnection())
.CreateSchema(true)
.TimeoutManagerDataTableName("TimeoutManagerPriority")
.TimeoutDataTableName("TimeoutDataPriority")
.CatchUpInterval(3600)
.PartitionKeyScope("2018052400");
var transport = endpointConfiguration.UseTransport<AzureStorageQueueTransport>()
.ConnectionString(Helpers.GetStorageConnection());
var routing = transport.Routing();
routing.RegisterPublisher(typeof(UpdateCarLockedStatus), "api-publisher");
routing.RegisterPublisher(typeof(ClearDatabase), "api-publisher");
Endpoint.Start(endpointConfiguration).GetAwaiter().GetResult(); ;
Transport Queue for Sending Messages Implementing ICommand
The command messages are sent to a specified endpoint via the nonpriority queue.
var endpointConfiguration.UsePersistence<AzureStoragePersistence>()
.ConnectionString(Helpers.GetStorageConnection());
var transport = endpointConfiguration.UseTransport<AzureStorageQueueTransport>()
.ConnectionString(Helpers.GetStorageConnection());
Endpoint.Start(endpointConfiguration).GetAwaiter().GetResult();
There are two ways to state the destination of each message:
transport.Routing().RouteToEndpoint(assembly: typeof(UpdateCarOnlineStatus).Assembly,
@namespace: "Shared.Messages.Commands", destination: "server-endpoint");
- In the controller, when sending the message:
await endpointInstance.Send("server-endpoint", updateCarOnlineStatus).ConfigureAwait(false);
To understand how this works, please refer to the code.
NSB Folders for License.xml and Diagnostic Logs
To run NSB, a license is needed. The license file License.xml is stored in folder PackageRoot/Data and read by NSB during startup. NSB also needs a folder for writing diagnostic logs and folder PackageRoot is chosen for this. Please refer to the code for details.
endpointConfiguration.SetDiagnosticsPath(Directory.GetParent(pathToData).ToString());
endpointConfiguration.LicensePath(pathToData + "\\License.xml");
Bug in NSB 7.0.0-rc0002
If your default culture in Visual Studio is "sv-SE" then in Main(), you have to write:
new ArgumentException();
Thread.CurrentThread.CurrentUICulture = new CultureInfo("en-US");
Bug in SF Sample
The application in article 2 is similar to the SF Voting sample found here.
However, this sample has a bug when run in Azure. The stateful Voting sample runs perfectly in a local cluster but in Azure with the cluster set up by Visual Studio using certificate security, the Voting sample will not run correctly. This is due to the bug which is corrected in article 2. (Voting uses reverse proxy http://localhost:19081 when it should use https://localhost:19081 in Azure.)
History
- 2018-06-06: First version
