Introduction
This article does not attempt to delve into too much theory. The purpose of this article is to present working examples of WF 4.0 Persistence, Tracking, and Bookmarks.
If you have followed any of the previous articles, you will know I usually dedicate much writing time to explain the theory before the practice. Well here, I found it does not make much sense for a single reason: the theory of the topics this article discusses is covered very well in many resources and books, but I found - at the time of this writing - that practical useful examples are not easy to find. So I decided to write simple but sufficient examples to show the features in action. I hope my attempt will benefit as much people as possible.
WF 4.0 Persistence
The first step is setting up the new persistence databases. Go to "C:\Windows\Microsoft.NET\Framework\v4.0.30319\SQL\en" and there you will find two files:
- SqlWorkflowInstanceStoreSchema.sql
- SqlWorkflowInstanceStoreLogic.sql
In SQL Server (Express or above), create a database with a convenient name (my database is called "SqlWorkflowInstanceStore") and run the above scripts to create the required objects.
The persistence model in WF 4 is different than that of WF 3.5. The new model is now called InstanceStore instead of PersistenceProvider. For WF 3.5, the PersistenceProvider databases are still available in the same location:
- SqlPersistenceProviderLogic.sql
- SqlPersistenceProviderSchema.sql
In WF 3.5, there was a service for managing WFs such as waking up workflow instances... the SQL files for this service are still available in the same location:
- SqlPersistenceService_Logic.sql
- SqlPersistenceService_Schema.sql
Tracking will be covered next, but just for the sake of completion, in 4.0, there is no tracking participant that writes to a SQL. Tracking_Schema.sql applies to 3.5 WFs. Out of the box, with WF 4.0, there is a tracking participant that writes to ETW (Event Tracing for Windows). Again, those files for WF 3.5 are:
- Tracking_Schema.sql
- Tracking_Logic.sql
Building the WF
Create a new Console Workflow Application, and use the default Workflow1.xaml to build the following simple workflow:
The Receive and SendReplyToReceive shapes receive an empty request and return the workflow instance ID to the caller. CodeAcitivty1 is a custom activity that retrieves the workflow instance ID. Shown below is the code for this activity:
public sealed class CodeActivity1 : CodeActivity
{
public OutArgument<Guid> WFInstanceId { get; set; }
protected override void Execute(CodeActivityContext context)
{
context.SetValue(WFInstanceId, context.WorkflowInstanceId);
}
}
The important catch is to set the property PersistBeforeSend on the SendReplyToReceive shape. This will cause the WF to persist just before sending the response to the caller. This means that now the WF will have a point to go back to in case the WF is suspended or unloaded... more on that later.
The Delay activity is used to delay the WF for 1 minute...we will soon see why that is required.
Now, go to the Program.cs file and let's write the code required to host the WF and set up Persistence.
Use the following code and then I will explain it:
class Program
{
static void Main(string[] args)
{
string baseAddress = "http://localhost:8089/TestWF";
using (WorkflowServiceHost host =
new WorkflowServiceHost(new Workflow1(), new Uri(baseAddress)))
{
host.Description.Behaviors.Add(new
ServiceMetadataBehavior() { HttpGetEnabled = true });
host.AddServiceEndpoint("IService", new BasicHttpBinding(), baseAddress);
SqlWorkflowInstanceStore instanceStore = new SqlWorkflowInstanceStore(
@"Data Source=.\SQL2008;Initial Catalog=" +
@"SqlWorkflowInstanceStore;Integrated Security=True");
host.DurableInstancingOptions.InstanceStore = instanceStore;
host.Open();
Console.WriteLine("Car rental service listening at: " +
baseAddress);
Console.WriteLine("Press ENTER to exit");
Console.ReadLine();
}
}
}
By default, WorkflowInvoker is used to invoke the WF. I changed this into WorkflowServiceHost, the reason is that WorkflowInvoker cannot be configured with Persistence. WorkflowServiceHost is used to host WF Services. The other method of calling (hosting) workflows is WorkflowApplication which - like WorkflowServiceHost - can be used to host long running asynch workflows with extensions (Persistence, Tracking); however, only for workflows that are non services.
After creating my WCF endpoint and adding it to the host, I use the SqlWorkflowInstaceStore class to configure my persistence store, which is the same store I prepared when I first started.
Creating the Client
Run the WF so that the host is running and the WCF endpoint is listening for requests on "http://localhost:8089/TestWF".
Now create a client (console) application and add a service reference to the above URL. Write the following code to invoke the service operation:
ServiceReference1.ServiceClient proxy = new ServiceReference1.ServiceClient();
Guid workflowId = (Guid)proxy.Operation1();
Console.WriteLine("client done");
Console.WriteLine("enter key");
Console.ReadLine();
Run the client and observe the scenario: once the operation is invoked (and the workflow instance ID is returned), open the table "InstancesTable" and you will see a record for a persisted WF. Why is that? Recall that we set a property on the SendReply shape to persist just before sending the reply. For this, the WF has persisted state in the database, but is still running as we did not suspend or unload the WF. See images below:
For how much time will this record appear in the database? For 1 minute, because we have a Delay shape in the WF to delay completion for 1 minute, after which the WF execution will complete and the record will disappear from the database.
Next, I will add support (behavior) to persist and unload the WF as well as add control endpoint to execute remote operations on the WF.
More on Persistence and Control Endpoints
So to recap, so far I configured the Persistence store for WF 4.0 workflow services hosted by WorkflowServiceHost. I showed how the workflow is persisted just before sending a response to a client application and observed the persisted state in the database.
Now I will add a host behavior to persist the workflow based on an idle time as opposed to before sending the response from the workflow itself; moreover, I will also add a host behavior to unload the workflow after a specified time interval, thus showing the difference between just persisting a workflow and unloading it from memory.
Let's start by unchecking (disabling) the property PersistBeforeSend of the SendReplyToReceive shape. Next, add the following code to the Program.cs file of the workflow application just before the "host.Open()" statement:
host.Description.Behaviors.Add(new WorkflowIdleBehavior()
{
TimeToPersist = TimeSpan.FromSeconds(5),
TimeToUnload = TimeSpan.FromSeconds(20)
});
This code adds a behavior to persist the WF instance after 5 seconds of inactivity and to unload it completely after 20 seconds of inactivity. What's the difference? Persistence - as explained previously - persists the workflow state in the database, but keeps the workflow instance running. Unloading, however, does two things: it persists the WF state and also unloads it from memory; typically for a long running process.
Build the WF and run it. It's ready to listen to requests on its endpoint.
Now run the client. Once the service sends the response back to the client (recall - no persistence on send here), it enters a delay shape for 1 minute. Meanwhile, we have configured to persist the WF after 5 seconds. So wait 5 seconds and open the table "InstancesTable"; just like in Part 1, you will see the WF state persisted:
However, now we have also configured the WF to unload after 20 seconds. So after an additional 15 seconds, this time open the table "RunnableInstancesTable" and you will see a record corresponding to the same workflow. Why the "RunnableInstancesTable" table? Because, this time the WF is not only persisted, but is also unloaded, meaning that it is ready to be loaded again and continue execution:
After the delay shape finishes, the WF instance will be loaded again into memory and finishes execution. Both records will disappear from the database.
Finally, let's add a control endpoint to our WF. Control endpoints allow sending commands to a WF instance from a client. Add the below code in the host application:
WorkflowControlEndpoint controlEndpoint =
new WorkflowControlEndpoint(
new BasicHttpBinding(),
new EndpointAddress(new Uri(baseAddress) + "/wce")
);
host.AddServiceEndpoint(controlEndpoint);
This code adds a special type of endpoint called the control endpoint, with the following address: "http://localhost:8089/TestWF/wce".
Run the service and update the service reference at the client application. Add the below code at the client application just after invoking the service operation:
System.Threading.Thread.Sleep(new TimeSpan(0, 0, 30));
WorkflowControlEndpoint ep = new WorkflowControlEndpoint(new BasicHttpBinding(),
new EndpointAddress("http://localhost:8089/TestWF/wce"));
WorkflowControlClient client = new WorkflowControlClient(ep);
client.Terminate(workflowId);
This code creates a workflow control client and uses the workflow instance ID to send a command to terminate the WF instance. To test this, I have caused the thread to sleep for 30 seconds. This way, I am sure that at this point, the WF instance has been unloaded (review the WF configuration). Now, when I send the Terminate command, notice how the WF instance does not continue execution and is terminated. To be sure, see how the database persistence records are deleted after just 30 seconds, as opposed to the 1 minute as explained previously.
WF 4.0 Tracking
In this section, I will do some modifications on the same example to add tracking support.
Below is our simple workflow (Workflow1.xaml):
It receives a request from a client application. Using a custom activity, it retrieves the workflow instance ID and returns it as a response to the client. Finally, a Delay shape delays the execution for 30 seconds.
Below is the code for the custom activity:
public sealed class CodeActivity1 : CodeActivity
{
public OutArgument<Guid> WFInstanceId { get; set; }
protected override void Execute(CodeActivityContext context)
{
context.SetValue(WFInstanceId, context.WorkflowInstanceId);
CustomTrackingRecord customRecord = new CustomTrackingRecord("CustomInfo")
{
Data =
{
{"Date", DateTime.Now.ToShortDateString()},
}
};
context.Track(customRecord);
}
}
The custom activity returns the workflow instance ID and creates a new CustomTrackingRecord. CustomTrackingRecords are any custom information you want to be able to track. In this simple example, I am tracking a custom variable which stores the current date... more on this in a moment.
Below is the code from the Program.cs file needed to set up the host and tracking:
class Program
{
static void Main(string[] args)
{
string baseAddress = "http://localhost:8089/TestWF";
using (WorkflowServiceHost host =
new WorkflowServiceHost(new Workflow1(), new Uri(baseAddress)))
{
host.Description.Behaviors.Add(new
ServiceMetadataBehavior() { HttpGetEnabled = true });
host.Description.Behaviors.Add(
new ServiceBehaviorAttribute() { IncludeExceptionDetailInFaults = true });
host.AddServiceEndpoint("IService", new BasicHttpBinding(), baseAddress);
TrackingProfile fileTrackingProfile = new TrackingProfile();
fileTrackingProfile.Queries.Add(new WorkflowInstanceQuery
{
States = { "*" }
});
fileTrackingProfile.Queries.Add(new ActivityStateQuery()
{
ActivityName = "*",
States = { "*" },
Variables =
{
{ "*" }
}
});
fileTrackingProfile.Queries.Add(new CustomTrackingQuery()
{
ActivityName = "*",
Name = "*"
});
FileTrackingParticipant fileTrackingParticipant =
new FileTrackingParticipant();
fileTrackingParticipant.TrackingProfile = fileTrackingProfile;
host.WorkflowExtensions.Add(fileTrackingParticipant);
host.Description.Behaviors.Add(new WorkflowIdleBehavior()
{
TimeToPersist = TimeSpan.FromSeconds(5),
TimeToUnload = TimeSpan.FromSeconds(20)
});
host.Open();
Console.WriteLine("Car rental service listening at: " +
baseAddress);
Console.WriteLine("Press ENTER to exit");
Console.ReadLine();
}
}
}
public class FileTrackingParticipant : TrackingParticipant
{
string fileName;
protected override void Track(TrackingRecord record,
TimeSpan timeout)
{
fileName = @"c:\tracking\" + record.InstanceId + ".tracking";
using (StreamWriter sw = File.AppendText(fileName))
{
WorkflowInstanceRecord workflowInstanceRecord =
record as WorkflowInstanceRecord;
if (workflowInstanceRecord != null)
{
sw.WriteLine("------WorkflowInstanceRecord------");
sw.WriteLine("Workflow InstanceID: {0} Workflow instance state: {1}",
record.InstanceId, workflowInstanceRecord.State);
sw.WriteLine("\n");
}
ActivityStateRecord activityStateRecord = record as ActivityStateRecord;
if (activityStateRecord != null)
{
IDictionary<string,object /> variables = activityStateRecord.Variables;
StringBuilder vars = new StringBuilder();
if (variables.Count > 0)
{
vars.AppendLine("\n\tVariables:");
foreach (KeyValuePair<string,object /> variable in variables)
{
vars.AppendLine(String.Format(
"\t\tName: {0} Value: {1}", variable.Key, variable.Value));
}
}
sw.WriteLine("------ActivityStateRecord------");
sw.WriteLine("Activity DisplayName: {0} :ActivityInstanceState: {1} {2}",
activityStateRecord.Activity.Name, activityStateRecord.State,
((variables.Count > 0) ? vars.ToString() : String.Empty));
sw.WriteLine("\n");
}
CustomTrackingRecord customTrackingRecord = record as CustomTrackingRecord;
if ((customTrackingRecord != null) && (customTrackingRecord.Data.Count > 0))
{
sw.WriteLine("------CustomTrackingRecord------");
sw.WriteLine("\n\tUser Data:");
foreach (string data in customTrackingRecord.Data.Keys)
{
sw.WriteLine(" \t\t {0} : {1}", data, customTrackingRecord.Data[data]);
}
sw.WriteLine("\n");
}
}
}
}
Let me explain what's going on: in a nutshell, when you deal with WF 4.0 Tracking, you have to understand three concepts:
- Tracking records: these are the information records emitted by your workflow. There are four derived classes that correspond to the four types of tracking records:
WorkflowInstanceQuery: events about workflow instance state. For example: started, suspended, unloaded, etc...ActivityStateQuery: events about activities inside the WF.CustomTrackingQuery: any custom information you want to track.BookmarkResumptionQuery: bookmark name you want to track whenever this bookmark is resumed (bookmarks are covered next).
- Tracking profiles: act like filters over tracking records in order to track only the required information.
- Tracking participants: the medium where tracking information will be written to. WF 4.0; comes with one participant which writes to ETW (Event Tracking for Windows). Custom participants can be easily developed to write tracking data into SQL Server, for example, or like I did in this example, into the File System.
In my code, I create an instance of the class TrackingProfile and added three types of records: WorkflowInstanceQuery, ActivityStateQuery, and CustomTrackingQuery.
For WorkflowInstanceQuery, I asked the profile to track all workflow states using the "*" operator. I could also limit the number of states that I wanted to track (check commented code). Recall that in my workflow, I have states of persist and unload (review behavior configuration and delay timespan) so you will see these states in the tracked data. For ActivityStateQuery, I asked the profile to track all states for all activities as well as any variables declared for these activities. Similarly, for CustomTrackingQuery, I asked the profile to track any custom data I defined (recall that I did define custom data in my custom activity).
Now that I have defined the tracking records as well as the tracking profile, the final step is to define the tracking participant. I could have used the out of the box EtwTrackingParticipant class which writes to ETW, but to make things more interesting, I have created a custom tracking participant class (FileTrackingParticipant) which derives from TrackingParticipant. Check the code for that class which simply gets the information for each of the tracked records and writes it into a file.
Back to the main program, I associate my tracking profile (fileTrackingProfile) with my tracking participant (fileTrackingParticipant) and finally add the participant to the set of the host WorkflowExtensions.
To make sure you will have a running program, below is the trivial client program code:
static void Main(string[] args)
{
ServiceReference1.ServiceClient proxy = new ServiceReference1.ServiceClient();
Guid instanceid = (Guid)proxy.Operation1();
Console.WriteLine("Client Done");
Console.ReadLine();
}
Run the service so that it is listening to requests on "http://localhost:8089/TestWF"; now run the client console and wait until the service console window prints "Workflow Ended". Now examine the file where tracking is written, and you will see workflow events, activity events, variables, and custom information, all tracked... see the below image (reformatted for presentation):
WF 4.0 Bookmarks
Bookmarks are used to mark places in your workflow where you want it to wait for something to happen (like getting an input). Here, you want the WF to stop execution and release the working thread. So a bookmark is a resumption point created and given a name inside your code activity; a separate code (typically the host) knows about this named point and will, at sometime, use it to pass in data and resume the WF. As you might have guessed, we will also configure Persistence for the WF instance to be correctly persisted and resumed using the bookmark...
One more time, let's create a workflow console application, and using the generated file "Workflow1.xaml", build the following flowchart workflow:
It's simple: once the WF starts, it prints the thread ID of the instance, then it displays a message prompting the user to enter a product name; then a custom activity called "SubmitOrderName" is executed. This activity creates a bookmark where the WF will be waiting for the user to input the product name. Next - after the WF is resumed - again the executing thread ID is printed, and finally the message "Workflow Ended" is printed.
Let's take a look at the SubmitOrderName activity:
public sealed class SubmitOrderName : NativeActivity
{
protected override bool CanInduceIdle
{
get
{
return true;
}
}
protected override void Execute(NativeActivityContext context)
{
context.CreateBookmark("OrderNameBookmark",
new BookmarkCallback(OnBookmarkCallback));
}
void OnBookmarkCallback(NativeActivityContext context,
Bookmark bookmark, object val)
{
Console.WriteLine("Order Name is {0}", (string)val);
}
}
It derives from NativeActivity and overrides the property CanInduceIdle to return the true meaning that this activity can cause the workflow to become idle. The Execute method creates the bookmark and gives it the name "OrderNameBookmark" and also defines the bookmark callback - which is the code to be executed when the bookmark is resumed (by the host, as we will see in a moment).
Finally, the bookmark callback method is defined, and it simply displays the user entry into the console.
Now the complete scenario will be clear once we investigate the code of the host. Open Program.cs and observe the code. I will explain it in chunks to make it easier to grasp. The first part is shown below:
static AutoResetEvent instanceUnloaded = new AutoResetEvent(false);
static Guid id;
AutoResetEvent allows threads that need access to a resource to communicate with each other by signaling. A thread waits for a signal by calling WaitOne on the AutoResetEvent. If the AutoResetEvent is the non-signaled state (a thread becomes in signaled state by calling Set on AutoResetEvent), the thread blocks waiting for the thread that currently controls the resource to signal that the resource is available by calling Set.
So why do we need this thread management in our example? Simply because in WF, the host (WorkflowApplication, in this example) is working on a thread while the workflow instance will be kicked on another thread (this is why I am printing the thread IDs in the example...). So how do we coordinate work between the host and the instance itself when it comes to creating the bookmark and then switching control back to the host to submit the input and resume the bookmark? The answer is using the AutoResetEvent... if this is not clear yet, do not worry, I will explain this thoroughly in a moment...
static void Main(string[] args)
{
WorkflowApplication app = new WorkflowApplication(new Workflow1());
InstanceStore store = new SqlWorkflowInstanceStore(
@"Data Source=.\SQL2008;Initial Catalog=" +
@"SqlWorkflowInstanceStore;Integrated Security=True");
InstanceHandle handle = store.CreateInstanceHandle();
InstanceView view = store.Execute(handle,
new CreateWorkflowOwnerCommand(), TimeSpan.FromSeconds(30));
handle.Free();
store.DefaultInstanceOwner = view.InstanceOwner;
app.InstanceStore = store;
Next, we use the WorkflowApplication to host our workflow. In the previous examples, I used the WorkflowServiceHost to host my WF services; here, I am using WorkflowApplication since I am not using the communication shapes.
Next I set up the persistence store and attach it to my application host.
app.PersistableIdle = (e) =>
{
return PersistableIdleAction.Unload;
};
app.Unloaded = (workflowApplicationEventArgs) =>
{
Console.WriteLine("WorkflowApplication has Unloaded\n");
instanceUnloaded.Set();
};
Next, I set up two properties of my application host: PersistableIdle and Unloaded.
PersistableIdle sets the action to be invoked when the workflow instance is idle and can be persisted. Possible actions are one of the PersistableIdleAction enumeration values of None (nothing happens), Persist (persist but keep instance executing), or Unload (persist and unload the instance from memory). In my case, I specified that the instance is to be unloaded.
Note: The PersistableIdle function is invoked when the scheduler has no more pending work items and the workflow runtime can persist. In our example, this will happen once we create the bookmark, as we will see in a moment.
Unloaded sets the action to be invoked once my workflow instance hits the unloaded lifecycle event. Recall that we have just instructed our workflow to be unloaded using the PersistableIdle property. Well here, we say what code will be executed once the instance is unloaded. We simply print a message to the console and then call Set on the AutoResetEvent: this means that the thread executing my workflow has signaled that it's free and control can be given to another thread (host thread in this case).
id = app.Id;
app.Run();
Console.WriteLine("Host thread: " +
System.Threading.Thread.CurrentThread.ManagedThreadId.ToString());
instanceUnloaded.WaitOne();
Next, I extract the GUID of my workflow - which I will later use for resumption of the workflow - and then call Run on my host to start workflow execution. But wait, there is a catch here: even though I called Run, the workflow won't start execution until I call WaitOne() on AutoResetEvent. Why? Because this signals that the host thread needs control and it is now waiting to get it (when AutoResetEvent is in the signaled state).
string name = Console.ReadLine();
app = new WorkflowApplication(new Workflow1());
app.InstanceStore = store;
app.Completed = (workflowApplicationCompletedEventArgs) =>
{
Console.WriteLine("\nWorkflowApplication has Completed in the {0} state.",
workflowApplicationCompletedEventArgs.CompletionState);
};
app.Unloaded = (workflowApplicationEventArgs) =>
{
Console.WriteLine("WorkflowApplication has Unloaded\n");
instanceUnloaded.Set();
};
app.Load(id);
app.ResumeBookmark("OrderNameBookmark", name);
Console.WriteLine("Host thread: " +
System.Threading.Thread.CurrentThread.ManagedThreadId.ToString());
instanceUnloaded.WaitOne();
Console.ReadLine();
The final piece of code is that of the workflow resumption. As we will see, when we run the workflow, like explained in the previous code, at this stage, the workflow has executed and the activity that creates the bookmark is executed, and the workflow enters an idle state waiting for user input. So the workflow is unloaded from memory and host thread gets control again.
So now the host is waiting for the user input. Once it gets it, we re-initialize our workflow's host and persistence store. Next, we set its Completed and Unloaded actions. Note that in the Unloaded action, we again call Set on the AutoResetEvent meaning that the thread has completed its job...
Finally, we actually load the instance using the correct ID and actually resume the workflow.
Let's run the program and see this in action:
Set a breakpoint at the first line. Keep executing the code until you reach the "instanceUnloaded.WaitOne()" line, as follows:
To recap, until this moment, we have initialized the workflow, set up Persistence, set up (but not executed - obviously) the PersistableIdle and Unloaded actions, and finally run the workflow. However, examine the console window and you will see that the only message printed is the host thread ID, which is done by the host as shown above in the last line executed; this indicates that the workflow has not started execution although we have called the Run method. Why is this again? Because we are yet to call WaitOne on AutoResetEvent to signal that the host can free the thread and execution can be delivered to the workflow instance.
Now hit F10. Immediately, the workflow will start execution. You can verify this by observing two additional messages printed on the console. Moreover, a breakpoint which I have set inside the custom activity is hit, indicating that execution reached this far, as shown below:
Hit F10. Now the bookmark will be created. Can you guess what happens next? Right! The action of PersistableIdle gets executed because once the bookmark is created, this signals that the workflow can be persisted. Next, the action of Unloaded gets executed since we have selected PersistableIdleAction.Unload... this is shown in the images below:
Once the Unloaded action finishes execution by calling the Set method as shown above, execution control moves back to the host application, and you will see that by observing another breakpoint set on the resume code, as shown below:
Now before resuming execution, let's validate that persistence is working. After all, creating the bookmark and unloading the workflow instance should mean that it is now persisted in the DB. Open your SQL Server and observe the table "InstancesTable", and you will see a record corresponding to the persisted workflow instance.
Let's continue: execute the resume code:
First you will be prompted to present an input. Then the workflow instance will be reloaded using the instance GUID and ResumeBookmark will be invoked. This will cause the OnBookmarkCallback method to be executed, and finally the Completed and Unloaded actions will be executed.
The final console output should be as follows:
What's Next?
I already started writing a full article about Windows Server AppFabric. In this article, all the examples used self hosting; in my next article, I will utilize AppFabric for IIS 7 hosting.
I already have video recordings on my blog about AppFabric (click here) taken from my last community session; but I am working on an article with source code for easier reference...
