Contents
- WABS Bridge Hosting
- VETER + WORKFLOW = VETEW Pattern
- Loosely decouple pattern
- Based on WCF Custom Channel
- Encapsulation of the Workflow Definitions
- Workflow Definitions (xaml) stored in the Azure Blob
- Using a blob metadata for workflow configuration
- Message Exchange Pattern via workflow arguments
- Full transparent pattern to the VETER
- Allow to use Workflow Custom Activities
Microsoft recently released the general availability (GA) of Windows Azure
BizTalk Services (WABS) in the Windows Azure Platform. WABS enables business
integrations using a declarative programming within the pre-built message
pipeline pattern known as VETER (Validate-Enlarge-Transform-Enlarge-Route) in
the multitenant cloud environment. Received message by source entity flows via
VETER pattern to the destination entity such as Windows Azure Service Bus Relay
Endpoints. Service Bus Queue/Topic, Azure Blobs, FTP/SFTP Endpoints and External
Endpoints via WCF Stack.
WABS represents a bridge between the source and destination endpoints, where
an inbound message can be targeted only at one of the many destination
endpoints. The message pipeline is initiated in the memory and all states are
persisted only in memory and/or message, therefore the WABS Bridge is a stateless
scenario and the message lifetime process is short as much as possible. In other
words, WABS Bridge has a responsibility to mediate and deliver a message to the
destination in a very short time within the message lifetime. In the case of the
bridge crash, error, etc. the error message is returned back to the caller with
a
detail reason.
For this kind of pattern, the WABS Bridge is unified and simplified using a
VETER pattern. This simplicity allows to hide the bridge complexity and technology
integration stack while focusing more on the business scenario such as message
validation and mediation. The WABS built-in VETER pattern enables the WABS
Bridge to use in the Pre-Processing and Post-Processing business integration
stack, where the centralized processing part is holding a state for business
composition. Any technology can be used for the
business centralized part, such as BizTalk Server, IBM WebSphere ESB, Oracle ESB,
etc. or custom built composite application.
Basically, the WABS is a cloud web role service with pre-built boilerplate
stack for Enterprise Application Integration (EAI) scenario to run in a secure,
auto-scalable, dedicated and per-tenant environment. Besides that, the bridge
boilerplate has built-in monitoring and tracking capabilities. The VETER
bridge pattern has pre-built extensibility points such as
On Enter
Inspector
and
On Exit Inspector
, where the state can be
extended by a custom code. This extension is imperative and must be considered
during the design time, in other words it must be embedded into the bridge.
On the other side, the WABS Bridge Pattern allows to extend a message
pipeline using a loosely decouple manner, fully transparent and declarative
way such as using a WCF ExternalServiceEndpoint
. This is a destination
(client) proxy WCF channel for outbound message. Using this channel in the
bridge pattern, the outbound message is forwarded to the target endpoint, which
can be considered as a remote mediator, business processor, etc. The
ExternalServiceEndpoint
is based on the WCF stack, therefore we have an
additional extensibility point within the WABS Bridge Pattern.
Notice, that the ExternalServiceEndpoint
is for outbound
message only. In other words, this channel is only for the client (proxy), so there is no need
to have any listener in the channel stack. From the hosting point of the view,
the WABS represents a hosting process for Destination
proxies. This is a
great feature of the Windows Azure BizTalk Services.
Now, coming to the conceptual point of the extensibility WABS Bridge Pattern
for Workflow, where the EnternalServiceEndpoint
allows to
plug-in a WCF Custom Channel for outbound message.
Having a special WCF Custom Channel for Workflow invoker, the outbound
message can be forwarded to the Workflow skeleton, declared by xaml definition
and executed within the WABS hosting process.
This article describes this process. It shows you how easily WABS Bridge
can be extended for this feature in the message pipeline such as Workflow, where the
message can be mediated with a workflow orchestration capability in the full
transparent and loosely decupled manner. Using a WCF custom WorkflowChannel in
the WABS Bridge is giving you a new pattern such as <code>VETER + WORKFLOW
= VETEW pattern.
I do strongly recommend reading my previous articles such as
WorkflowChannel for Routing Service (detail
WorkflowChannel description),
RoutingService on Azure and
Azure
Virtual Bridge. The last article is conceptually very close to the WABS Bridge,
where WCF Routing Service is hosted in the Azure Web Role.
Within the Visual Studio Bridge Designer, there is no change and as you will see later, only config file for
a specific destination needs to be configured for using a custom
WorkflowChannel.
The following picture shows an example of the message Itinerary for 3
different bridges, where an outbound message is routed to the wcf custom
WorkflowChannel.
The above picture (message Itinerary) shows that the outbound message is sent
to the External Service Endpoint, so the message should flow to
the external service, where it will be executed outside of the WABS runtime
domain.
But, wait a moment. Until we don't see each destination config file for
a specific External Service Endpoint, it doesn't have to be true, because of the plug-in the
WCF custom WorkflowChannel. The outbound message will not exit the WABS runtime
domain, thus the outbound message will be executed in the WABS hosting process,
but under the Workflow orchestration, not under the VETER pattern.
This is the conceptual extension and from the abstraction point of the view, I
can call this pattern VETEW
, where R-outer is replaced by
W-orkflow. Note, that the W represents an in-memory workflow, its lifetime is
done by WABS lifetime. In other words, this is a non-persisted Workflow run in
the WABS Bridge.
That's great. Let's continue. The following picture shows a simple example of
the Pass-Through Bridge to the WorkflowChannel, where the outbound message is
mediated for Publisher, for example writing a message payload to the Azure Blob
and notification on the Service Bus Topic:
The above picture shows additional comments that indicate the
destination connectivity is calling a custom channel (net.wf://repository/bridges
)
on One-Way External Service Endpoint. So, any inbound message received by this
bridge (Fire
) is forwarded to the Custom WorkflowChannel (Publisher
).
Basically, the above picture shows a basic pattern without the full VETER
message mediation, it is a pass-trough workflow bridge.
The next picture shows more capable message mediation within the WABS
Xml Request-Reply Bridge
. The inbound message can be fully mediated in the
Router entity such as
Xml Request-Replay Bridge
and then its
outbound message is forwarded to the Custom WorkflowChannel.
In the above message itinerary, the destination entity is called as
WorkflowInvoker
. The reason for that is, that any inbound message
received by
Router
can invoke a specific xaml workflow definition,
as you will see latter.
Now, it is a time to show a custom config file for WorkflowChannel:
As you can see, the configuration of the client endpoint with a custom binding
is very straightforward. The client address
net.wf::
declares a custom binding with
net.wf
schema and a xaml
resource stored in the repository (Azure Blob) below the container
bridges
.
Authorization access to the repository account is declared in the binding
section.
Really, this is very straightforward in the 1-2-3 steps such as adding a custom
WorkflowChannel
assembly to the bindingElementExtension
,
then adding a customBinding
and lastly configuring a client
endpoint for WorkflowChannel. Basically, this configuration can be used for any
WorkflowChannels
in your Bridges where all xaml workflow
definitions are stored in the same place.
Behind the WorkflowChannel is a xaml workflow definition stored in the
Repository, The WorkflowChannel has a responsibility to load and execute a
specific xaml definition from the storage.
The following picture shows an example of the very simple workflow such as a
message LoopBack
, where an incoming message is passed back to the
invoker.
The LoopBack
Workflow is stored in the container
bridges
and has only one Assign activity for message loopback. Note, that the
_inMessage
and
_outMessage
are Workflow mandatory
Arguments used for message integration with a Bridge, as you can see later in more
detail. Note, that this LoopBack Workflow is very useful piece for creating a
custom message mediator primitive within the VETER pattern, where a
transformation is a place to provide a complex message mediation to encapsulate
a business model. In additional, the LoopBack Workflow can participate in the business mediation
as well.
Let's bring up the following picture to show a position between the VETER Bridge
and Workflow Definitions:
The above picture is a great conceptual picture of the VETEW
Pattern. The VETER Bridge part represents a Pre/Post Message Processing for
Workflow Definition. The xaml workflow resources, known as the workflow bridge
definitions are stored in the Azure Blob Storage (repository). We can deploy the
WABS Bridge once for a business oriented library of the workflow definitions and call
them to compose a business model based on the some rule sequence.
Of course, the WABS Bridge can be built for more complexity, using a
filtering, for example, like it is shown in the following picture:
The above Bridge shows a routing message to the 2 custom destinations such as
WorkflowChannels.
Based on the Route Action the message can
be routed to the Weather
or LoopBack
.workflow
definitions.
Wait a moment; how does the custom WorkflowChannel know which workflow definition
to use?
Well, as you may know from my previous articles, the name of the xaml workflow definition has been passed via an
address uri query string, for instance:
Unfortunately, the current version of the WABS doesn't allow to pass a query
string via an address Uri, therefore the original WorkflowChannel for Routing
Service was necessary to modify its implementation for accepting a custom header
in the message in the case if the address Uri doesn't have a query string. I
hope, the Microsoft WABS Team will allow this feature in the next version.
Anyway, for now, the message forwarding to the WorkflowChannel must have a
simple string type custom header query, see the following screen snippet of the
Bridge Message Sender program:
The above example shows, how any workflow definition can be invoked behind
the Router Bridge. The Request message has a direct custom header query for
invoking a xaml workflow, in this example it is a LoopBack
workflow. As you will see later, the query header can also specify a name of
the container, etc.
That's great, what about a runtime invoking a workflow definition from the
repository based on the message payload, etc.
Basically, we can use the same principle for adding and modifying a header in
the message. The following example shows how a custom header can be added into
the outbound message:
In the above example, the header value is created explicitly in the
Expression TextBox. Note, that the value is overwritten for existing header.
I hope by this point you get an idea of what
W
means in the Bridge
VETEW
Pattern.
As I mentioned earlier, the WABS Bridge is a stateless bridge to make a Pre
and/or Post Message Processing for mediation primitives, transformations,
enlarge, etc. Adding the W
into the VETER
Pattern we
can enable WABS Bridges for decomposition of the business model which will
require a message splitter, message accelerator, business context streaming,
etc. without using more sophisticated (and expensive business server) for small
and/or middle size solutions.
The following picture shows an example from my previous article
Using Azure Lease Blob, where the business composition is done using an
EventStream
based on the Lease Blob.
As you can see, the above picture shows a composition of the Business Model
based on the EventStream
, where each WABS Bridge exclusively published an
Event
to the
EventStream
, therefore each
Publisher can see a business context state for message acceleration, triggering
events, etc.
In the above introduction section I mentioned few my previous articles. Allow
me to add one more article (WF4 Custom activities for message mediation)
that will help you with the message mediation in the Workflow. This article will help you to
build custom message mediators, invoker based on the Contract-First, Splitter,
Analyzers, etc. in the Sequence and/or Flowchart fashion.
OK, let's continue with the concept and design. I am assuming you have a
working knowledge of the Windows Azure Platform (BizTalk Services, Azure Blob,
Service Bus, etc.)
The concept of adding a Workflow into the WABS Bridge is based on the
outbound WCF Custom Channel used in the message itinerary for generic
destination WCF External Service Endpoint. Plug-in the WCF Custom Workflow
Channel to the destination proxy enables the message exchange pattern - MEP
(one-way or two-way) between the VETER and Workflow Pattern in sync or async
manner.
The following picture shows an example of the simple one-way message pattern:
The outbound message from the VETER is forwarded to the
One-Way
External Service Endpoint
configured for custom WorkflowChannel, see the
following configuration:
The message flows into the Workflow orchestration. As you can see in the above
message itinerary, the message in the Workflow can be processed in the custom
"second" VETER pattern orchestrated by workflow activities in the Sequence
and/or Flowchart skeleton.
The WorkflowChannel has a responsibility to load and invoke a xaml workflow definition
based on the query string located in the address Uri or message header. The
source repository can be configured for default place such as a container in the
Azure Blob (this article support only this feature) or folder in the project,
for instance: Artifacts.
The query string contains at least one name-value pair such as the name of the
xaml workflow definition. In addition, this is the place to configure loading of
the
xaml workflow definition, specifically by the request. The following are few
examples of the query string:
xaml=LoopBack
xaml=LoopBack&repository=bridges
xaml=LoopBack&repository=bridges&settings=bridges/Settings/MySettings
xaml=LoopBack&repository=bridges&ns=mynamespace&
st=2014-01-18T20%3A00%3A51Z&se=2014-01-18T21%3A00%3A51Z&sr=c&sp=r&sig=yFriLn2xiJ...
Note, that the last example of the query string shows an example of the blob
addressing with a SAS arguments.
Storing a xaml workflow definition in the Azure Blob is the default and
the preferred way. The major reason for that is the blob metadata, where the
workflow configuration can be stored. The following picture shows a data integration between
the VETER and Workflow:
As you can see, the outbound VETER message is flowing to the Workflow via an
_inMessage
and returned back via an _outMessage
arguments. There
is a third mandatory workflow argument such as _metadata
. The
WorkflowChannel populated the input arguments such as
_inMessage
and
_metadata
arguments and based on the MEP fashion, it is
using an
_outMessage
argument for Reply message. Also, if the xaml workflow resource is an Azure Blob
resource, then all blob's metadata are copied to the
_metadata
argument.
Using the blob metadata for workflow configuration is a great feature, it
enables us to decompose the workflow variables in the business model and share them across
the
multiple workflows knows as Enterprise Variables. This is an advanced feature of
the custom WorkflowChannel, see more details about this scenario in my article
Azure
Virtual Bridge.
This article includes a full source code of the WCF Custom WorkflowChannel
and it is very easy to modify loading of the xaml resource based on the
requirements, for
instance, a xaml definition from specific folder/database/blob/etc. and its
metadata from other sharable (centralized) database/blob/folder/etc.
Custom message header - query
The custom message header
query
is a string type qualified header for declaring a xaml workflow
definition such as name, location, etc. Every outbound message forwarded to the
WorkflowChannel Endpoint must have this header in the message.
The following snippet shows an example of the qualified header query
:
<query xmlns="urn:rkiss/2014/wf">xaml=Weather</query>
The header can also be created declaratively during a runtime in the VETER
Pattern or Route Action
.
The following screen snippet shows an example of the routing a message to the
Weather workflow definition in the case of zip > 90000:
The WCF custom channel
for Workflow has a very simple model supporting only
IOutputChannel
and
IRequestChannel
interfaces:
All custom logic related to the workflow target is encapsulated into the
custom WfHelper
static class, see the following code snippet:
public IAsyncResult BeginRequest(Message message, TimeSpan timeout, AsyncCallback callback, object state)
{
base.ThrowIfDisposedOrNotOpen();
base.RemoteAddress.ApplyTo(message);
WorkflowApplication wfa =
WfHelper.CreateWorkflowApplication(message, repositorySettingName, trackingParticipent);
return wfa.BeginRun(timeout, callback, state);
}
public Message EndRequest(IAsyncResult result)
{
return WfHelper.EndRun(result);
}
At this point, I would like to refer you to my previous articles where detail
description of the design and implementation of the
WorkflowChannel is described. There are few changes of the original WorkflowChannel made for WABS Bridge
are couple, such as
signing an assembly and adding a query header logic in the GetQuery
Method:
if (string.IsNullOrEmpty(query))
{
int index = message.Headers.FindHeader("query", "urn:rkiss/2014/wf");
if (index >= 0)
query = message.Headers.GetHeader<string>(index);
}
Let's spend more time with Usage and Test of the
WorfklowChannel
in the WABS Bridge.
First of all, the following are prerequisites (based on my development
environment):
In this section, I am going to show you step by step usage of the
WorkflowChannel in the WABS Bridge.
Step 1. Download a package from this article
The following
screen snippet shows BizTalkServiceVirtualBridge
solution package included
in this article:
As you can see, the above solution includes 2 projects such as a WABS project -
BizTalkServiceVirtualBridge
and
WokflowChannel
library
project. Basically for usage of the WorkflowChannel in your next WABS solution
only the
WorkflowChannel.dll
assembly is required. Therefore,
the above WABS package is only for our test purposes of the WCF Custom
WorkflowChannel. As you can see, there is a folder
Tools
with some assemblies used
for our testing. One more thing, the Artifacts folder has two xamlx resources
for our testing such as
LoopBack
and
Weather
. These
workflows are for testing purposes only.
Do not compile this solution before the step 2, You have to update the
solution with your accounts.
Step 2. Modify solution for your accounts.
The solution must be modified for your accounts such as
BizTalk Service
URL
property and configuration file for all following destination
endpoints:
As I mentioned earlier, the WorkflowChannel is using an Azure Blob Storage
for repository of the xaml workflow definitions. For test purposes, this
solution uses (like it is shown in the above config) two containers, one is for
assemblies
and the other one is bridges
for xaml
resources. Please, create those private containers in your blob storage.
Once these changes are made, the solution is ready for deployment to your WABS endpoint.
One more help, if you will have an issue with building this solution, please close the Visual Studio and reopen it again, but use only
the Build feature. I believe there is a bug in the Visual Studio 2012 when holding
assembly reference. That's my experience and I hope that upcoming WABS
for Visual Studio 2013 will resolve it.
Step 3. Deployment
To deploy WABS solution from the VS2012, the following dialog prompt
will be show up on your screen:
The deployment Endpoint will show your WABS namespace and you must fill in
the AcsNamespace
and SharedSecret
properties. After that, you can watch the
deployment process in the Ouput
panel and I hope without any errors.
If the deployment succeeded, you should see the following in the Server Explorer this
solution in your WABS namespace:
The
Windows Azure BizTalk Service Explorer is a great tool. It allows to refresh, restart, update, etc. your WABS.
Also, you can see a list of the uploaded assemblies. The above picture shows those
assemblies which we need to have for our next testing. Note, that the missing
assemblies can be manually uploaded or added in the solution reference with the
flag True for Copy Local
property. By the way, this flag is useful for
controlling an assembly upload, it will speed up the re-deployment process
when the bridge itinerary has changed, for instance.
Now we can focus on Workflow part.
Before we proceed, let's pre-test for our deployed WABS which will indicate
if we are on the right path to the Workflow.
Let's invoke some Workflow, at this moment we can invoke any kind of name
because our blob storage is empty.
Step 4. Invoking a non-existing Workflow
For easy way to send a message to the WABS, we can use my "dirty tool"
included in the solution folder Tool.
Please, launch the MessageSender
program and populate your account values for
your WABS namespace. Setup a query string value for xaml=ABCD
. After that,
press the button Send. You should get a response message from your WABS
indicated the following:
That's a good sign, that your WABS should work in the following steps.
Step 5. Invoking an existing Workflow
In this step we are going to upload blobs to your Azure Storage. Note, you
will need to use your own credit for WABS and Blob Storage.
Any 3rd party Azure Storage Explorer, but in
this step let's used very popular and free
Azure Storage Explorer from my friend
David Pallmann.
Please, upload the following xaml resources to your Blob Storage container
bridges
:
- Artifacts/LoopBack.xamlx
- Artifacts/Weather.xamlx
make the following changes:
- delete an extension
.xamlx
in the name of the blob (this is
an option, but in this case we have to use a full name
of the xaml resource in the query string) - use content type
application/xaml+xm
(this is an option in
the opposite type text/plain
, where we can redirect a
resource to another blob) - add the following metadata for Weather blob:
Please, upload the following assemblies to your Blob Storage container
assemblies
:
- Tools/MessageMediationActivityLibrary.dll
make the following changes:
- delete an extension
.dll
in the name of the blob
In the case of using a custom activities in your Workflow, you have to upload
its assembly into this container and restarted a WABS be sure that this assembly
is going to load into the WABS Bridge process (web role).
That's all. Now it is the time to do some testing. Let's start with something very simple and using only a Microsoft Workflow Activities from the .Net
4.5, some simple workflow such as a LoopBack.
Step 6. Test LoopBack Workflow (Router)
This test is using a Router WABS Bridge. Its address is
<a href="https://your_namespace.biztalk.windows.net/default/Router">
https:
Any query string will be forwarded into the WorkflowInvoker
destination with
a WCF custom WorkflowChannel.
Use the MessageSender program to send a request message with a query header:
<query xmlns="urn:rkiss/2014/wf">xaml=LoopBack</query>
The following screen snippet shows a response from the LoopBack WABS Bridge:
You can work with different zip code
in the request message payload to see the changes in the response message. The
response panel shows a TrackingID
. This id
can be used
for tracking a message through the Router Bridge using the WABS Explorer or
Azure Portal.
Note, that this Router Bridge can be used for invoking any xaml workflow
definition stored in the container bridges
. I am sure, you will use
it to invoke your workflows as well.
Step 6. Test Weather Workflow (WeatherBridge)
This test will demonstrate the capability of the VETER bridge to select two
different xaml workflow definitions from the Azure Blob Storage. The Weather
Workflow is a very straightforward workflow using a custom activity
CreateMessage
(from my recommended article
WF4 Custom activities for message mediation) to create a request and response
message from/to call to the Weather public service. In this test we are testing
the capability of the loading a custom assembly from the container
assemblies
.
The following picture shows a WeatherBridge
with two destinations configured
for WorkflowChannel:
Note, that the inbound message doesn't need to have a query header, because
this header is added in the outbound message based on the filter.
As you can see, based on the payload value ZIP, the message is routing to the
properly destination. There are 3 groups of the filters. The third one is when
ZIP = 90000. In this case, the caller will receive an error message.
The following screen snippets show these cases:
LoopBack Workflow:
Weather Workflow:
As you can see, there is a custom header in the last response message configured in the weather's metadata. You can
use this
configuration during your testing without re-deploying the bridge or updating blob
content.
That's all for testing. I hope both tests succeeded on your WABS deployment.
In addition, the demonstration showed you the capability of the VETEW Pattern using the
WorkflowChannel.
Advanced Tooling
My article
RoutingService on Azure is described a small modification of the
Azure Storage Explorer which will associate
a program with blob content. The following picture shows this user interface
extension:
Clicking on the Edit
link, the re-hosted WorkflowDesignerTester
will launch for editing
of
the xaml definition. The changes in the xaml can be saved in the blob. Note,
that the fresh version of the WorkflowDesignerTester
program can be found in the
solution Tools folder.
This article described how we easily
we can extend the pre-built VETER Pattern in the Windows Azure BizTalk Service
(WABS) by Workflow (WF) capability. Using a WCF Custom WorkflowChannel in the
Destination Endpoint, the outbound VETER message pipeline continues in the
Sequence and/or Flowchart skeleton of the Workflow. This article described a new
concept pattern such as VETER + WORKFLOW = VETEW. I hope you enjoyed it.
[1] Windows Azure
[2]
Windows Azure SDK 2.2 for .Net
[3]
Windows Azure BizTalk Service Explorer
[4] Azure Storage Explorer
[5]
Windows Azure: General Availability Release of BizTalk Services, ...
[6] BizTalk Team
Blog
[7]
Walkthrough of New Windows Azure BizTalk Services
[8]
Azure
Virtual Bridge
[9]
RoutingService on Azure
[10]
WorkflowChannel for Routing Service.
[11]
Windows Azure BizTalk Services: EDI Capabilities
[12]
Developing custom bridge components for Windows Azure BizTalk Bridges
[13]
BizTalk360 Blog
[14]
Extending Windows Azure BizTalk Services
[15]
Securing Azure BizTalk Services using ADFS
[16]
Extending Windows Azure BizTalk Services Using Message Inspectors