Contents
- Push Model
- Message Mediator (Pipeline) using VETER pattern
- Business Integrator
- One-time deployment
- Xaml Bridge Definition stored in the Azure Blob Storage
- Web Role and Azure Storage Requirements
- Windows Azure Service Bus Messaging (November 2011)
- WF4 Technology
- WCF4 Technology
Azure Virtual Bridge represents a dynamic connectivity pipeline for declaratively message mediation using the Microsoft Workflow Foundation (WF) and Windows Azure Platform. The Virtual Bridge for specific connectivity is declared by the Bridge Definition, stored in the Runtime Repository for its later invoking.
The following picture shows its highlight:
As you can see, the Virtual Bridge is a message mediator between the Service and its Consumer for enabling their integration for some reason such as different protocol, message exchange pattern, data contract, message version, etc. The Virtual Bridge Mediator has predefined pattern in the message pipeline (bridge template) and specific business integration can be declared via the modeling tool.
The Virtual Bridge has a runtime service to project a selected Bridge Definition stored in the storage container. In other words, deploying the Virtual Bridge package, we can manage the integration between the endpoints on the fly based on the business needs. Note, that the Virtual Bridge is not persisted, it is not a long running process, it is a memory running message mediator process, where the message represents a state.
The Virtual Bridge focuses on the integration process and/or for message mediation. Its process should be run fast through the message pipeline transparently and scalable manner.
Basically, how the caller consuming the Virtual Bridge, we can have many different bridges such as XML, SOAP, REST, X12, flat file, SQL, etc. bridges. This article will discuss only the SOAP bridges with OneWay and TwoWay message exchange patterns, so the Input message resp. Request/Response message must be Soap Messages.
The message flows through the Bridge pipeline via bridge states. Basically, the Bridge can be defined by 5 states for each direction. The pattern of these states are called VETER, where:
Validate - in this state the message is validated, for instance, by its schema Enrich - is a state for message modification, for instance, adding new types, etc. Transform - in this state, the message can be transformed to another schema using a XSLT Technology Enrich - is a state where the message can be "massaged" again Route - in this state, the message is dispatching to the targets based on the filtering
Note, that each VETER state is optional and can be bypassed (turn off) based on the integration scenario and business needs.
The Designer for VETER message pipeline (bridge) can be hardcoded and embedded into the host process. Recently, Microsoft release a update of the CTP version for Service Bus EAI and EDI Labs, where the VETER pattern is built-in the Visual Studio 2010 and extended Service Bus templates for EDI and EAI projects. It's a great feature; I do recommend to evaluate this CTP version.
The following pictures are screen snippets of the sample OneWayBridge project:
Clicking on the link in the Transform state, the following Xml Mapper Editor will show up:
Very nice features for the CTP version. However, in this version the EAI & EDI Designer can not be re-hosted it is very tight to the Visual Studio project solution including assemblies. I hope, the Microsoft team will take a challenge to make this great capability of the Software Bus EAI & EDI rehostable and VETER pattern such as custom activity for WF Designer including a Mapper Editor as a WF extension for EAI & EDI business. (Note, that this CTP version is built on the top of the WCF/WF Technologies.)
Well, back to this article, its implementation didn't offer specific custom activity for "boxing a message pipeline" into the VETER pattern, however, the WF Designer allows to build this pattern based on the "patterns & practices" of the XAML Workflows. In other words, the Bridge Definition is a pipeline of the sequences - templates, which can be customized based on the business needs.
The following picture shows an example of the Validate State template, where inbound message payload is validated by schema stored in the Azure Blob Storage:
As you can see, the Sequence Validate is very lightweight process with custom activity from the codeplex project WF Azure Activity Pack CTP 1 and custom extension class for XElement:
XElement.Load(_inMessage.GetReaderAtBodyContents.ReadSubTree).MustBeValid(schema)
One more example. The following picture shows a Sequence of the Enrich and Route a pipeline message to the Service Bus Queue:
That the above state uses a custom class extension for message copy to the specific message version (in this case must be Soap12WSAddressing10) and custom activities from my previously article Using Windows Azure Service Bus Messaging. The first custom activity will create a brokered message and the second one will deliver this brokered message to the Service Bus Entity (queue).
Note, that I included all fresh assemblies from my mentioned articles in this download, see the chapter Usage and Test.
OneWay Bridge
This is a basic bridge that understands all states, integration process and message mediation. The following picture shows this OneWay Bridge pattern:
The message is pushed into the Bridge in the FireAndForget manner, therefore the caller will not wait for its delivery to the target destination. The incoming untyped message is going first to the Validate state where is validate based on the schema, then message is enriched if it is necessary and continue to the Transform state. Basically, this state represents a business core of the bridge, where the message is transformed to the target schema. The next state is again for enriching message, for example adding headers, etc.. This is a last state in the pipeline, where the message can be modified. The next step is a finally state, the Route state, where the message can be routed the destinations based on the filtering of the message headers or payload.
The following picture shows an example of the Route state, where the message is delivered based on the filter:
The filtering using If activity within the Parallel activity. The condition is for Label property of the brokered message and the Targets are different Topics of the Service Bus.
The VETER states in the Bridge can disable (skipped) or activate their usage based on needs. The following picture shows an example of the ByPass (or short-circuit) Bridge, where the inbound message to the bridge is by passed trough the all states in the pipeline and delivered to the target:
The Virtual Bridge can be chained with other bridge like is shown in the following picture:
Chaining bridges can be In-Process, where chained bridge is in the scope of the parent bridge or Remotely invoking like another service. As I mentioned early, the Virtual Bridge is a Dynamic Activity, therefore In-Process chained bridge can be under the ambient transaction scope.
My recent article Azure Service Bus Pusher shows how a brokered message can be pushed to the Target destination such as a Service Bus and/or http/https service. In the case of the legacy Target or different protocol, etc. we need to integrate systems via Bridge, where the message can be mediated.
The following picture shows an example of this scenario, where the brokered message is pushed to the OneWay Virtual Bridge. The Bridge will care this integration between the Azure Service Bus and Target Service:
That's great integration. Note, that the above scenario doesn't require a service (listener) for this bridge. Another great example for OneWay Virtual Bridge is shown in the following picture, where the Bridge is bridging two different Service Bus Namespaces, mediating a brokered message, etc.:
One more thought here, As I mentioned early, the Virtual Bridge VETER is driven by metadata, where the bridge definition is stored in the runtime repository for its projecting in the host service process. For its Route state is used a WCF paradigm where declaratively way is setup their channel stacks such as transport, binding, contract, etc. This abstraction allows for virtualization of the VETER Bridge and encapsulate a specific communication layer into the custom channel. Selecting different Targets such as Service Bus, SQL, File System, etc. without modifying a VETER pattern can be done by configuration in the Bridge Definition. In other words, instead of building a custom adapter/activity we can used an extensibility of the WCF stack. As a good example for this design and architecture is the Workflow Custom Channel, which is the core of this Virtual Bridge - see more details later or in my articles RoutingService on Azure and WorkflowChannel for Routing Service.
TwoWay Bridge
The TwoWay Bridge is another message exchange pattern for consuming a Bridge Definition. In this communication, the consumer sends a request message to the Bridge and waiting for its response in synchronously or asynchronously manner. Basically, the TwoWay Bridge is combined from two OneWay Bridges in one Bridge Definition, where the second one has a special Route state for Reply message to the caller. The following picture shows this model:
TwoWay Bridge allows to mediate a request and response message in the Bridge pipeline. Both patterns are a VETER pattern and their states are based on the integration and business needs.
The following picture shows a TwoWay Bridge where a Response is VOID. In other words, there is nothing to mediate on the Response Bridge and the consumer wants to wait for completion of the call or received a fault message back:
In the above example, the message in the Request Mediator (pipeline) is processing based on the business integration process, the Route state will dispatch the message to the service(s) based on the filtering rules and after this state the void response is reply through the empty Response pipeline. Note, that the Response Route state is always state for dispatching a Reply message, therefore we can say, the TwoWay Bridge has a mandatory Route-Reply state for Response pipeline for "anonymous reply address" such as back to the caller.
The Bridge definition (pipeline) for TwoWay Bridge can be simplified based on the message mediation needs. I can explain it in the following picture, where we have a TwoWay Bridge without any external call at any pipeline state, in other words, the Bridge Definition using a local or embedded mediation primitives and message state for processing this pipeline:
Basically, in this scenario, we can this Bridge Definitions (templates) shared across the Bridges. For example, the Bridge Definition for transforming documentation in EDI X12, EDIFACT, Flat File, CSV, images and many other formats.
The caller Bridge in the Transform state can call this "tiny Bridge" in the full transparent manner like calling a regular service, see the following picture:
As you can see, the "tiny Bridge" represents a typical message mediation primitive for transforming an input message to the output message with different format. Invoking this Bridge can be In-Process or Remotely across the process boundary.
Untyped Message
Inbound message to the Bridge is untyped message, therefore is the Bridge Definition responsibility to validate its contents and deserialize into the object. Message is mediated in the pipeline using a XML Technology. I do recommend to read my article WF4 Custom activities for message mediation, where in details is described this technique in the xaml workflow.
The following picture shows an example of this technique for dynamically custom https binding with Basic Authorization. The first custom activity is a generic custom xslt activity for runtime transformation xslt resource for binding object. The next activity such as BindingScope will use this object for Send Activity.
The Bridge Definition is a xaml Dynamic Activity with VETER Template. Basically, the minimum declaration of the Bridge is shown in the following example, where a "loopback" Bridge is declared with assigning message _inMessage = _outMessage:
Note, that the Bridge Definition doesn't have any activity for idle workflow state such as Receiver, Timer, etc. The Bridge is for short in memory running workflow (dynamic activity).
The Virtual Bridge is based on the Contract First model, where the contracts, schemas, resources, artifacts, etc. are stored in the Repository during the Design Time. The Virtual Bridge has a virtual endpoint for all Bridge Definitions.
The following picture shows an example of the calling a HelloBridge definition:
As you can see. the second addressing allows to specify name of the Repository where is stored the HelloBridge Bridge Definition. Note, that this url query is passed into the Bridge Arguments for its usage in the VETER pipeline.
More conceptually details about the manageable services driven by metadata can be found in my articles Contract Model for Manageable Services and Manageable Services.
From the conceptually point of the view, there is a logical central Repository for Contract (Model) First resources which the client/consumer and also Tools can make a query to obtain a metadata (mex, wsdl, xsd, etc.) for modeling, creating proxy, etc. The Contract Model can be stored in the Windows Azure Platform including a web role service for resource querying. Note, that this model is not included in this article and the reason why I am mentioning about this is giving to you a full picture and architecture styles for metadata driven services.
I do recommend looking at Microsoft Announcing the Refresh of Service Bus EAI & EDI Labs and I hope this article can help you to understand integration and message mediation between the systems using an edge of the Microsoft Technologies.
OK, let's continue with the concept and design of the Virtual Bridge. I am assuming you have a working knowledge of the Windows Azure Platform and WCF/WF Technologies.
The Concept of the Azure Virtual Bridge is based on the WCF Custom WorkflowChannel. I do recommend reading my articles WorkflowChannel for Routing Service and RoutingService on Azure, where I described in details how to build and use the WorkflowChannel. I rebuild this custom channel specifically for Windows Azure Platform and plumbing to Azure resources. In this section I will focus on these features.
The following picture shows a concept of the VirtualBridge on Azure:
As you can see, the concept is very simple, there is a regular WCF Routing Service, Azure Blob Storage and one "magic component" such as a WorkflowChannel. This custom channel is built for specifically purpose (support only IOutputChannel and IRequestChannel) such as runtime projecting a Bridge Definition under the Routing Service hosted by Web Role.
The Consumer inbound message is received by Routing Service and based on the message exchange pattern (OneWay or Request/Reply) is forwarded to the outbound routing channel such as WorkflowChannel. The plumbing of this scenario is configured in the web.config file.
The following picture shows a parts of this config file:
The VirtualBridge is addressed in the REST style, where a query part of the url address is dedicated for bridge definition such as what and where is metadata. The following picture shows an example of the bridge definition HelloBridge:
The WorkflowChannel can recognize a full entry address of the inbound message which it is necessary for its loader logic to pick-up a requested bridge definition dynamically from the Azure Storage. The name of the storage container can be configured in the config file as a default container or it can be added to the query part of the url address, for instance: xaml=HelloBridge&repository=demo
The physical url address for Virtual Bridge can be rewrite based on the business needs. For Windows Azure is very simple and straightforward because this IIS module is install by default. The following picture shows examples of this feature:
and the rewrite section in web.config file for first example is shown in the following screen snippet:
I like this feature, it allows mapping a service physical address path to the logical one.
The other interesting part of the Routing Service configuration is bindings. The following picture shows this part:
As you can see, there are two bindings, the first one is for outbound custom WorkflowChannel and the other is a custom binding for inbound secure channel based on Basic client credentials using a username and password.
The next section is a behaviors.
In the behaviors section we need to configure a routing, serviceCredentials, publishing unhandled errors on the ServiceBus Topic and workflow tracking. The first one such as routing extension is a mandatory element for Routing Service, the other ones are optional depends if we need to use secure transport, error handling and tracking. The ClientCredentials is authorized in the WorkflowChannel and it is individually for each Bridge Definition.
Note, that the sqlTracking extension is not included in this article, but it is not difficult to implemente custom tracking and logging. On the other side, I included in this solution a Service Bus Publisher, which can be used for your custom tracking on the Topic.
As you can see, the routing service config file is a place for "cooking our desert". All features of the Virtual Bridge is based on the pluggable components, therefore is easy to replace based on the business needs.
One more configuration. The Routing Service is hosted by Web Role, therefore all important configuration are encapsulated into the web role settings to allow changes "on the fly":
The following picture shows these settings:
Bridge Definition
The Bridge Definition (BD) represents a xaml workflow dynamic activity for message mediation using a VETER pattern. From the architecture point of the view the BD is a metadata for representing an integration process using a declaratively way. During the design time we are creating this metadata based on the business needs using a re-hosted workflow designer and storing in the runtime repository such as Azure Blob Storage. Once this resource is stored in this repository, the WorkflowChannel can pick it up and project it in the Router Service.
The following picture shows this scenario:
The WorkflowChannel has built-in a "sender" for IOutputChannel and IRequestChannel to process all steps for loading Bridge Definition, creating its instance, passing arguments, settings, etc. and then invoking this dynamic activity. Virtual Bridge is scalable, multitenant manageable service hosted in the Web Role with multiple instances. This feature allows runnig multiple bridges concurrently and elastically in the manner of one time deployment. Any time during the runtime, the Bridge Definition can be modified, add new one, remov it, switch to another version, rollback version, redirect Repository or Blob, etc.
Note, that this article (public version) doesn't have support for caching a Bridge Definition from the Blob Storage either its clr type. It is not difficult implementation to include a cache invalidation for specific Bridge Definition across the Web Role instances. Solution implemented in my recently article Azure Service Bus Pusher can be great resource for the intial feature.
Addressing Bridge Definition
Invoking a specific Bridge Definition is driven by inbound message address, where url query contains the name of the Bridge Definition located in the default Blob Container configured in the web role settings. Adding additional query parameters in the message address such as repository and settings, the consumer have option to select another container/blob/metadata from the same storage account.
Same examples for addressing (query) a Bridge Definition:
- xaml=HelloCloud
- xaml=HelloCloud&repository=Demo
- xaml=HelloCloud&repository=Development
- xaml=HelloCloud&settings=bridges/Settings/HelloCloud_Demo
- xaml=HelloCloud&repository=Development&settings=bridges/Settings/HelloCloud_Dev
Note, that for accepting the settings parameter in the address query, the Default (Primary) Bridge Definition need to enable this feature inserting a metadata parameter _metadataLocation=user, see more detail later.
Another capability to manage address path for Bridge Definition is to redirect a primary address path defined in the url query (see the above examples) to any container within the same storage account. The following picture shows this situation:
The Default (primary) container is configured by either web role settings or directly in the url query. The WorkflowChannel has built-in a capability to check the content type of the blob. In the case of text/plain with url address schema https, it will download a bridge definition resource from this address. The above picture shows example for HelloCloud2, where the body of the its bridge definition is pick-up from the container Demo and specifically blob with name HelloCloud_version_1.0.0.0.
That's great. This mechanism allows redirecting a bridge definition based on the needs, such as new version, rollback version, temporary versions, etc. This article version didn't include a Virtual Bridge Management Portal (web role) to manage a Virtual Bridge, its definitions, metadata, settings, deploying, etc. I hope, you got this idea to build one for your needs.
One more note, as you know, each specific blob has own user-defined metadata mini block (header) as a collection of the name-value pairs (see mode details next section). The WorkflowChannel has an option dynamically selected which metadata will be loaded with the blob resource. This option is activated and processed by inserting a special property (_metadataLocation) into this user-defined metadata - see more detail later. Basically, there are capable to use a local, remote, user or specific metadata blob.
Consuming Bridge Definition
From the abstraction point of the view, the Bridge Definition can be consumed by manner of message exchange pattern (MEP). The WorkflowChannel supports only two MEPs such as OneWay and TwoWay (Request/Reply) message exchange pattern.
The following picture shows this abstraction, where the _inMessage is inbound Routing Service message and for second MEP is used _outMessage for its reply back to the caller:
The WorkflowChannel has a responsibility how to use this MEP based on the inbound message. In other words, the Bridge Definition can have this _outMessage, but it will not use in case of OneWay MEP.
Note, that the default setup of the Routing Service for inbound/outbound messages is Soap11.
Arguments for Bridge Definition
The plumbing between the WorkfowChannel and Bridge Definition is via the xaml Dynamic Activity arguments. This is very important point and required element for design time and for VETER templates. Every Bridge Definition must have the following arguments:
We can call these arguments as Bridge Definition Arguments. The WorfklowChannel is checking these argument for name and type and if is not match it will throw Exception with reason message for missing a specific argument.
I know, you can recognize two arguments such as _inMessage and _outMessage as has been described above, but what is the _metadata and for what we need it in the Bridge Definition?
Well, this argument represents settings for specific Bridge Definition; it is a collection of the name-value pairs like in the config settings section. Basically, this is a copy of the blob metadata entity and we can call it as Bridge Definition Metadata. Besides that, like you will see later, this is a good place for managing specific Bridge Definition on the fly and also runtime configuration for WorkflowChannel.
The blob metadata has the following features and limitations:
- Names are case-insensitive. Note that metadata names preserve the case with which they were created, but are case-insensitive when set or read. If two or more metadata headers with the same name are submitted for a resource, the Blob service returns status code 400 (Bad Request).
- The total size of the metadata, including both the name and value together, may not exceed 8 KB in size.
- Metadata name/value pairs are valid HTTP headers, and so they adhere to all restrictions governing HTTP headers.
- Metadata can be retrieved or set directly, without returning or altering the content of the blob resource (body).
Note, that the Bridge Definition Metadata can be also modified, added, etc. on the runtime via a custom activity like another resource in the Azure Storage.
The following picture shows a "data schema" for Bridge Definition:
I hope, you can see now the position of the blob metadata for Bridge Definition. It is local settings for specific Bridge Definition which can be managed by custom portal, bridges, etc. Let's discuss little bit more about this metadata.
Bridge Definition Metadata
The Bridge Definition Metadata represents a settings for specific Bridge Definition in the Azure Blob Storage. This dedicated blob entity of the name/value string pairs allows us to decouple the Bridge Definition dependences into local small storage (max 8k) and make a more generic VETER Bridge. The WorkflowChannel can read this metadata without downloading a full blob body, therefore this collection of the properties can be shared for customizing a custom channel on the fly. The value of the properties in the blob metadata is a text.
The following picture shows an example of the Bridge Definition Metadata (such as a blob metadata):
As you can see there are a few highlighted names. These are a special properties recognized by WorkflowChannel for very cool features, see more details below.
Reserved names for Bridge Definition Metadata properties:
_username, this is a mandatory property for client authorization _password, this is a mandatory property for client authorization _trackingProfileName, optional property for tracking level _metadataLocation, optional property for selecting a metadata blob (see early description), valid values are: local, remote,user or url address of the any blob located within the current storage account _urlQuery, this is a hidden property of the bridge address url query parameters in the format name/value pairs
The _username and _password are for client Basic Authorization for accessing a specific Bridge Definition. To turn off this feature (accepting any u/p), please setup both of them for character '*'.
The next property such as _trackingProfileName is for bridge runtime tracking if we have it. This feature allows to change level of the tracking for the specific bridge dynamically or administratively via a custom portal.
The property _urlQuery is giving us to get a collection of the inbound address url query into the Bridge as additional arguments for business needs.
The last property such as _metadataLocation is giving us power of the Virtual Bridge. This is a property for WorkflowChannel to manage a source of the Bridge Definition Metadata located in the blob metadata. If this property doesn't exist or if its value is local, the WorfklowChannel will use a Default (Primary) blob storage and specified blob. In the case to use a remote (indirect) Bridge Definition, this value must be setup for remote.
The next option is for user, if user wants to select his/her settings for Bridge Definition (per call) it can be done via the address url query part, for example: settings=bridges/Settings/HelloCloud_Dev.
Finally, when we want to have multiple settings, for instance, based on the environment scope such as development, staging, qa, demo, production, etc., we can setup value for this property explicitly with url address of the blob resource. This address also works for relative addressing, for example: ~/bridge/Settings/HelloCloud_Dev where ~ represents a default blob storage configured by web role settings.
The following picture shows an example of the mapping Bridge Definition:
The inbound message (xaml=HelloCloud2) will pull-up the Default (Primary) Bridge Definition. The WorkflowChannel will exanimate its contents and based on that its loader is redirected for Bridge Definition HelloCloud_version_1.0.0.0. Now, the WorkflowChannel need to pick-up the Bridge Definition Metadata. Here again, there is an option defined in the _metadataLocation property where in this example is a full url address for that, therefore its loader will get it from the HelloCloud_Demo blob.
That's great feature that allows us to map a physical Bridge Definition to the logical one. Note, that this mapping can be done manually using the custom portal or programmatically via other bridges, etc.
Now, let's continue with other cool feature such as Enterprise Variables across multiple Bridge Definitions.
Enterprise Variables
Enterprise Variable is a variable which can be shared across multiple Bridge Definitions, clients, etc. These variables are stored in Azure Table Storage and can be queried by name of the variable (PartitionKey) and environment scope such as DEV, QA, etc. (RowKey) The variable entity are very simple like is shown in the following code snippet:
public class VariableEntity : TableServiceEntity
{
public string Value { get; set; }
public string ApplicationScope { get; set; }
public string Description { get; set; }
}
and their storage in the Azure Table:
The Enterprise Variables are for Bridge Definition, therefore their implementation is separate from the WorkflowChannel in their own assembly with name EnterpriseVariables. It is an option to use it in the Bridge Definition. Using an enterprise variable is easy with built-in static class RuntimeRepository.Variables. This class has 4 static methods such as:
public static string GetValue(string name, string scope = "*", string format = null, params object[] args)
public static string GetValueOrDefault(string name, string defaultValue = "", string scope = "*")
public static VariableEntity Get(string name, string scope = "*", bool bThrowNotExist = true, string format = null, params object[] args)
public static IEnumerable<VariableEntity> GetAll(string filter = "*")
Examples of the usage of the Enterprise Variables:
- RuntimeRepository.Variables.GetValue("Hello Roman") // get a value of the variable 'Hello Roman', if doesn't exist then throw exception
- RuntimeRepository.Variables.GetValue("Hello " + "Roman") // get a value of the variable with name 'Hello Roman', if doesn't exist then throw exception
- RuntimeRepository.Variables.GetValue(
localVariable1) // get a value of the variable name stored in the local variable localVariable1, if doesn't exist then throw exception - RuntimeRepository.Variables.GetValueOrDefault("Hello Roman") // get a value of the variable 'Hello Roman', if doesn't exist then returned value is empty string
- RuntimeRepository.Variables.GetValueOrDefault("Hello Roman", "Hello Cloud") // get a value of the variable 'Hello Roman', if doesn't exist then returned value is 'Hello Cloud'
- RuntimeRepository.Variables.GetValue("Hello Roman", "*", "Missing variable '{0}''", "Hello Roman") // get a value of the variable 'Hello Roman', if doesn't exist then throw exception with message "Missing variable 'Hello Roman'"
- RuntimeRepository.Variables.Get("Hello Roman", "DEV") // get a variable entity with name 'Hello Roman' in the
DEV scope, if doesn't exist then throw exception - RuntimeRepository.Variables.GetAll("RowKey eq 'DEV'") // get all entities for
DEV scope - RuntimeRepository.Variables.GetAll("PartitionKey eq 'Hello Roman') // get all entities with name 'Hello Roman'
The Bridge Definition can encapsulated some variables into the Bridge Definition Metadata for their managing. We can get their value from the collection _metadata. The following examples shows this usage:
- RuntimeRepository.Variables.GetValue(_metadata.Item("text")) // get a value of the variable with name from
_metadata property 'text', if doesn't exist then throw exception - RuntimeRepository.Variables.GetValue(_metadata.Item(
localVariable1)) // get a value of the variable with name from _metadata property localVariable1, if doesn't exist then throw exception
That's great, by using a Bridge Definition Metadata properties we can manage these variables outside of the Bridge Definition. The value of this property represents a physical name of the Enterprise Variable. If we want to map a logical name of the variable to the physical one, we can use an alias name for enterprise variable. The prefix character '@' will indicate that this name is an alias name.
The following picture shows this mapping:
The Bridge Definition calling the static method to obtain a value of the enterprise variable via _metadata property 'text' such as RuntimeRepository.Variables.GetValue(_metadata.Item("<code>text")) In the case of the alias name, the method will get the value from the Table Storage under this alias name for specific scope. This is a very power full feature to allow us remapping, centralized and sharing variables for Bridges and Composite Applications.
Note, that this described concept of the Enterprise Variables has simple design and implementation, for more sophisticate design, please have a look my article Enterprise Variables for WF4.
Assemblies Blob Storage
The WorkflowChannel has built-in a handler for AssemblyResolve. This handler is setup during the binding extension. Its implementation is shows in the following code snippet:
public class WorkflowTransportElement : BindingElementExtensionElement
{
public WorkflowTransportElement()
{
System.AppDomain.CurrentDomain.AssemblyResolve += delegate(object sender, ResolveEventArgs args)
{
string assemblyName = args.Name.Split(',')[0].Trim();
try
{
Trace.TraceInformation("ResolveInternalAssembly: Resolving = '{0}'", args.Name);
string container = RuntimeRepository.GetConfigString(RuntimeRepository.VirtualBridgesStorageContainerNameForAssemblies);
if (string.IsNullOrEmpty(container) == false)
{
Resource resource = RuntimeRepository.GetResource(RuntimeRepository.VirtualBridgesStorageSettingName, container, assemblyName, false);
if (resource.Body != null)
return System.Reflection.Assembly.Load(resource.Body);
}
}
catch (Exception ex)
{
Trace.TraceInformation("ResolveInternalAssembly name = '{0}' failed. Error = {1}", assemblyName, ex.Message);
}
return null;
};
}
}
The logic is straightforward; the AppDomain.CurrentDomain will trigger an event for missing a clr type declared in the Bridge Definition. The above handler has responsibility to load a requested assembly from the Azure Blob Storage and return it back or null if the requested assembly is not recognized.
The name of the Blob Container is specified in the web role settings:
Publishing Errors on the Topic
Unhandled errors by WCF Custom WorkflowChannel can be published on the Service Bus Topic. This is an option to the WorkflowChannel configured in the web.config file and it is implemented in the separate assembly Notifications. The following code snippet shows an implementation of this handler:
public class EndpointPublishErrorHandler : IErrorHandler
{
public void ProvideFault(Exception error, MessageVersion version, ref Message fault)
{
}
public bool HandleError(Exception error)
{
Trace.TraceError("HandleError: {0}", (error.InnerException == null ? error.Message : error.InnerException.Message).Replace("\n", "\n "));
MessageFault fault = MessageFault.CreateFault(new FaultCode("VirtualBridge"), error.Message);
Publisher.Publish(Severity.Error, fault);
return true;
}
}
As you can see, the logic is very simple using a static method for this magic. Please have a look at a source code in the Notification.cs file, where is an implementation of the wcf client for sending a brokered message to the Azure Service Bus Topic. The Publisher can be configured in the web role settings.
OK, it's show time. Let's demonstrate Virtual Bridge on the Azure. I am assuming you have a working knowledge of the Windows Azure Platform.
First of all, the following are prerequisites:
The AzureVirtualBridge solution requires knowledge and experience from building and deploying application on the Azure, Blob/Table Storage and usage of the Azure Service Bus Messaging included usage of the above 3rd party tools. Therefore, I am assuming you have a working knowledge of the Windows Azure Platform.
The solution included in this article has 6 projects like it is shown in the following picture. Basically there are 4 major folders such as AzureVirtualBridge, VirtualBridgeWebRole, Lib and Test. The Test folder is included only for test purposes and it is not a part of the Azure package.
The Lib folder contains a core of this solution such as WorkflowChannel project, where is an implementation of the custom wcf channel for Bridge Definition. The other assemblies are optional and they can be plugged to the WorfklowChannel via config file like has been described early. Before compiling a solution, please change the web role settings for your accounts. The following picture shows these locations in the settings. The default name of the container where are stored the Bridge Definitions is bridges. In the case of naming conflict within your Storage account, please change it, for instance bridge2.
One more thing, the VirtualBridge Service has secured endpoint, therefore please, add your certificate or temporary reconfigure this endpoint for http binding.
Now it is a time to compile the above solution. I hope you make it without any error.
I know, this is a hard section, but let's start with step by step. Basically, I divided this steps into 4 major groups such as Preparing a Azure Storage, Preparing a Service Bus (optional step), Running in the combination of the Emulator and Azure Storage and last one after we succeeded is deploying a VirtualBridge package on the Azure.
Step A: Preparing Test Data for Azure Storage
This is a first step, when we want to create containers in the Azure Storage. For this step we can use any 3rd party tools, I do recommend free tool Azure Storage Explorer, which can be modified (extended) for additional features, for example launching a workflow designer.
The following AzureStorage solution folder contains data for uploading to specific containers on the Azure. First of all, create three Blob Storage containers such as assemblies, bridges and schemas. After that, we need one table. Please create a table Variables. The containers and table are within the same storage account, but they can be in the different accounts based on the settings.
That's great, now we need to upload files.
The following picture shows additional assemblies for our test purposes. Please, upload these assemblies from the 3rdParty folder and remove their file extension. You can upload multiple files or one by one.
Next upload is for all files located in the bridges folder. After upload process, please change their Content Type for application/xaml+xm.
For resources such as schemas I allocated separate blob container. Please upload this resource and change its Content Type:
In the case of the Variables we don't need to make any manual adjustment, just upload the file Variables.xml to the container Variables:
That's all for Azure Storage.
Next step is related to the Notification on the Service Bus. You can skip it and added later. In this case, please disable this feature in the web role settings:
< Setting name="RKiss.Publisher.ServiceBusPublisherEnabled" value="false" />
Step B: Create Topic and Subscription on the Service Bus (optional)
In this step we need to create a topic 'Bridges' and one subscription, for example Notifications with filter publisherKey='911'. Please, launch the 3rd party tool Service Bus Explorer to make these entities in your Service Bus namespace. Note, that the web role settings must match these entities.
Once we have a subscription on the Service Bus, we can subscribe a virtual subscriber to receive an interesting events on the Topic 'Bridges'. The following picture shows another useful tool such as Azure Service Bus Tester.
As you can see, there is one received message from the VirtualBridge. Specifically this is a notification that the message has been sent to the not existing Bridge Definition.
Step C: Emulator + Azure Storage
In this step, I assuming that all previously steps have been done successfully. Let's make a very simply test such as Hello Cloud. Note that in this time we will run under Windows Azure Emulator, but the Storage is on the Azure. Using the emulator allows us to debug a WorkflowChannel and other interesting parts.
In the solution is included a client tester for sending message in the one way or request/reply MEPs.
Step C1 - Invoke HelloCloud Bridge
This is your first test of the VirtualBridge with a HelloCloud Bridge Definition. The following picture shows its definition.
As you see, this is a very straightforward bridge (mediator) to generate a reply message with a text body. The text body is a local variable initialized from the Bridge Definition Metadata.
Before we run this test, we need to setup a Bridge Definition Metadata (such as a blob's metadata user block) with the following contents:
Note, that the properties with underscore prefix are mandatory for WorkflowChannel. In this case, the bridge must be authorized with a client credentials (Roman/12345).
Ok, let's make a test. Please launch the BridgeClient Tester from the Test folder. Tester has predefined defaults, therefore you need to press only the button Invoke.
But wait a moment, of course we need to run Emulator and then pressing the Invoke button. The following picture shows a result of this test - invoking a bridge (mediator) Hello Cloud:
You can stay here and play with client credentials, debugging a WorkflowChannel, changing a message version, value of the text, etc.
In the next step, I will show you how can be moved a local property text into the Enterprise Variables.
Step C2 - Using an Enterprise Variable
Using an Enterprise Variables in the Bridge Definition will require to make some changes. WF4 doesn't support a variable resolver which will be very useful to plug-in the source to resolve the value of the specific variable out of the scope. This feature is not included also in the upcoming version WF4.5, therefore we need to use some workaround, for example using a static method like I mentioned early.
Anyway, we need to initialize variable text using the following expression:
RuntimeRepository.Variables.GetValue(_metadata.Item("text"), "QA")
This expression say: get the value of the text property located in the metadata for QA scope.
As a second and last change we need to make the following change in the Bridge Definition Metadata:
Adding a prefix '@' in the value of the text property, we actually redirecting a storage to the Table Storage where are located our enterprise variables. The default setup in the web role settings shows a name of this table storage such as Variables.
I included this version of the Bridge Definition under the subdirectory V2, so the address for invoking this bridge is the following:
https://127.0.0.1:444/VirtualBridge/TwoWay?xaml=V2/HelloCloud
Please, copy this new address to the Tester and press the button Invoke. You should see the text from the Variables table for QA scope such as 'Hello Cloud from QA'.
Step C3 - Invoking Test.ZipCode Bridge
This test demonstrates TwoWay bridge with calling a remote mediator in the Transform state of the VETER pipeline. Public web service for weather (http://wsf.cdyne.com/WeatherWS/Weather.asmx) is used as an example of the remote mediator. For creating this bridge definition is used a custom activity CreateMessage from my article WF4 Custom activities for message mediation (the fresh assemblies are included in the 3rdParty folder located in this solution).
The Bridge Definition for this example is shown in the following picture:
The above model is simple, the first custom activity CreateMessage will create a request message for remote web service based on its wsdl contract (Contract First approach) and the second one will create a reply message for consumer of this bridge definition. More details about the custom activity can be found in the article WF4 Custom activities for message mediation.
Before we run the test, we need to create a Bridge Definition Metadata. The following picture shows these properties:
Ok, now we are ready to do some testing. Please select the address for this Test.ZipCode bridge and press the button Invoke. You should have the same result like is shown in the following screen snippet:
That's great, now we would like to add to the VETER pattern a state for validation of the input message. The next step shows how we can accomplish this task.
Step C4 - Invoking Test.ZipCode Bridge with message validation
This test is marked as a version V2 of the Test.ZipCode Bridge Definition, where the incoming message is validated with the schema located in the Blob Storage. I will show you more capabilities of the VirtualBridge solution, but first of all we need to prepare and plug-in a re-hosted Workflow Designer to the Bridge Definition
- Extending Azure Storage Explorer for
View With and Edit of the selected Blob resource like is shown in the following picture. I described in details about this extension in my previously article RoutingService on Azure. This extension will allow us to launch a WorkflowDesigner for modeling a Bridge Definition. Of course, it can be associated any designer, for example Visual Studio.
Note, that the WorkflowDesigner program is included in the 3rdParty folder. It is a fresh update of the solution described in my previously article WF4 Custom activities for message mediation.
If everything is going smoothly, you should see V2/Test.ZipCode Bridge Definition in the WorkflowDesigner like is shown in the following screen snippet:
That's great. One more manual step we need to do for Bridge Definition Metadata. Please, insert the following properties into this blob metadata with your storage account:
Now back to our test, where I want to demonstrate Validate State in the VETER pattern. This state is declared in the following sequence scope:
As you can see, it is very simple sequence. The first activity will get the resource (schema) from the Blob Storage. This is a custom activity from the WF Azure Activity Pack CTP 1 with my small changes in the ConfigurationName property. Updated assembly is included in the 3rdParty folder.
Once we have a body of the schema resource from the Blob Storage, we can call XElement extension method Validate. Implementation of this method can be found in the EnterpriseVariables project, file ClassExtensions.cs.
XElement.Load(_inMessage.GetReaderAtBodyContents.ReadSubTree).Validate(bodySchemaResource)
That's all. Now we are ready to invoke the V2/Test.ZipCode Bridge Definition. Please, copy and paste this address to the Tester and press the button Invoke:
https://127.0.0.1:444/VirtualBridge/TwoWay?xaml=V2/Test.ZipCode
I hope, you can see correct reply message from this bridge. You can change the request message in the Tester, for example, modify element ZIP and press the button Invoke to see now response from the bridge.
That's all for this test.
Let's make one more advanced test. The use case is to fire a "legacy message" to the Service Bus Topic.
Step C5 - Publish "legacy message" to the Service Bus Topic
This test will demonstrate a VETER pattern, where a message in the bridge pipeline is validated, transformed, enriched and routed to the Service Bus Topic. The name of the Bridge Definition is Publisher.
The following picture is a screen snippet from the Workflow Designer:
The Validate state is described in the previously test. Next state is Transform, where is used a custom activity from the article WF4 Custom activities for message mediation. The mediator in the Transform activity is very simple xslt resource for copy elements and it is embedded into the Bridge Definition. The other choice, this xslt resource can be stored in the Blob Storage and retrieved similar like I described in previously step for schema resource.
Next state is Enrich state, where is demonstrated how can be created a brokered message for Service Bus. The last state is a Route state, where the brokered message is sent to the Topic (in this example to the bridges)
The bridge is configured based on the Bridge Definition Metadata properties. Please, create these properties for this blob using your accounts:
Once we have the above settings, we can run the test. Please, select the following address and press the button Invoke. You should see reply message "Done":
The brokered message from the topic bridges can be received by the virtual subscriber in the following Tester:
That's all for this test.
Merging web role with worker role
In my previously article Azure Service Bus Pusher I introduced a worker role for pushing a message from the Service Bus to the target service. We can merge a web role (VirtualBridge) with worker role (Pusher) very easy, just adding a reference to this assembly (WorkerRolePusher.dll) and inserting the following section in the ServiceDefinition.csdef and of course we need to manually insert all Pusher's settings into the VirtualBridge settings.
<Runtime>
<EntryPoint>
<NetFxEntryPoint targetFrameworkVersion="v4.0" assemblyName="WorkerRolePusher.dll"/>
</EntryPoint>
</Runtime>
After that, we can run VirtualBridge and Pusher in the same instance which will decrees our cost on the Azure.
Finally we are here. If everything is going right, then we can move it to the finally step such as deploying a package to the Windows Azure.
Step D: Deploying to Azure
This is a standard step for deploying a package on the Azure from the Visual Studio. Package can be deployed also manually, uploading from Azure Portal. Note, that the web role requires upload of your certificate.
This article described a Virtual Bridge as an application integrator. It gives you a full implementation solution hosted on the Azure based on the wcf custom WorkflowChannel. This custom channel can project your Bridge Definition in the hosted service such as WCF Routing Service. The solution is driven by metadata stored in the Blob and Table Storage. It is a power architecture based on the Contract First concept. I hope you enjoyed it.
Appendix A - Azure Virtual Bridges Portal
Hosting Virtual Bridges on the Azure, modeling a Bridge Definitions, settings properties, variables, resources, diagnostics, etc. are a good reason to manage and administrate via a portal. The following picture shows my first draft of the Azure Virtual Bridges Portal, where the bridge can be designed, administrated and monitored:
The portal is built for multi tenant usage where users can see only their bridges and variables based on the settings. The above screen snippet shows for user rikss7 two hosted Virtual Bridges (result of the scanning connected subscriptions), their Bridge Definitions, Variables and Assemblies. Selecting a specific definition we can see xaml, settings, dashboard, etc. For editing (or modeling) definition can launch a WorkflowDesigner. Also each definition has own dashboard with logging messages. Adding a new bridge definition is very straightforward via a right clink on the TreeNode. This action will pop-up the repository of the VETER templates with predefined states such as transformations, destinations, etc. based on your login and subscription.
Note, that this portal is not included in this article.
[1] Announcing the Refresh of Service Bus EAI & EDI Labs
[2] Service Bus EAI and EDI Labs - April 2012 Release
[3] An Introduction to EAI Bridges
[4] WF Azure Activity Pack CTP 1
[5] Using Windows Azure Service Bus Messaging
[6] Setting and Retrieving Properties and Metadata for Blob Resources (REST API)
[7] Windows Azure Service Bus EAI & EDI
[8] How to: Use a Custom User Name and Password Validator
[9] How to Configure an SSL Certificate on an HTTPS Endpoint
[10] Service Bus Explorer
[11] Azure Service Bus Tester
[12] Azure Storage Explorer
