Introduction
Some time ago, I was developing a solution using Biztalk server and found very little information about how to call a web-service (or any other process) asynchronously. After doing some search in the Biztalk documentation, I found an interesting solution that consisted of two orchestrations and a port that is passed from one orchestration to the other as a parameter. To illustrate this solution, I'll use a hypothetical situation using the Northwind database.
The scenario
Let's say that you are working for a big company, and the company works with a legacy system to process orders. The company is now planning to replace the system, but it will take 1 year until the new system is ready to be run. The company is currently using Biztalk as its integration platform, and you are working as a Biztalk developer. Since your company needs to expand its business and can't wait until the new system is ready, your boss comes to you with a new task.
"Our querying support to our legacy system is very limited. Today our salesperson can only query orders from a single day, and not from a range of dates. This information is critical to us because in order to increase our sales, our sales staff needs to have information about an entire month or even an entire year, in order to analyze trends. I need you to create a Biztalk process capable of receiving a range of dates and querying our legacy system for each day in that range. This biztalk process will return all the orders from that range as a result".
Sounds like an easy task to Biztalk, isn't it ? Lucky for you, the guys from the legacy system have created a web service layer so that you can query orders from a specific day in the legacy system.
To simulate the legacy system's web service, I have created a simple web service that queries the information in the Northwind database. The source code of the web service is available in the article.
The architecture
Before we start constructing the biztalk solution, we need to understand how our order querying process will work. The diagram below shows the how the solution will work:
The process starts when the end user application sends an XML request to the Biztalk server. This request will have the date range that we need to query in our legacy system. When the message hits biztalk, it spawns one instance of another orchestration for each date in that range (in the diagram there is an orchestration instance to 7/1/1996, 7/2/1992, 7/3/1996 and 7/4/1996). This second orchestration uses the web service from the legacy system to query the orders for a specific date. Note that all such orchestrations are spawned asynchronously. As soon as each orchestration finishes its processing, it will return the response to the main orchestration. Note that as soon as the loop finishes in the main orchestration, we start another loop, waiting for the responses of each orchestration that we have spawned. After we receive all the messages, we aggregate them into a single one and return to the end user application. Using this approach, the time that it takes to query a range of dates will be nearly the same as querying a single day, since our orchestration spawns a new query to the legacy system at the same time for each day.
Since we don't have an end user application, we'll use XML files to simulate the end user application.
Creating the project
We'll start by creating our biztalk project. Start Visual Studio and create a new Biztalk server project named BTSOrderQuery. You need to add two references to the project. The first is a reference to the BTSUtils library. This library contains a generic class that is used to aggregate XML messages. You can find this project in the ZIP file available with the article. The other reference is a web reference to the web service of our legacy system (also available for download in a separate .zip file). Just click on references, add a web-reference and type the URL where you have installed the web-service (in my case it is http://localhost/legacyorder/orderquery.asmx).
Creating the schema
The first item we will create in our project is the schema used to send a query request to the Biztalk server. Right click on the project in the Solution Explorer and select the "Add New Item..." option. Select schema as the item type and name it as OrderRequest.xsd.
Open the schema that you have just created (if it's not already open) and change its "Target Namespace" property to "http://nwtraders" (click on <Schema> to access this property). After changing the target namespace, change the "Node Name" property of the root element to OrderRequest.
Now, add two child fields in the root element (right click on the "OrderRequest" element and select "Insert... Child Field Element") named StartDate and EndDate. Change the data type of both these elements to xs:dateTime.
To finish our schema, right-click on StartDate, select the "Promote" option and then "Show Promotions". Make sure that "StartDate" is selected on the left side of the screen and click "Add". Now, select the "EndDate" field and click "Add" again. The screen should look like the one shown below. Click OK to dismiss this dialog box:
Promoting these fields as "Distinguished Fields" will allow us to access them inside the orchestration code. At the end of the process, your schema should look like the one shown below:
Creating the main orchestration
Now that we have our schema and the references, we're going to create the main orchestration. Right click on the project, select the "Add New Item..." option and select "Orchestration". Name it "OrderRequestMain".
Open the Orchestration View window (if you don't see the orchestration view window, go to the "View" menu, "Other Windows", and select "Orchestration View") and right click on the messages folder, and select the "New Message" option. Change the name of the message to "msgOrderRequest" and change the "Message Type" property to "Schemas...", "BTSOrderQuery.OrderRequest". Create another message named "msgOrders" and select as its message type "Web Message Type", "BTSOrderQuery.localhost.OrderQuery_.GetOrders_response".
From the toolbox, drag a receive shape to the orchestration designer area and name it "RcvOrderQuery". Change the message property to "msgOrderRequest" and the Activate property to "True".
In the orchestration view, right click on the variables folder and select "New Variable". Name the variable as "oAgreg" and select as its data-type ".NET Class", "BTSUtils.Aggregator". We will use this class to aggregate the results from all the queries into one message. Create another variable dtDate of type datetime. Drag an expression shape below the receive shape, and enter the following expression on it. This will initialize our variable for the first date we'll query. Drag a loop shape to the orchestration designer, below the expression shape. Change its expression property to the following expression:
System.DateTime.Compare(dtDate, msgOrderRequest.EndDate) <= 0
Now add a "Start Orchestration" shape inside the loop shape and an "Expression" shape below it (inside the loop shape as well). In the expression shape, add the following expression:
dtDate = dtDate.AddDays(1);
Copy the first expression shape in the orchestration and paste it just below our loop (this will reset the date variable to our new loop). Now drag another loop shape and use the same expression property that we used in the other loop. Inside this loop, add a receive shape and an expression shape. In the receive shape, select msgOrders as its message property. Inside the expression shape, add the following code:
oAgreg.Aggregate(msgOrders.GetOrdersResult,
"/ns0:Result/ns0:Orders", "/ns0:Result/ns0:Orders/ns0:Order");
The expression above uses the generic aggregator to combine the results of each day into one message. To finish our main orchestration, drag a "Message Construct" shape below the last loop we've created. Select the msgOrders as the message to be constructed. Inside this "Message Construct", drag a "Message Assignement" shape and add the following code:
msgOrders.GetOrdersResult = oAgreg.GetAggregatedDocument();
The GetAggregatedDocument returns an aggregated version of the documents passed as parameter in the Aggregate method call. Now that we have our aggregated message, we need to send it to the end user. Drag a send shape below the message construct and select its message property to msgOrders.
At this point, your orchestration should look like the picture below:
Creating the ports
Now, we need to create the ports to receive and send our messages. Right click on the port surface on the left side of the orchestration designed and select "New Configured Port". Click next on the first screen. In the next screen, type "rpIN" as the name of the receive port and click next.
In the next screen, type "rpINType" as the port type and click next. Make sure that the "I'll always be receiving..." option is selected, click next and then finish. A new port will be created. Connect this port to the first receive shape in the orchestration.
Let's create the second port (the one that will receive the messages from the other orchestration). Right click on the port surface, now on the right side of the orchestration and select "New Configured Port". Click next on the first Screen. In the next screen, type "rpOrders" as the name of the receive port and click next. In the next screen, type "rpOrdersType" as the port type and click next. Make sure that the "I'll always be receiving..." option is selected, and select "Direct" as the "Port Binding" option. In the direct binding options, select "Self Correlating". Click next and then finish. We use the self correlating option here biztalk, we're going to pass this port as a parameter to our other orchestration (yeikes!), and by selecting this option we make sure that the response will be directed to the correct orchestration. Connect this port to the receive shape inside the second loop.
Now for the last port. Right click on the port surface on the left side of the orchestration designed and select "New Configured Port". Click next on the first screen. In the next screen, type "spOUT" as the name of the receive port and click next. In the next screen, type "spOUTType" as the port type and click next. Make sure that the "I'll always be sending..." option is selected, click next and then finish. A new port will be created. Connect this port to the last send shape in the orchestration.
After creating the ports, our orchestration should look like this.
Creating the second orchestration
The second orchestration is responsible for querying our legacy system's web service for a specific date and returning the result to the main orchestration. This second orchestration is simpler than the first one. Let's start by creating a new orchestration file. In the Solution Explorer, right click on the biztalk project and select "Add New Item...". Select "Orchestration" and name it "BTSQueryLegacy".
As opposed to other orchestrations, this one will be activated by parameters and not by a receive shape. Go to the orchestration view and right click on the "Orchestration Parameters" option. Select "New Variable Parameter". Change the parameter's name property to dtOrder and its datatype to "datetime".
In order to call the legacy web service, we need to create request and response messages. Create messages in the "Orchestration View" window called msgReq and msgResp. For the request message, select "BTSOrderQuery.localhost.OrderQuery_.GetOrders_request", from "Web Message Types" as its "Message Type" property. For the response message, select "BTSOrderQuery.localhost.OrderQuery_.GetOrders_response", from "Web Message Types" as its "Message Type" property. Another step needed to call our web service is to create a web port in the orchestration. Drag a "Port" shape from the tool box to the right side of the port surface area. In the port creation wizard, click next. Type "wsLegacy" as the port name, and click next. In the next screen, select "Use Existing Port Type", expand "Web Port Types" and select "BTSQuery.localhost.OrderQuery_.OrderQuery". Click next twice and then finish.
Now let's add some shape to the orchestration. Drag a message construct shape to the orchestration and select the "msgReq" message as the message to be constructed. Drag a "Message Assignment" shape inside the construct shape and type the following expression on it:
msgReq.date = dtOrder;
This will create a request message with the order date received as parameter. Now drag a send shape below the message construct and select "msgReq" as its message. Link the send shape to the web port on the right of port surface area. Drag a receive shape below the send shape and select "msgResp" as its message. Link it to the web port (in the response slot) as well.
Using a port parameter
Now we need to send the response back to the main orchestration. To to this, we'll use a port parameter. Select the "New Configured Port Parameter", click next on the first screen. In the next screen, select spOrders as the port parameter name, and click next. In the next screen, select the "Use an existing port type" option and select "BTSOrderQuery.rpOrdersType" as the port type. Now, click next. In the next screen, change the "Port Direction" option to "I'll always be sending messages from this port".
Passing a port as a parameter to another orchestration is the only valid situation where you can change the "polarity" of the port, change it from a receive port to a send port. The concept used here is very similar to a callback function. We pass a receive port from the main orchestration to the specific orchestration (callback function). When the specific orchestration finishes its work, it sends its response to the port received as a parameter (call the callback function to notify the caller).
Drag a send shape below the last receive shape in the orchestration and select msgResp as its message. Connect this shape to the port received as a parameter (spOrders). At the end of this process, your orchestration should look like the one shown below:
Final steps
The only thing that is missing is a property in one specific shape in the main orchestration, the start orchestration shape. Open the main orchestration and find the start orchestration shape. In the "Called Orchestration" property, select "BTSOrderQuery.BTSQueryLegacy". In the Parameters option click on the (...) button. A new window will be opened, and now the designer can pass the required parameters. Just click OK to dismiss the dialog.
In order to test our solution, you need to create a strong name for the biztalk assembly. You can do this by using the sn.exe tool. After doing this, deploy the solution to your biztalk server and then create receive and send ports using the FILE adapter in order to test our solution. I used the following message to test the solution:
<ns0:OrderRequest xmlns:ns0="http://nwtraders">
<StartDate<1996-07-04T00:00:00.000-00:00</StartDate>
<EndDate<1996-07-10T00:00:00.000-00:00</EndDate>
&</ns0:OrderRequest>
The result
If you drop a test message in your file receive port (you can use any date between 7/1/1996 and 7/30/1996), biztalk will drop in the out folder a message with the consolidated result. If you check Health and Activity Monitoring (HAT) when you drop the file, you'll see that the main orchestration really spawns another orchestration for each date in your Order Query request, as shown in the image below:
And that's how you can make asynchronous calls using biztalk. Using two orchestrations and passing a port as parameter solves all the problems. Hope you liked the article! :)
