Introduction
The way I previously created messages to send to the GovTalk service is I used the XMLDocument to create the request. While this worked, it left a number of problems; not least that for every message a special function would need to created. This is OK for the short term, but the biggest cost in any software project is maintenance and this would be a headache to maintain.
So the following is a somewhat better way of achieving the same thing. For the purposes of this article, I am going to be using the CompanyNumberSearch request of the GovTalk service – although this technique would work for any service that accepted XML. The C# functions which send and receive the messages remain the same. The magic sauce in this is the XSLT which defines the structure of the request, and the use of objects in conjunction with reflection to provide the content. It is a bit like Sweet Chilli Sauce added to Chicken on a bed of rice. So on to the Sweet Chilli Sauce.
The Sweet Chilli Sauce
The request to search for a company based on its number is as follows:
<GovTalkMessage xsi:schemaLocation="http://www.govtalk.gov.uk/CM/envelope
http://xmlgw.companieshouse.gov.uk/v1-0/schema/Egov_ch-v2-0.xsd"
xmlns="http://www.govtalk.gov.uk/CM/envelope" xmlns:dsig="http://www.w3.org/2000/09/xmldsig#"
xmlns:gt="http://www.govtalk.gov.uk/schemas/govtalk/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >
<EnvelopeVersion>1.0</EnvelopeVersion>
<Header>
<MessageDetails>
<Class>NumberSearch</Class>
<Qualifier>request</Qualifier>
<TransactionID>1</TransactionID>
</MessageDetails>
<SenderDetails>
<IDAuthentication>
<SenderID>????????????????????????????????</SenderID>
<Authentication>
<Method>CHMD5</Method>
<Value>????????????????????????????????</Value>
</Authentication>
</IDAuthentication>
</SenderDetails>
</Header>
<GovTalkDetails>
<Keys/>
</GovTalkDetails>
<Body>
<NumberSearchRequest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://xmlgw.companieshouse.gov.uk/v1-0/schema/NumberSearch.xsd">
<PartialCompanyNumber>99999999</PartialCompanyNumber>
<DataSet>LIVE</DataSet>
<SearchRows>1</SearchRows>
</NumberSearchRequest>
</Body>
</GovTalkMessage>
This is the XML that we send to the GovTalk Service and we get back a list of companies that match the criteria passed.
A message is structured in two parts; The envelope which identifies the person sending the request, with the name of the request, and the body which gives the detail of the company we are looking for.
The Chilli
What makes it possible is the use of XSLT to define the message – and serialization to convert each request object into XML. To start, we need to create an object which will represent the contents of the message we are sending. However, there are common properties in all the messages that we send to Companies House. These properties are as follows:
SenderId – the id of the person sending the messageSenderPassword – the password associated with IdTransactionId – unique identifier for the messageAuthenticationValue – authenticates the request
Because these properties are unique to the Companies House message, and because they are shared with all messages, they are perfect candidates for a base class.
The class is as follows:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Security.Cryptography;
using System.Text;
using System.Text.RegularExpressions;
using Microsoft.WindowsAzure.ServiceRuntime;
namespace CompanyHub.Services
{
public class GovTalkRequest
{
public GovTalkRequest()
{
try
{
SenderID = RoleEnvironment.GetConfigurationSettingValue("SenderId");
SenderPassword = RoleEnvironment.GetConfigurationSettingValue("SenderPassword");
TransactionId = DateTime.Now.Ticks.ToString();
AuthenticationValue = EncodePassword(String.Format
("{0}{1}{2}", SenderID, SenderPassword, TransactionId));
}
catch (System.Exception ex)
{
throw ex;
}
}
public String SenderID
{
get;
set;
}
public String SenderPassword
{
get;
set;
}
public String TransactionId
{
get;
set;
}
public String AuthenticationValue
{
get;
set;
}
public static String EncodePassword(String clearPassword)
{
MD5CryptoServiceProvider md5Hasher = new MD5CryptoServiceProvider();
byte[] hashedBytes;
UTF32Encoding encoder = new UTF32Encoding();
hashedBytes = md5Hasher.ComputeHash(ASCIIEncoding.Default.GetBytes(clearPassword));
String result = Regex.Replace(BitConverter.ToString(hashedBytes), "-", "").ToLower();
return result;
}
}
}
There is nothing particularly clever here, except for the EncodePassword method which hashes the value made up of the SenderId, Password and Transaction id.
Each message inherits from this object. So for the Company Number Search, in addition to the properties above, we need a partial number, which dataset to search – for the purposes of the project, we only need to search the LIVE set so this can be set in the constructor and the SearchRows. Again, all are set as properties. With the SearchRows and DataSet initialized in the constructor.
public class CompanyNumberSearchRequest : GovTalkRequest, IDisposable
{
public CompanyNumberSearchRequest() : base()
{
DataSet = "LIVE";
SearchRows = 1;
}
public String PartialCompanyNumber { get; set; }
public String DataSet { get; set; }
public int SearchRows { get; set; }
public void Dispose()
{
DataSet = String.Empty;
PartialCompanyNumber = String.Empty;
DataSet = "LIVE";
SearchRows = 1;
}
}
As well as inheriting from our base class, I have also inherited from IDisposable – not just because it is just plain good practice to dispose of objects when coding, but it gives also gives us more versatility when using the object.
There are four stages in making a request and this is reflected in the four methods we execute in making a call to the Companies House service:
- Create a request
- Send a request
- Check the status
- If OK, then get the results of the request
I’ve implemented each of these stages within a static class called Toolbox – which also means I don’t need to create an instance of the class to use it.
When making a request, there are three stages:
- Get the template for the message
- Serialize the object representing the message
- Transform the serialized object using a predefined XSLT file
Each of my templates I have defined as an embedded resource. When retrieving a resource of this kind, we have to include the full namespace to the resource. In making the code re-usable as much as possible, I defined the full ‘path’ within the GetRequest method.
requestFile = String.Format("CompanyHub.Services.Schemas.{0}", RequestFile);
So we now have the full path of the file within the assembly. Now all we need do is retrieve the assembly and get the resource.
asm = Assembly.GetExecutingAssembly();
sr = asm.GetManifestResourceStream(requestFile);
Once retrieved
So this can be returned to the calling function and we now have a stream of XSLT to define the message. Time now to serialize the request to create the other side of this message.
t = Obj.GetType();
ms = new MemoryStream();
serializer = new XmlSerializer(t);
xmlTextWriter = new XmlTextWriter(ms, Encoding.ASCII);
serializer.Serialize(xmlTextWriter, Obj);
ms = (MemoryStream)xmlTextWriter.BaseStream;
GovTalkRequest = Toolbox.ConvertByteArrayToString(ms.ToArray());
First off, we need the type of the object so we make a call to the GetType method of the object containing the Message properties.
Next, we need a MemoryStream, XmlSerializer and an XMLTextWriter so these can be initialized.
The object is serialized by making the call to the Serialize method of the serializer object. The result of that is then converted into a MemoryStream. That MemoryStream is then converted into a string.
ConvertByteArrayToString
This is a fairly simple function which uses an ASCIIEncoding object found within the System.Text namespace to convert an array of bytes into a string.
public static String ConvertByteArrayToString(byte[] bytes)
{
System.Text.ASCIIEncoding enc = new System.Text.ASCIIEncoding();
return enc.GetString(bytes);
}
I only put it into a function because I will be using this in various places.
The Sauce
When adding support for other messages outside of creating a new object to store the properties of the message, the C# components do not need to change. It is in the XSLT file that the versatility of the technique lies. The XSLT file determines the format of the message. For the CompanyNumberSearch, the XSLT file is as follows:
="1.0"
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
<GovTalkMessage xsi:schemaLocation="http://www.govtalk.gov.uk/CM/envelope
http://xmlgw.companieshouse.gov.uk/v1-0/schema/Egov_ch-v2-0.xsd"
xmlns="http://www.govtalk.gov.uk/CM/envelope"
xmlns:dsig="http://www.w3.org/2000/09/xmldsig#"
xmlns:gt="http://www.govtalk.gov.uk/schemas/govtalk/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >
<EnvelopeVersion>1.0</EnvelopeVersion>
<Header>
<MessageDetails>
<Class>NumberSearch</Class>
<Qualifier>request</Qualifier>
<TransactionID>
<xsl:value-of select="CompanyNumberSearchRequest/TransactionId"/>
</TransactionID>
</MessageDetails>
<SenderDetails>
<IDAuthentication>
<SenderID><xsl:value-of select="CompanyNumberSearchRequest/SenderID"/>
</SenderID>
<Authentication>
<Method>CHMD5</Method>
<Value>
<xsl:value-of select="CompanyNumberSearchRequest/AuthenticationValue"/>
</Value>
</Authentication>
</IDAuthentication>
</SenderDetails>
</Header>
<GovTalkDetails>
<Keys/>
</GovTalkDetails>
<Body>
<NumberSearchRequest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation=
"http://xmlgw.companieshouse.gov.uk/v1-0/schema/NumberSearch.xsd">
<PartialCompanyNumber>
<xsl:value-of select="CompanyNumberSearchRequest/PartialCompanyNumber"/>
</PartialCompanyNumber>
<DataSet>
<xsl:value-of select="CompanyNumberSearchRequest/DataSet"/>
</DataSet>
<SearchRows>
<xsl:value-of select="CompanyNumberSearchRequest/SearchRows"/>
</SearchRows>
</NumberSearchRequest>
</Body>
</GovTalkMessage>
</xsl:template>
</xsl:stylesheet>
The outer two tags define that this is a XSLT stylesheet and the root tag from which the nodes are searched for.
The GovTalkMessage is the format of the message that will be sent to Companies House.
We first set up the XslCompiledTransform object which will transform the XSLT template and the serialized object into the request to Companies House.
xslt = new XslCompiledTransform();
resultStream = new MemoryStream();
writer = new XmlTextWriter(resultStream, Encoding.ASCII);
doc = new XmlDocument();
The Serialize method requires XmlTextWriter to write the XML (writer) and a stream to place the transferred object into (writer). The XML will be loaded into an XMLDocument object (doc) prior to the transformation.
xslTemplate = Toolbox.GetRequest(Template);
xslTemplate.Seek(0, SeekOrigin.Begin);
templateReader = XmlReader.Create(xslTemplate);
xslt.Load(templateReader);
I have stored all the templates as a series of Embedded Resources and the GetRequestCall takes the name of the template and extracts the relevant XSLT file.
public static Stream GetRequest(String RequestFile)
{
String requestFile = String.Empty;
Stream sr = null;
Assembly asm = null;
try
{
requestFile = String.Format("CompanyHub.Services.Schemas.{0}", RequestFile);
asm = Assembly.GetExecutingAssembly();
sr = asm.GetManifestResourceStream(requestFile);
}
catch (Exception)
{
throw;
}
finally
{
asm = null;
}
return sr;
}
We first take the template name and expand it to include the full namespace to the Embedded Resource I like to keep all my schemas in the same directory and so the namespace reflects this. The rest is the default namespace for the project.
Then we get the currently executing assembly (which will contain the resources with the call to GetExecutingAssembly() )
Finally, we get a stream which contains the XSLT file. We use this stream and then load an XmlReader with the contents of the template, and that is in turn loaded into the XslCompiledTransform object.
We convert the object containing the message properties into XML by serializing it; calling the Serialize() method of the XmlSerializer object. To set up the object, we do the following:
t = Obj.GetType();
ms = new MemoryStream();
serializer = new XmlSerializer(t);
xmlTextWriter = new XmlTextWriter(ms, Encoding.ASCII);
We first determine the type of the object being transferred by calling GetType(). We create an XmlSerializer object by passing the type of the object being serialized. The serializer writes to a memory stream and that is linked to an XmlTextWriter.
Next job is to serialize the object and load it into an XmlDocument.
serializer.Serialize(xmlTextWriter, Obj);
ms = (MemoryStream)xmlTextWriter.BaseStream;
xmlRequest = new XmlTextReader(ms);
GovTalkRequest = Toolbox.ConvertByteArrayToString(ms.ToArray());
doc.LoadXml(GovTalkRequest);
Time to transform the XML to construct the full request.
xslt.Transform(doc, writer);
resultStream.Seek(0, SeekOrigin.Begin);
request = Toolbox.ConvertByteArrayToString(resultStream.ToArray());
So that creates the full request to be sent to Companies House.
Sending the Request
So far, we have a string with a request for the Companies House service. Now, we need to send the request to the Companies House Service.
Configuration Within an Azure Project
There are entire blog entries written about configuration within an Azure project – most of this is out of scope for this article but the following is a summary.
Configuration is defined in two files within the parent project *.csdef which contains the definition of configuration setting.
="1.0"="utf-8"
<ServiceDefinition name="OnlineCompanyHub"
xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
<WebRole name="CompanyHub.Host">
<InputEndpoints>
<InputEndpoint name="HttpIn" protocol="http" port="80" />
</InputEndpoints>
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString" />
<Setting name="DataConnectionString" />
</ConfigurationSettings>
</WebRole>
<WebRole name="CompanyHub.Services">
<InputEndpoints>
<InputEndpoint name="HttpIn" protocol="http" port="8080" />
</InputEndpoints>
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString" />
<Setting name="SenderId"/>
<Setting name="SenderPassword" />
<Setting name="GovTalkUrl"/>
</ConfigurationSettings>
</WebRole>
<WorkerRole name="CompanyHub.Worker">
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString" />
</ConfigurationSettings>
</WorkerRole>
</ServiceDefinition>
Above is the configuration definition from the project. What we are interested in however is the ConfigurationSettings tag of the CompanyHub.Services WebRole.
There are four configuration settings here, but at the moment, we are interested in the second to fourth settings; SenderId, SenderPassword and GovTalkUrl.
The value of these settings are defined in the ServiceDefinition.cscfg file:
="1.0"
<ServiceConfiguration serviceName="OnlineCompanyHub"
xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">
<Role name="CompanyHub.Host">
<Instances count="2" />
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString"
value="UseDevelopmentStorage=true" />
<Setting name="DataConnectionString"
value="UseDevelopmentStorage=true" />
</ConfigurationSettings>
</Role>
<Role name="CompanyHub.Services">
<Instances count="2" />
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString"
value="UseDevelopmentStorage=true" />
<Setting name="SenderId" value="UserID"/>
<Setting name="SenderPassword" value="Password"/>
<Setting name="GovTalkUrl"
value="http://xmlgw.companieshouse.gov.uk/v1-0/xmlgw/Gateway"/>
</ConfigurationSettings>
</Role>
<Role name="CompanyHub.Worker">
<Instances count="2" />
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString"
value="UseDevelopmentStorage=true" />
</ConfigurationSettings>
</Role>
</ServiceConfiguration>
Look for the Role tag that contains our project name (CompanyHub.Services).
Having configured the parameters, we can now transmit the request. This is done by ‘POST’ing a stream of XML to the Companies House servers.
govTalkUrl = RoleEnvironment.GetConfigurationSettingValue("GovTalkUrl");
request = WebRequest.Create(govTalkUrl);
request.Method = "POST";
request.ContentType = "text/xml";
writer = new StreamWriter(request.GetRequestStream());
writer.WriteLine(RequestMessage);
writer.Close();
We use the WebRequest object to send the object. Set the method of sending to ‘POST’ and the type of data as text/xml. Once set up, all we do is write the request to the writer – this sends the request to Companies House.
Did the Request Work Part I – Getting the Response
Having sent a request – we now need the result of that request.
response = request.GetResponse();
reader = response.GetResponseStream();
result = Toolbox.ConvertByteArrayToString(Toolbox.ReadFully(reader));
The WebRequest object has a GetResponse() method which allows us to get the response sent back. Like many of these calls, the results come in the form of a stream which we convert into a string.
Did the Request Work Part II – Translating the Response
Much like XSLT and XML were used to create the original request, so it can be used to extract the response and by deserializing the result, we create an object that contains the response.
Did It Work?
It would be really great if everything worked all the time. Of course, if it did, then I don’t suppose people would pay me and others the big bucks so that our programs do not:
- Collapse in a heap (this is an area of memory)
- Blow every fuse in the place in a shower of sparks (this will probably not happen this being real life and not a Hollywood movie, but it was possible to blow the sound system of a BBC Model B with a poorly coded setting)
- Go nuts and trap everyone outside the airlock (this was from a movie, and unless NASA get a manned moon/mars mission set up unlikely to happen)
- Go nuts and take over the world (this was also from a movie, but please note life has a habit of being of exceeding the wildest imaginations of Hollywood writers (note writers – Hollywood executives have no imagination and judging by recent output of that town have turned plagiarism into an art form).
- Freeze in total confusion because the cleaner pulled the plug to the internet router (this has happened)
So anyway – we need to check to see if our request actually worked. Within the GovTalk response, there is a section that details the status of the message and a description of what went wrong (if anything did). I have defined an XSLT template which will extract these into an XML document.
="1.0"
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:ev="http://www.govtalk.gov.uk/CM/envelope"
xmlns:gt="http://www.govtalk.gov.uk/schemas/govtalk/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<xsl:template match="/">
<GovTalkStatus xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Status>
<xsl:value-of select="ev:GovTalkMessage/ev:Header/ev:MessageDetails/ev:Qualifier"/>
</Status>
<Text>
<xsl:value-of select=
"ev:GovTalkMessage/ev:GovTalkDetails/ev:GovTalkErrors/ev:Error/ev:Text"/>
</Text>
<Location>
<xsl:value-of select=
"ev:GovTalkMessage/ev:GovTalkDetails/ev:GovTalkErrors/ev:Error/ev:Location"/>
</Location>
<Number>
<xsl:value-of select=
"ev:GovTalkMessage/ev:GovTalkDetails/ev:GovTalkErrors/ev:Error/ev:Number"/>
</Number>
<Type>
<xsl:value-of select=
"ev:GovTalkMessage/ev:GovTalkDetails/ev:GovTalkErrors/ev:Error/ev:Type"/>
</Type>
</GovTalkStatus>
</xsl:template>
</xsl:stylesheet>
Only thing different about previous XSL files is the references to two namespaces ev & gt. These are defined in the GovTalk response at the top of the response;
xsi:schemaLocation="http://www.govtalk.gov.uk/CM/envelope
http://xmlgw.companieshouse.gov.uk/v1-0/schema/Egov_ch-v2-0.xsd"
xmlns="http://www.govtalk.gov.uk/CM/envelope"
xmlns:dsig="http://www.w3.org/2000/09/xmldsig#"
xmlns:gt="http://www.govtalk.gov.uk/schemas/govtalk/core"
If we do not put these references into the XSLT template, then the XslCompiledTransform object will not be able to find the relevant tags.
Deserialization is a fairly simple activity.
encoder = new ASCIIEncoding();
ms = new MemoryStream(encoder.GetBytes(statusXML));
serializer = new XmlSerializer(typeof(GovTalkStatus));
xmlTextWriter = new XmlTextWriter(ms, Encoding.ASCII);
messageStatus = (GovTalkStatus)serializer.Deserialize(ms);
We set up a serialization object using the object type containing the error state and pass to it the results of a transformation between the XSLT above and the GovTalk response.
Now we have an object containing any error state, and the error message. All we need to do is check the status. If there is an error, then we can flag an error. If not, then we extract the results and pass that as an object back to the calling function. We go about this by guess what – defining an XSLT template for the result and using that to create an XML Stream which can be deserialized into a .NET object. In this instance, the XSLT to create the result of a Company Number Search is:
="1.0"="us-ascii"
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:ev="http://www.govtalk.gov.uk/CM/envelope"
xmlns:sch="http://xmlgw.companieshouse.gov.uk/v1-0/schema"
exclude-result-prefixes="ev">
<xsl:template match="/">
<CompanySearchResult xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<CompanyNumber>
<xsl:value-of select=
"ev:GovTalkMessage/ev:Body/sch:NumberSearch/sch:CoSearchItem/sch:CompanyNumber"/>
</CompanyNumber>
<CompanyName>
<xsl:value-of select=
"ev:GovTalkMessage/ev:Body/sch:NumberSearch/sch:CoSearchItem/sch:CompanyName"/>
</CompanyName>
</CompanySearchResult>
</xsl:template>
</xsl:stylesheet>
and the object definition is:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
namespace CompanyHub.Services
{
public class CompanySearchResult
{
public CompanySearchResult()
{
CompanyNumber = String.Empty;
CompanyName = String.Empty;
}
public String CompanyNumber { get; set; }
public String CompanyName { get; set; }
}
}
Our entire code to make calls to send a request, and interpret the results are:
String request = String.Empty;
String response = String.Empty;
GovTalkStatus status = null;
fault = null;
try
{
using (CompanyNumberSearchRequest requestObj = new CompanyNumberSearchRequest())
{
requestObj.PartialCompanyNumber = CompanyNumber;
request = Toolbox.CreateRequest(requestObj, "CompanyNumberSearch.xsl");
response = Toolbox.SendGovTalkRequest(request);
status = Toolbox.GetMessageStatus(response);
if (status.Status.ToLower() == "error")
{
fault = new HubFault()
{
Message = status.Text
};
}
else
{
Object obj = Toolbox.GetGovTalkResponse
(response, "CompanyNumberSearchResult.xsl", typeof(CompanySearchResult));
}
}
}
catch (FaultException<ArgumentException> ex)
{
fault = new HubFault()
{
FaultType = ex.Detail.GetType().FullName,
Message = ex.Detail.Message
};
}
catch (System.Exception ex)
{
fault = new HubFault()
{
FaultType = ex.GetType().FullName,
Message = ex.Message
};
}
finally
{
}
Wrap Up
So there we have it – a reusable set of functions to send and interpret XML results from an internet based service. The code is reusable with a little change with any service which uses XML as a transport mechanism – and as for the Companies House GovTalk service, all I need to do is create various objects for the result and message sent and the relevant XSLT files. I might need minor changes for other services, but something like 70-90% will be exactly the same.
