Introduction
This article tries to take an in-depth look at how WCF Services can be integrated into an enterprise SOA environment. The SOA is based on a Microsoft UDDI V2 registry that is shipped with Windows Server 2003 and 2008. This article covers the aspects of WCF UDDI integration as well as the creation of dynamic WCF proxies.
Background
Is WCF out of the box a Service Oriented Architecture?
If you think that WCF is an SOA, then let's review part of the definition of "Service Oriented Architecture". OASIS (the Organization for the Advancement of Structured Information Standards) defines SOA as: "A paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains. It provides a uniform means to offer, discover, interact with, and use capabilities to produce the desired effects consistent with measurable preconditions and expectations." Now, let's translate "capability" with "service". So, we get in slightly other words: "An SOA consists of distributed services, some sort of mechanism to offer and find them, a mechanism to interact with the service, and finally, a mechanism to invoke the service." If you try to break down this into web service technology terms, you now translate this into:
- Services (this is what WCF is all about)
- Find and Offer (this could be a UDDI Registry (Enterprise), or WS-Discovery (UPnP like))
- Interact (this could be WSDL)
- Invoke (this could be SOAP)
So, what part of SOA is covered by WCF?
The service, the interaction, and the service invocation. The runtime discovery is not part of WCF.
Conclusion
WCF offers no proper mechanism to offer and find services at runtime, and hence is a classical 3-tier architecture technology that we basically have has for 15-20 years now. We just called it DCOM or whatever before, and now the same technology gets relabeled with the current buzzword "Service".
Just a service is not an SOA.
WCF follows Web Services
WCF basically combines all the different communication technologies of .NET 2.0 (Remoting, Queues etc.) under one roof and a common programming model. The model itself follows the Web Services model introduced with .NET 2.0. One of the biggest disadvantages of Web Services was that the communication layer for Web Services was pretty much fixed to the WS-I standard (basic HTTP binding in WCF terms). What Microsoft now basically did for WCF was to address this issue and introduce a completely flexible communication layer. So, instead of a pre-assigned WS-I protocol, we can now choose from a wide array of communication protocols that nearly fit every situation. But, from a programmer's point of view, the implementation of WCF services/clients follows pretty much the way we programmed Web Services. On the client side, instead of a "Web Service reference", you now just add a "service reference" to your project, but then, it feels just like before. On the service side, you now have to define a service interface, and you use some other attributes, but then again, it feels like being with Web Services. But, this flexibility has some implications.
- With every service reference, you now add a configuration file with a "
ServiceModel" section that keeps all the protocol specific configuration options. - The path to the WSDL for Web Services was pretty much always http://ServiceAddress?WSDL. Now, with non HTTP protocols, there must be another way to access the Service metadata. So, with WCF, the "Mex" or "MetadataExchange" endpoint, that is supposed to be on the "ServiceURL/mex" address, was introduced.
Target Enterprise Architecture
Take a look at the following scenario: I have 10 different types of Services running, with 100 clients each. An hour of unscheduled service downtime during day hours can cost me up to a 100 million bucks.
So what are my requirements?
- I must be able to relocate my services without any clients failing.
- Services must be redundant and clients must support failover.
- I must be able to dynamically add more services of the same type for load balancing and fault-safety.
- I want to have a quick overview of where my services are running.
- I must be able to change communication parameters (e.g., enable transport encryption on the service) without any of my clients failing.
Does standard WCF implementations meet our requirements?
By "standard", I refer to the way the service and the client has been created. Means, setting up the service, then creating a client, adding the service reference to the service, and then calling the service function of interest. What happens if we:
As a result, for every above change, I would need to update 100 hundred client config files, but I have no clue on what workstations the clients are located. As a result, any above change breaks my production process, which is not acceptable in an enterprise environment.
Conclusion
The standard implementation of WCF is not suited for use in enterprise environments.
Introducing a central Service Registry to our Architecture
As a mechanism to find and offer Web Services in an enterprise environment, this rings the bell for UDDI which is included since Windows Server 2003, but which seems to be fairly unnoticed so far. If I say documentation and support is sparse, this is quite an extenuation.
Web Services in a UDDI Environment
Handling web services in a UDDI registry environment is pretty much straight-forward. You simply strip everything but the Types, Message, and Operations sections out of your WSDL, publish it as a model, and then you add the service endpoints to the registry (where there is just one per Service). As a client, you add the Web Service reference to the stripped WSDL, then at runtime, you simply put in the address from the UDDI, and everything is supposed to work just fine as the communication layer is fixed to WS-I and there is just one endpoint.
WCF Services in a UDDI Environment
With WCF, we have a whole bunch of possibilities on how our service can communicate. This includes the option that a service has multiple endpoints offering different communication protocols at the same time. If we have multiple instances of a service, we no longer can assume that they will all use the same communication protocols. But, if we try to handle WCF Services with UDDI just like Web Services, then what happens?
First, we strip everything but the Types, Message, and Operations sections from our WSDL. This is where you stop for the first time... Looking at the WSDL you will find out that the "Types" section is not declared inline, but every namespace just has an import reference. You have three options now.
- You open all the import URLs in a browser, and then you replace the imports with the results (sucks like hell after the tenth time), or
- You modify the WSDL behavior of your service with, e.g., FlatWSDL. But, this is not as good as it sounds, as there is a bug in Visual Studio that does not let you choose your array types any more under some circumstances. So, if your Service returns an array of
T, you could configure it to be a List<t> in your code, but under some circumstances, Visual Studio now will not let you change the array type to be anything but array. This is quite nasty as the WSDL is perfectly valid and the type is perfectly recognized as T[], but you just will not get is as List<T> anymore, no matter what you select as your ArrayType. - You skip this step and you develop against a running instance of your service. This option has the disadvantage that you can not enforce the developer to use UDDI.
But, after we make a decision on this first problem now, what happens if you simple exchange the client's ServiceEndPoint address with the URL you get from the UDDI Registry?
What happens if we:
- Move the service to another server?
The client works if it does not use encryption, otherwise it fails.
- A service fails?
The client works if it does not use encryption, otherwise it fails.
- Change communication parameters of the service such as binding or encryption?
The client fails.
- Want to introduce another service of the same kind for load balancing?
The client works if it does not use encryption, otherwise it fails.
- I want to get a quick overview of where my services are running?
I get a nice overview from my Rregistry any time.
So, if we use any kind of encryption, the concept we used for Web Services fails all but one of our architecture requirements.
Can't WCF Services be integrated into a UDDI environment?
The problem is basically the "ServiceModel" section in our client's config file. It freezes the state of the service to the point where we added the service reference. If you use a binding with encryption, which is by default based on Windows users, you find, for example, a tag called "Identity" which is Windows authentication, usually something like "Hostname\Username", and which is used for the client's encryption to the service. But, once we move the service to another server, this, of course, will fail due to the identity mismatch.
Integration of WCF Services into a UDDI Environment
How to query the Service metadata at runtime?
I remembered I had a little Web Service project where we dynamically queried a service's WSDL, created a DOM object, and then compiled the client at runtime. This project was based on Eshans Golkar's article "Dynamic Discovery and Invocation of Web services". So, my first approach to WCF was similar, and it works. But then, what I basically want to do is more or less the same that the WCFTestClient.exe does. You simply put in a URL to a service, WCFTestClients discovers it, and then creates a client. But, doing a little little reverse engineering using Lutz Roeder's excellent Reflector tool revealed that....
WCFTestClient.exe just invokes svcutil on the command line. :-))).
Who dared to ship an awkward solution like this? Butn when I did the same with "svcutil", I stumbled across two classes in the System.ServiceModel.Description namespace.
MetadataExchangeClient and MetaDataResolver
What both basically do is read the service's WSDL and then return a ServiceEndpointCollection. Now, you just have to choose which endpoint you want to use for communication with your service at runtime. But here, the first questions start. What do I need MetaDataResolver for, that needs a MetadataExchangeClient and returns a ServiceEndpointCollection, when I can use the MetadataExchangeClient's ImportAllEndpoints function? (Update: My implementation is solely based on MetadataexchangeClient. I tried to save some lines of code by using MetaDataResolver, but it never worked out. So, I just can discourage the use of MetaDataResolver.)
If you try get some light into the situation, you will soon find out that in regards to WCF, you now walk off the beaten track. Documentation and references will get very sparse from this point on. But anyway. This is basically exactly what we need to do, if we want to integrate WCF Services into a UDDI environment.
- We query the UDDI
- We get the main ServiceURL from the UDDI Registry
- We consume the service metadata on ServiceURL/mex or ServiceURL?WSDL
- We pick an endpoint of choice and configure the binding
- We return a client
- We use the client to call the desired service function
So, what we basically do is instead of freezing the services state to the point we add the service reference to our client project, we do the "Add Service reference" functionality every first time we want to talk to our service in our code! This basically means we do not need the <ServiceModel> in our config file anymore. This works a while, but then....you will find out...
Why we need the <ServiceModel> Section
The first time a crash happens is when you receive a service response that is larger than 64K. Your client will report that your MaxReceivedMessageSize quota is exceeded. And there, quite frankly, I ask myself the following question: Who the heck decided that in the age of Tera- and Petabytes 64KB would be a good default limit for receiving a service response? My electric toothbrush could handle a response larger than 64KB! The next superfluous default parameter modification that you will encounter is the maxItemsInObjectGraph that has the same arbitrary limit of 64K.
How to modify an unknown binding at runtime?
The core of the problem is that we do not know what bindings the service will offer.
MaxReceivedMessageSize and MaxBufferSize
Let's look at the bindings inheritance. All bindings are derived from the class Binding. But, except for some timeout parameters, there is nothing we can do here. Let's look at the binding element stack. In the stack of Binding Elements, this parameter belongs to the Transport Binding Element, which is part of any binding. But, I tried two days to find a way to get access to the binding elements, but only for CustomBinding will this lead you somewhere. For all other bindings, this is a dead end street. The CreateBindingElements() function will just create a copy of the actual binding elements, and the GetBindingProperty<t>() function which comes along undocumented in official documentation is of no help either. As we have no clue what binding the Service will offer, we now have three options:
- The IT-Hall of Shame candidate
Write a function that tries to cast the discovered binding into any known binding and then sets the parameter.
- We use Reflection
We use a function that first tries to cast the object into a custom binding. If the cast is successful, it tries for every property of every binding element to set a property with "Name" to "Value", and if the cast fails, just try to find a property with "Name" and then to set it to "Value".
- We return a
CustomBinding We use the discovered binding's CreateBindingElement() function, modify the parameter on the TransportBindingElement, and return a CustomBinding instead of the original binding.
I don't like any of these solutions, but finally, we decide to go for the Reflection solution.
How to modify MaxItemsInObjectGraph at runtime?
Interestingly, "MaxItemsInObjectGraph" is a property of the DatacontractSerializer which is also used in all bindings. If you use the config file, you now have to add an Endpoint Behavior of type DataContractSerializer, but if you try to find this in the ServiceModel namespace, you will find out that you can not instantiate this class at runtime. But looking further, there is a DataContractSerializerOperationBehavior that can be instantiated at runtime, but what can only be applied to an operation. So, what I did was to create a class that implements IEndPointBehavior and then simply set this DataContractSerializerOperationBehavior with a useful value to all operations. Now, I could get rid of my <ServiceModel> config section, and all my functions worked just as intended.
So we got it?
If you think this is enough struggling with 64K default parameters (yes, I mean it sarcastically), you are soon to be disappointed. Everything worked now, I could switch my service binding forth and back, and my clients just worked fine until one day..... they stopped working. MetadataExchangeClient reported that "Metadata contains a reference that cannot be resolved". It took me two days to find out what happened. Our data model had gotten bigger and bigger until one day.... Guess it. Yes, the WSDL got larger than 64K. And, this is the point where all reference implementations of MetadataExchangeClient and MetadataResolver will fail. And, guess again. To solve the issue, you have to create a binding on your own, set the MaxReceivedMessageSize parameter, and then use some constructors which are not mentioned in any example.
Finally!
This is so far the last submarine that submerged in front of my target architecture's course. I won't say this was the last issue until the end of the project, but yet, I am confident that we will reach all architecture requirements with WCF without any limitations. So, what happens now if:
- Move the service to another server?
I change the address in UDDI, and all my clients simply follow and continue working.
- A service fails?
The clients does a nice failover and reconnects no matter what.
- Change communication parameters of the service such as binding or encryption?
The client adapts to it.
- Want to introduce another service of the same kind for load balancing?
Just register it to the UDDI and enjoy.
- I want to get a quick overview of where my services are running?
I get a nice overview from my Registry any time.
All our target architecture goals are met.
Hooray! Awesome.
WCF implemention flaws
- The 64K default quotas on
MaxReceivedMessageSize, MaxBuffersize, and MaxItemsInObjectGraph, and maybe elsewhere. - The default WSDL schema that uses "
Import" statements in the "Types" section. - The bug in Visual Studio that does not let you choose your array type for flat WSDLs (MSConnect ID:387245).
- The inheritance of bindings that have highly redundant properties in the specific bindings, and/or the lack of access to the binding elements for non "Custom Bindings".
- Inconsistencies between between config file configuration and runtime configuration.
- Lack of documentation.
Conclusion
- WCF standard implementations are due to their "all is static" approach not enterprise ready.
- WCF Services can be integrated into an enterprise SOA architecture, but several implementation flaws prevent a seamless integration and increase project risk.
Suggestions to Microsoft for next WCF release
- Default binding quotas should be disabled by default.
Any quota you set by default is arbitrary, and will never fit all project scopes. If I do not trust the service, it is nice that I can set this quota, but please let me choose what is best here. And especially, do not use quotas that later blow up your own framework in a time bomb manner.
- "Imports" in WSDL by default.
This is only causing problems, and has absolutely no use. Get rid of it.
- The inheritance of
Binding. Instead of a two level hierarchy, it should be much more fine grained. For example, in the class BasicHttpBinding, there should be only configuration properties that are absolutely unique to BasicHttpBinding. All others should be inherited. Adding more inheritance could easily be done without breaking anything. This would smooth the way to a sophisticated runtime configuration of arbitrary bindings. I prefer this much more over an access to the binding elements as in CustomBinding.Elements.
- Runtime configuration and config configuration should be consistent.
Using the code
The code uses the Logging Application Block from Microsoft Enterprise Library 4.1. Either comment out all Logger.Write(...) lines and remove the entries from the config file, or make sure it is installed. I strongly recommend using the Enterprise Library anyway.
The solution comes with four projects.
UDDIServiceFactory
Manages the connection to the UDDI and provides a pattern for client failover. UDDIServiceFactory uses DynamicWCFFactory to create the client. It uses Microsoft.UDDI.dll from the Microsoft UDDI SDK to connect to the UDDI and keep a constant connection within its ManagedURL class.
Configuration parameters:
UDDIServiceFactory looks for the following parameters in app.config:
- "
ReturnDefaultClient" If set to "1", UDDIServiceFactory, before giving up, will return a standard WCF client in case the UDDI connection or all UDDI registered services fail. This client will be configured through the <ServiceModel> section in your app.config file. This is considered an emergency failsafe. In production environments, make absolutely sure that the configuration points to the right service before setting it to 1.
- "
LoadBalaceUDDIServices" If set to "1", UDDIFactory picks a random service from the list of available services rather than taking the first. This does some nice load balancing.
DynamicWCFFactory
This is used for creating a dynamic WCF client by discovering its WSDL at runtime. It also provides facilities for runtime configuration of a WCF client as well for callback instances in duplex channels. It supports WSDL's larger than 64K.
Configuration parameters:
DynamicWCFFactory is looking for the following parameters in app.config:
- "
UseBindingDefaults" If set to "1", DynamicWCFFactory sets the generic binding properties MaxReceivedMessageSize, MaxBufferSize, and MaxItemsInObjectGraph to int.MaxValue. This is to avoid any unnecessary hassles with 64K limits.
WcfService
An out of the box WCF Service. Used for Unit Tests.
DynamicWCFFactoryTests
This is a Unit Test project that contains two tests.
TestDynamicWCFFactory
Creates a WCF client using DynamicWCFFactory and invokes a function of our test service.
[TestMethod]
public void TestDynamicWCFFactory()
{
var dynWCFFactory =
new DynamicWCFFactory.DynamicWCFFactory<ServiceReference.Service1Client,
ServiceReference.IService1>();
ServiceReference.Service1Client client = dynWCFFactory.CreateProxyFromWSDL(
"http://localhost:4873/Service1.svc");
string someString = client.GetData(2);
}
TestUDDIFactory
Creates a WCF client using UDDIServiceFactory and invokes a function of our test service.
Before using the UDDIServiceFactory, you must have set up a UDDI registry and your service must have been registered properly with UDDI.
[TestMethod]
public void TestUDDIFactory()
{
try
{
UDDIServiceFactory<ServiceReference.Service1Client,
ServiceReference.IService1> uDDIFactory =
new UDDIServiceFactory<ServiceReference.Service1Client,
ServiceReference.IService1>("UDDIURL","serviceKey","bindingKey");
ServiceReference.Service1Client client = uDDIFactory.GetFirstProxy();
string someString;
while (true)
{
try
{
someString = client.GetData(2);
client.Close();
break;
}
catch (Exception ex)
{
if (client.State == CommunicationState.Faulted)
client.Abort();
else if (client.State == CommunicationState.Opened)
client.Close();
Logger.Write(ex, new List<string> { "General" }, 1, ERROR_SERVICE_CALL,
System.Diagnostics.TraceEventType.Warning, "Service Call failed.");
client = uDDIFactory.GetNextProxy();
}
}
}
catch (Exception ex)
{
Logger.Write(ex, new List<string> { "General" }, 1,
ERROR_UDDI_FACTORY_CREATION,
System.Diagnostics.TraceEventType.Warning,
"Creation of UDDIServiceFactory failed.");
throw;
}
}
Invoke the Unit Tests over the Visual Studio Test Toolbar. If it is not visible, go to "View -> Toolbars - Test Tools".
