Introduction
This article and sample presents the design of an extensible configurable WebService and its counter-partners, UserControl - based clients.
The WebService consists of:
- an invariable part (further referred to as "Server") handling general tasks (like security, scaling, objects pooling, configuration, data encrypting and compression, etc.) and
- add-in assemblies containing objects (further referred to as "Processors") to carry out specific tasks (e.g., database queries, text parsing, or, say, image processing).
The Server pre-processes the client's query and then dispatches it to an appropriate Processor for ultimate treatment. Server also carries out post-processing of response data on their way from a specific Processor to the client. The add-ins must implement well-known interface defined by the Server. Such an approach allows to considerably simplify and standardize the add-in design sparing it from the common tasks code and setting rules for its development.
The clients are .NET controls within Internet Explorer (IE). Their design addresses to client security ("sandboxing") and the control-host collaboration issues. The WebService client side proxy depends only upon the Server's Web Methods and invariant to Processors.
Brief workflow description
Administrator browses Admin.html file contained in its <object> tag reference to a UserControl-based control. The control connects to the Server and sends initialization data to it. Using these data, the Server starts its "security" mechanism (or rather its placeholder, since the security mechanism in this sample should not be taken seriously :) ), loads specific task assemblies and instantiate required number of Processor objects for each assembly. Maximum number of simultaneously running threads in Web Service process is also set. After WebService initialization has been accomplished successfully, clients may browse Client.html file and with its help, query the WebService. After been positively identified, client forms query, sends it to the WebService as a parameter of appropriate Web Method, and gets response. All data moving to and from WebService are packed in a well-defined object serialized to XML, in order to unify their processing on both client and server sides.
Test sample
Sample to this article is given in forms of demo project and source. The Composition and Compilation parts of this chapter address to the source, while Installation parts addresses to both demo project and source.
Installation
Unfortunately, manual installation of WebService requires some tedious work to be carried out. So, please be patient and excuse me if the sample will not run with your first attempt. :) The following steps should be performed to install demo.
On Server machine:
On Client machines:
- Copy DotNetPermissions.exe application (it is located in ..\ExtensibleWebService\DotNetPermissions folder of the demo; for the source sample it should be compiled) to all your Administrator and Client machines, and run it on each of them. In the dialog appeared, change Permitted Site from "Igor" to the name of your server machine and do not change other fields. (By setting permission set to "FullTrust" we are giving our
AdminCtrl and ClientCtrl controls, power to do virtually everything on client machine. In our case this is clear overkill since the only permissions required by the AdminCtrl is "File IO" for reading ConfigAdmin.xml configuration file, and "Security->Allow calls to unmanaged assemblies" for ClientCtrls. But thorough discussion on "sandboxing" client security is out of this article's scope. For more information about programmatic change of security settings, see e.g. Rajiv Sharma's notes and code here).
- Copy file ConfigAdmin.xml from ..\ExtensibleWebService\AdminCtrl folder of the sample to your Administrator machine (preferably to Desktop since this will be default directory of the Administrator control, and thus save some typing while running the sample).
Composition
The sample consists of the Server (ExtensibleWS, Framework and InHouseServerSecurity assemblies), Processor add-ins (ProcessorAssembly1 and ProcessorAssembly2 assemblies), administrative client (referred to as Administrator, AdminCtrl assembly) and ordinary client (referred to as Client, ClientCtrl assembly). Assembly ProcessorData handles data moving between clients and WebService. The assembly is used in Server, Processors and clients. Finally, assembly XmlWalker used on server side, implements XML parser.
Compilation
- Load ExtensibleWebService.sln (folder ..\ExtensibleWebService) to Visual Studio .NET. It is quite possible that you are using Microsoft Development Environment (MDE) 2003. In this case, you will be probably requested to convert solution from the MDE 2002 that I used for this project.
- Build projects ExtensibleWS and DotNetPermissions.
- Make sure that ExtensibleWebService and ExtensibleWebService/ExtensibleWS IIS virtual directories have been already created (see Installation section). Run WSProxyGenerator.bat file (in folder ..\ExtensibleWebService\ExtensibleWS) to generate ExtensibleWSProxy.dll proxy and move it to the root folder. (See comments in WSProxyGenerator.bat file for the full proxy generation procedure). Reference to the ..\ExtensibleWebService\ExtensibleWSProxy.dll assembly has to be added to the AdminCtrl and ClientCtrl projects before they are built.
- Build projects AdminCtrl, ClientCtrl, ProcessorAssembly1 and ProcessorAssembly2.
- Run file CopyDlls_Release.bat (or CopyDlls_Debug.bat, folder ..\ExtensibleWebService) to copy Release (or Debug) version of compiled assemblies to appropriate folders.
- Accomplish steps listed above in the Installation portion.
Now we are ready to run the sample.
How things work
Below description of user actions to run demo project is complemented with explanation of the source code behind them.
Administrator control
First, Admin.html is browsed with IE:
http://Igor/ExtensibleWebService/Admin.html
(as usual, replace "Igor" with the name of server machine). Administrator control appears within IE. Change Server Machine name, check and correct if needed, other edit boxes (Name and Password fields are not analyzed in the sample, so leave them empty) and press Submit button.
Submit button handler AdminControl.btnSubmit_Click() (in AdminCtrl project) creates instance of WebService proxy, packs contents of ConfigAdmin.xml configuration file to an object of ProcessorData.Data type and calls WebMethod ProcessAdmin() of the WebService.
Server
The ExtensibleWS.WS WebService has two WebMethods (in Service.asmx.cs file), namely, ProcessAdmin() and ProcessClient(). They accept serialized variable of ProcessorData.Data type (in real world, this variable should be encrypted and may be compressed). ProcessorData.Data is a general type to present data of clients' requests and WebService's responses. The data contain specific command to WebService, parameters of this command and GUID of Processor assembly that should process the command. They also have string members for server response and possible server error message.
The ProcessAdmin() method initializes WebService using data received from Administrator. This is done by InHouseServerSecurity.SecurityManager singleton class. Administrator control sends content of ConfigAdmin.xml file to the WebService. Server extracts from it the path to ConfigServer.xml file and actually initializes itself. In file ConfigServer.xml MaxProcessorsrNum denotes maximum number of Processors running simultaneously, each in its own thread. Then all Processor assemblies to be loaded are listed. IDs (0-based sequential numbers) define particular instance of a corresponding Processor. In the sample, two Processor objects of assembly ProcessorAssembly2 (ID = 0, 1) and three Processor objects of assembly ProcessorAssembly1 (IDs = 2, 3, 4) are instantiated during WebService initialization.
Main functionality of the Server is implemented in namespace Framework, chiefly in ProcessorManager and ProcessorFactory classes. Method ProcessorFactory.CreateProcessor() loads Processor assemblies and instantiates Processor objects. Singleton ProcessorManager performs the most of query processing. Method ProcessorManager.Run() called by WebMethod ProcessClient() chooses appropriate Processor object and makes it process client's data. The Run() method also insures compliance with maximum number of simultaneously running Processor threads.
WebService administrator may vary number of simultaneously running threads and number of instantiated Processor objects of each type (by making appropriate changes in file ConfigServer.xml before initializing the WebService) to achieve maximum productivity for a specific system configuration.
The rest of the ConfigServer.xml file's content defines security settings. Server is equipped with simple role-based security mechanism (or rather illustration of it). It is assumed Source->Part granularity of "protected" resources (e.g., MDB File (Source)->Table (Part)). To get access to a particular Source->Part fragment, its Source name should be in the query's ProcessorData.Data.ServerObjectName, and ProcessorData.Data.CommandText should contain its Part name. The code is in InHouseServerSecurity namespace and covers all Processor types. Please note that such a "security" is not secure at all and therefore may not be used in commercial applications and serves as a placeholder and illustration only.
Result of the WebService initialization is packed to the same ProcessorData.Data object and sent back to Administrator. If WebService has been properly initialized then it is ready to handle clients' requests.
Client control
Next, Client.html is browsed with IE (as usual, replace "Igor" with the name of server machine).
http://Igor/ExtensibleWebService/Client.html
Client control appears within IE. Insert to edit boxes, the same data as in Administrator control, specify some client's Name and Password (see them under the <Users> tag in file ConfigServer.xml, folder ..\ExtensibleWebService), or use default, and press Submit button. In case of successful operation, appearance of Client control in IE changes; the name of client written in blue, appears on top.
New appearance of the Client control facilitates SELECT SQL query construction. In real world applications directing client's SQL query to the service is not acceptable particularly for security reasons. But for the sake of simplicity, Processor of ProcessorAssembly1 is designed SQL query-oriented. Default values of query parameters correspond to Northwind database (SQL Server and Access alike). User has to select SOURCE, fill query fields and press Submit button (keep Load Test check box unchecked). Response from the service will be shown as raw XML in text box control and in user-friendly form in grid below.
Playing with different users and databases you will notice "security restrictions" (e.g., users of "Friends" group are not allowed to access Customers table in Northwind database, see file ConfigServer.xml).
Administrator and Client are UserControl-based controls. In the sample they are used from within IE (but may be also used from stand-alone WinForm applications). Important thing is collaboration between embedded control and its IE host. Clicking on Client control leads to activation of JScript ClientControl::ClickEvent() function in file Client.html (see more about script function call from UserControl here). The script function, in its turn, calls a method of Client control. For both calls appropriate message boxes are shown to user.
Processor assemblies
Each Processor assembly is designed to process client queries of specific type. For being loaded and activated by the Server, Processor assembly should implement Framework.IFactory interface, Framework.Processor class and have its unique GUID and name. A Processor assembly may configure itself with its own XML configuration file. The sample presents two Processor assemblies, namely, ProcessorAssembly1 and ProcessorAssembly2.
The ProcessorAssembly1's Processor1 object processes simple SQL queries with ADO.NET. It reads connections strings for database resources from its configuration file ConfigProcessor1.xml (folder ..\ExtensibleWebService\ProcessorAssembly1). Each Processor1 object establishes in its constructor, connections to all databases listed in the configuration file, and keeps connection objects in its htConnectionObject hashtable. This allows each Processor1 object to quickly serve user's query to any database.
The ProcessorAssembly2 has no specific functionality and may serve only as a template for development of more useful Processor objects.
Load test
A simple WebService load test is available if it is running from VS .NET in Debug mode. It may be activated by checking Load Test button in Client control. In this mode, Client queries WebService every [Time] interval (default 200 ms) 100 times. Test result is shown in VS .NET Debug window. You may play with the time interval, change duration of one query processing (change attributes of <Debug> node in file ConfigProcessor1.xml) and change number of simultaneously running threads and number of specific Processor instances in file ConfigServer.xml.
Further development
Of course, the most straightforward way is to add functional security, encryption and zip capabilities to the Server. It is also worth to consider caching of frequently requested data. Then more fancy things come to mind.
STA COM object in the Processor
For example, quite often developers want to built WebService using existing Win32 code. If we'd like to incorporate existing Single-Threaded Apartment (STA) COM object to our Processor (using COM Interop) then some extra-work has to be carried out. We probably want to create COM object once in the Processor constructor (since this is an expensive operation) and then use it to process user requests. The problem is that our Server is designed in such a manner that the Processor object's code may be executed in different threads. So, if the COM object is simply created in the Process constructor then it is accessed directly by different threads. Clearly, this is not acceptable for the STA COM objects.
In order to avoid such unfortunate situation, the following design is proposed. The Processor constructor creates an additional thread (referred to as thread B). The thread B function creates COM object, starts internal loop and waits on synchronization auto-reset event. When the Processor.Process() method is called by a thread-pool's thread (referred to as thread A), it prepares ProcessorData.Data structure for COM object and sets synchronization event releasing thread B. Synchronization event is reset, and thread A starts to wait on the event, while COM objects does the job in thread B. On completion of its work, thread B prepares output data structure and sets the event releasing thread A. Thread B again waits on event while thread A completes Processor.Process() method and returns output ProcessorData.Data structure to caller. Such a design allows Processor to execute COM object's code always in the thread where it was created. Threads A and B are never running simultaneously. Resources required to implement this approach are an additional thread and a synchronization event per Processor object.
Conclusion
A WebService provider and consumer are presented. Design of the provider permits to extend it with relatively simple add-ins to process specific user's queries. Provider's invariable part takes care of important common tasks, such as security, scaling, data encryption and zipping, etc. allowing add-ins to focus on their specifics. This approach simplify development of new WebServices reducing this process to the design of relatively small dedicated add-in components. UserControl-based WebService consumer collaborates with the provider and also with its Internet Explorer host. Demo project and source demonstrate these features.
References
Thanks
I'd like to express my deep gratitude to my friends and colleagues Timur Igamberdiev, Vitaly Shelest, Victor Katz and Marina Kogan who helped me in different ways to accomplish this article.
