Introduction
In this article, I will explain the implementation of an Open Source lightweight framework (named Simple Client Server Library (SCS)) that is developed to create client/server applications using a simple Remote Method Invocation mechanism over TCP/IP. It is a highly scalable and robust framework that is entirely developed in C# and .NET Framework 4.0 (You can take a look at the performance of the SCS framework section of this article).
This is the second article about SCS and explains how I implemented it. To learn what SCS is and how to use it, please see the first article at the link below. There are running sample applications upon SCS in the first article. I suggest you to take a look at the first article before reading this one:
Article Outline
What is TCP?
TCP (Transmission Control Protocol) is a transport layer protocol in the TCP/IP protocol suite (another common transport protocol in TCP/IP is UDP (User Datagram Protocol)). It is a layered protocol suite that uses a strong DoD model. DoD is a four layered model that consists of Link Layer, Internet Layer, Transport Layer, and Application Layer. TCP and IP protocols are the core protocols of TCP/IP (therefore its name TCP/IP). Here is a list of some common protocols in TCP/IP layers from top to down:
- Application Layer: HTTP, FTP, SMTP...
- Transport Layer: TCP, UDP...
- Internet Layer: IP, ARP, ICMP...
- Link Layer: Ethernet, Token Ring...
Also, all your TCP based applications (as SCS framework) stands on Application Layer. There are many other protocols on each layer.
TCP provides reliable, ordered delivery of a stream of bytes from a program on one computer to another program on another computer. TCP is a connection-oriented, asynchronous, and double-way communication protocol (see figure below). Let's explain these concepts:
- Ordered delivery of a stream of bytes: In TCP, applications communicate over byte streams. Minimum data transfer size is one byte. The first sent byte first reaches the destination (FIFO queue, ordered delivery). There are two independent communication streams (double-way communication). Applications can send data anytime, independent from each other (asynchronous communication).
- Reliability: TCP uses a sequence number to identify each byte of data. The sequence number identifies the order of the bytes sent from each computer so that the data can be reconstructed in order, regardless of any fragmentation, disordering, or packet loss that may occur during transmission. So, if the physical connection with the remote application does not fail and you don't get an exception, your data is almost guaranteed to be delivered. Event if a single byte of your data is not delivered (because of an error), subsequent bytes are not delivered before it. If your data reaches destination, you can be sure that it is not corrupted and not deficient.
Stream based double-way TCP communication
Like all communication protocols, in TCP, each application has a unique address. This address consists of an IP address and a TCP port.
IP address (for IPv4) is a four octet number separated by dots, like 85.112.23.219. All computers that are connected to the internet have an IP addresses. (If you are connected to the internet behind a router etc., you may not have a public IP address, but your router has one and your computer has a local network IP, thus the router dynamically uses NAT to allow you to use TCP/IP. Anyway, a remote application can send IP datagrams to your computer.)
TCP port is just a number (2 byte unsigned integer, therefore a TCP port can be 0 to 65536) that allows TCP to distinguish your application from other applications in the same computer.
The last TCP concept that I will mention about is the socket. A socket is an interface to send data to and receive data from the TCP/IP stack that is provided by the Operating System. In .NET, a socket is represented by a Socket object. A Socket object is used to send/receive data between two TCP endpoints. A special kind of socket is the Listener Socket. In server/client architecture, the server first opens a TCP listener socket to allow clients to connect to the server. After communication is established with the client, a dedicated socket object is created with the client and all communication with that client is made on this new socket.
Why TCP?
Communication between two applications is generally called as IPC (Inter-Process Communication). There are many ways of IPC. TCP/IP is one of them. The advantage of TCP/IP is that the communicating applications can be running on different computers and different locations. Since Internet also works on TCP/IP, remote applications can communicate over the internet. TCP/IP is completely platform independent, standard, and implemented by all operationg systems (and many other devices). Also, it can be used for communication of applications that is on the same computer. Surely, other techniques such as Named Pipes and Shared Memory can be used for communication of two applications but they have an important restriction that the two applications must run on the same computer (or on same network at the best case) and can not communicate over the internet. Anyway, the SCS framework is independent from TCP and can be extended to support other IPC techniques or event other communication frameworks.
What are the stages of building a communication framework upon TCP?
In this section, I will discuss how to make a plan to build a communication framework over TCP and the problems to be considered.
Stream based data transfer
From the perspective of TCP, everything that is being sent from an application to another are bytes of a stream. TCP does not know if it is bytes of a picture, a file, a string or a serialized object. Also TCP does not know if a data (say a serialized object) has been sent and we are now sending another data (I mention about message boundaries).
Messaging over streams
From the application perspective, the data transferred over a TCP stream are separated packets. I call it messages. An application wants to send messages to another application when needed. Also, an application wants to be informed (via some callback mechanism) when a new message is received from a remote application.
So, we must build a mechanism over TCP streams to write a message to a stream and read a message from a stream. We must provide a way of separating messages from each other.
Wire protocol (message to stream protocol)
The way of writing a message to a stream and reading a message from a stream is called wire protocol. A wire protocol has some knowledge about the message that is being sent and received. In .NET, if we represent a message by an object, we can use serialization to build a byte array from the object. We can use binary serialization, XML serialization (and then string to byte array encoding), or a custom serialization. Anyway, after creating a byte array from a message, we can write it to a TCP stream. Also, we can use a kind of deserialization to re-create an object from a TCP stream.
If we create a custom wire protocol that can be implemented in any other programming language or platform, we can build a platform independent system. But if we use .NET binary serialization, our application can only be used in .NET world.
Request/Reply style messaging
In most scenarios, an application sends a message to a remote application and waits for a response message. Because of the asynchronous nature of TCP, this is an important implementation detail. In this case, we must use multithreading tools (such as ManualResetEvent class in .NET) to block/wait a sender thread until a response is received and to notify when a response is received. We also must provide a way of matching request and reply messages. Last but not least, we must provide a timeout mechanism if the remote application does not send a response message within a specified time.
Mapping messages with classes and methods
This is the most important and distinctive feature of a TCP based communication library from a user (that uses the framework to build applications) perspective. Many different approaches can be preferred according to the needs of the user, capabilities of the programming language or platform etc... Let's discuss a few approaches here.
The first approach is message-oriented communication. In this way, the sender application sends an object from a special type of class using serialization (as mentioned before), and the receiver application deserializes the object, and enters a switch or if statement to call a method according to the properties of the object. Message classes can be derived from a base class and can define common methods. Even, message objects can be a simple string. Applications parse this string and performs the needed operations. The disadvantage of this approach is that the user of the library must define his own messaging logic and write switch statements. If a new message type is added to the system, many classes, methods, or code blocks must be changed. Also, it is not type safe, and messages are resolved at runtime. So an application can send unknown message types etc...
An extending to messaging can be accomplished using Reflection. Thanks to .NET allows reflection, we can send a message that will cause a method call on the remote application using reflection. Assume we have a message class like (really, it is a class in the SCS framework):
[Serializable]
internal class ScsRemoteInvokeMessage : ScsMessage
{
public string ServiceClassName { get; set; }
public string MethodName { get; set; }
public object[] Parameters { get; set; }
}
In the sender application, you can create an object from this class, set the appropriate class, method name, and parameter list, serialize it, and send it over the TCP socket. Then, in the receiver application, you can find the right class, create an instance of this class (or you can use a pre-created object from this class), and call the right method by the received parameters using reflection techniques. With this approach, there is no need to modify our communication layer to add new methods. Simply add a new method to your business class as you usually do, that's all. If the sender application sends a message with a new method name, your method is called automatically without changing anything. You can also return another message to the sender that contains the return value of the method call.
All right, this way, we have created a good communication layer in the receiver side. But, the sender application still must know the name of the class, method, parameters.. And also, it must create the message object and send it for all method calls. For sure, you can write a proxy class that has the same method signature as the target class. Your application calls this proxy class methods, it internally sends the right messages to the remote application, waits for response messages, and returns the values that are received from the remote application. This is really nice, but you must do that for all new methods or method changes on the target application. It will be a tedious work. So, maybe you can write an application that does this proxy class creation job for you using Reflection. In this case, you must re-create your proxy class for all changes on the target application service signature (Web Services and WCF services do exactly that).
The SCS framework provides a better approach to call methods on remote applications. It uses a dynamic proxy mechanism using the RealProxy class and reflection. This is one of the key points of SCS. This will be explained in the implementation of SCS.
Server/client architecture and multithreading
Simple TCP server/client architecture
TCP sockets can be used in blocking (synchronous) or nonblocking (asynchronous) mode. In the blocking mode, receiving and sending data is a blocking operation and sender/receiver thread is blocked until operation is completed. Especially in blocking receiving, the thread waits until a data is sent from remote application. So, we must make use of multithreading to do another operations when waiting data from remote application. This is not a major problem in client side since only one socket is used and one additional thread is enough to listen incoming data. But in server side, there is a dedicated socket for every clients. So, if there are 100 clients connected concurrently, there are 100 sockets to communicate with clients and 100 threads to wait incoming data from clients in blocking mode.
Blocking mode is easy to implement but not scalable for the reasons mantioned above. So, we must use asynchronous sockets with callback mechanism and a thread pool to process data.
Implementation of the SCS framework
After brief discussion on developing a TCP based Server/Client communication framework, now i will explain how i implemented SCS framework. SCS framework is consist of two layers as shown below.
Layers of the SCS framework
Messaging Layer
The Messaging Layer is responsible for sending messages (objects) to and receiving messages from a remote application. It is independent from the Remote Method Invocation Layer, and can be used separately for building message oriented communicating applications. It also allows you to define your custom wire protocol (so your application can communicate with non-.NET applications). All classes in the Messaging Layer are defined in the Hik.Communication.Scs namespace (and it's sub namespaces).
Messages
All messages that are sent or received implements IScsMessage interface. ScsMessage class implements it and all message classes are derived from ScsMessage.
Message classes in the SCS Messaging Layer
The ScsMessage class has two string properties that are key properties for the SCS framework: MessageId and RepliedMessageId. MessageId is a unique string (it is an auto generated GUID) that is used to distinguish message objects from each other. RepliedMessageId is null by default. If this message is a reply to another message, it contains the MessageId of the original message. This property is used to match request messages with reply messages. ScsMessage does not contain any data.
ScsTextMessage can be used to send/receive simple string messages. ScsRawDataMessage can be used to send/receive byte arrays. Using ScsRawDataMessage, you can send/receive a picture, file, a serialized .NET object... etc. PingMessage has no additional data, and is used by SCS framework to send Ping messages from the client to the server to keep the TCP connection alive and to check whether the connection is alive or not.
You can also write your own class that is derived from ScsMessage (or its subclasses) to send/receive your own objects. Surely you can directly implement the IScsMessage interface instead of deriving from the ScsMessage class. If you decide to send/receive objects from your own classes, you must mark the class as Serializable. Also, it is a good approach to define your message class in a separate class library and to add a reference of this library from both the server and client applications.
Wire protocols
A wire protocol is used to serialize and deserialize messages to communicate with remote application. All wire protocol classes in SCS must implement IScsWireProtocol interface.
Wire protocol classes in the SCS Messaging Layer
SCS receives bytes from remote application (via socket streams) and sends received bytes to CreateMessages(...) method of IScsWireProtocol. Wire protocol may create zero (if received bytes are not sufficient to create a message) or more messages using given bytes. Given byte array may not contain whole message, wire protocol must store it to concatanete with next receiving bytes. Wire protocol must not block thread.
SCS calls GetBytes(...) message serialize a message to send to remote applications. Finally, Reset() method is called to reset internal state of the protocol when needed (for instance, in a disconnect/re-connect situation).
A client directly uses IScsWireProtocol but a server uses IScsWireProtocolFactory since server needs to create a wire protocol object for each client. IScsWireProtocolFactory has just one method (CreateWireProtocol) that is used to create a wire protocol object.
As you see in the figure above, currently, the only wire protocol defined in the SCS framework is BinarySerializationProtocol. It uses the .NET BinaryFormatter class to serialize/deserialize objects. You can inherit it to change serialization mechanism or directly implement IScsWireProtocol and IScsWireProtocolFactory interfaces to create a custom wire protocol (Will be explained later). You can communicate with non-.NET applications easily by using custom protocols.
EndPoints
As I mentioned before (in the What is TCP section), an end point is used to define an address for your application. In SCS, all end point classes must be derived from the ScsEndPoint abstract class.
End point classes in the SCS framework
The ScsEndPoint class defines two important internal abstract methods to create a server (that listens to this end point for incoming client connections) and to create a client (that connects to a server that listens to this end point). I will return to these methods later. ScsEndPoint also defines a static CreateEndPoint(...) method to create an end point object from a string address. So, to create a ScsTcpEndPoint object for the 127.0.0.1 IP address and 10048 TCP port, you can pass the "tcp://127.0.0.1:10048" string to the CreateEndPoint(...) method. Surely, you can directly create a ScsTcpEndPoint object. As you see, the only class that is derived from ScsEndPoint is ScsTcpEndPoint. It has two properties: IpAddress and TcpPort.
Connection listeners
Connection listeners are used to listen and accept incoming client connections by the server. They create a communication channel (will be explained in the next section) for a new connected client.
Connection Listener classes in the SCS framework
As you can see in the class diagram, the IConnectionlistener interface defines two methods to Start/Stop the listener and an event named CommunicationChannelConnected that is raised when a new client connected to the server and communication channel is successfully created. The only concrete connection listener class is TcpConnectionListener. Some important parts of the TcpConnectionListener class are shown below:
internal class TcpConnectionListener : ConnectionListenerBase
{
private readonly ScsTcpEndPoint _endPoint;
private TcpListener _listenerSocket;
public TcpConnectionListener(ScsTcpEndPoint endPoint)
{
_endPoint = endPoint;
}
public override void Start()
{
StartSocket();
_running = true;
_thread = new Thread(DoListenAsThread);
_thread.Start();
}
private void StartSocket()
{
_listenerSocket = new TcpListener(System.Net.IPAddress.Any, _endPoint.TcpPort);
_listenerSocket.Start();
}
The TcpConnectionListener class defines a constructor that takes a ScsTcpEndPoint to listen. In the Start() method, it creates and starts a TcpListener (System.Net.Sockets.TcpListener) object with the specified port number of endpoint and then starts a separate thread to listen and accept new clients. The DoListenAsThread method is defined as below:
private void DoListenAsThread()
{
while (_running)
{
try
{
var clientSocket = _listenerSocket.AcceptSocket();
if (clientSocket.Connected)
{
OnCommunicationChannelConnected(
new TcpCommunicationChannel(clientSocket));
}
}
catch
{
StopSocket();
Thread.Sleep(1000);
if (!_running)
{
return;
}
try
{
StartSocket();
}
catch
{
}
}
}
}
The DoListenAsThread method calls the AcceptSocket() method of the TcpListener object (_listenerSocket). This is a blocking method, so the thread is blocked until a new client is connected to the server. It continues when a client is connected, checks if the socket is properly connected, then raises the CommunicationChannelConnected event using the OnCommunicationChannelConnected method. It creates a new TcpCommunicationChannel (will be examined in the next section) object and passes it as an argument to the OnCommunicationChannelConnected method (it is defined in the ConnectionListenerBase class as protected). You can take a look at CommunicationChannelEventArgs (that is used as argument in the event) in the SCS framework source code.
Communication channels
You have seen how a communication channel is created in the server side. Also, channels are created in the client side to connect to a server (we will see that later). In SCS, all communication between applications (server-to-client and client-to-server) are made on communication channels. Channel objects are responsible for sending message objects to remote application and to listen and receive incoming messages from the remote application over network. Communication channel hierarchy in SCS is shown in the figure below.
Communication channels in the SCS framework
All communication channels must implement the ICommunicationChannel interface or inherit from the CommunicationChannelBase abstract class. CommunicationChannelBase implements ICommunicationChannel and provides a good template for concrete communication channel classes.
As you can see, the ICommunicationChannel interface extends the IMessenger interface. The IMessenger interface represents an object that can send and receive messages. SendMessage(...) method defines the method to send a message to a remote application. MessageReceived event is raised when a new message is received from the communicating remote application (so, messages are received by events) and MessageSent event raised when a message successfully sent. IMessenger also defines the WireProtocol property that can be used to change the wire protocol (as mentioned before). Other properties explain themselves.
ICommunicationChannel inherits IMessenger and adds some functionality. It defines a property named CommunicationState to learn whether a channel is connected to a remote application or not. Defines RemoteEndPoint to obtain address of the remote application (For TCP, it is a ScsTcpEndPoint object.). It also defines the Start() and Disconnect() methods to start/stop communication. Lastly, if defines the Disconnected event that is raised when the channel disconnected.
The only concrete communication channel class is TcpCommunicationChannel in SCS. It is derived from CommunicationChannelBase. I'll examine this class to see how the communication channel is implemented using TCP. It defines just one constructor and gets a Socket (System.Net.Sockects.Socket) object.
public TcpCommunicationChannel(Socket clientSocket)
{
_clientSocket = clientSocket;
_clientSocket.NoDelay = true;
var ipEndPoint = (IPEndPoint)_clientSocket.RemoteEndPoint;
_remoteEndPoint = new ScsTcpEndPoint(ipEndPoint.Address.ToString(), ipEndPoint.Port);
_buffer = new byte[ReceiveBufferSize];
_syncLock = new object();
}
As you saw in the Communication listeners section, this constructor is used by the TcpConnectionListener class when a new client is connected to the server. NoDelay = true is used to disable Nagle algoritm (Because it limits the speed of the SCS message sending and is not needed because SCS writes whole message to the socket at once). A ScsTcpEndPoint object is created as remote end point. It also creates a buffer to receive messages asynchronously.
protected override void StartInternal()
{
_running = true;
_clientSocket.BeginReceive(_buffer, 0, _buffer.Length, 0, new AsyncCallback(ReceiveCallback), null);
}
StartInternal method is called by base class to start receiving messages from remote application. Since we use asynchronous socket methods, we call non-blocking BeginReceive method by supplying a buffer (to get received bytes) and a callback method to be called when any bytes are received from the remote application.
private void ReceiveCallback(IAsyncResult ar)
{
if(!_running)
{
return;
}
try
{
var bytesRead = _clientSocket.EndReceive(ar);
if (bytesRead > 0)
{
LastReceivedMessageTime = DateTime.Now;
var receivedBytes = new byte[bytesRead];
Array.Copy(_buffer, 0, receivedBytes, 0, bytesRead);
var messages = WireProtocol.CreateMessages(receivedBytes);
foreach (var message in messages)
{
OnMessageReceived(message);
}
}
if (_running)
{
_clientSocket.BeginReceive(_buffer, 0, _buffer.Length, 0, new AsyncCallback(ReceiveCallback), null);
}
}
catch
{
Disconnect();
}
}
In the ReceiveCallBack method, receiving operation is finished using EndReceive method of the socket. Received bytes are given to the current wire protocol to create messages. If wire protocol could created any messages, a MessageReceived event is raised for each message using OnMessageReceived method of the base class. Lastly, non-blocking BeginRecieve method of Socket is called again to receive next incoming bytes from socket (Note that this is a kind of recursive call since callback method is same as caller method).
Now, let's see how a message is being sent. The ICommunicationChannel.SendMessage(IScsMessage message) method is implemented by the CommunicationChannelBase class and it calls SendMessageInternal method of it's subclass TcpCommunicationChannel:
protected override void SendMessageInternal(IScsMessage message)
{
var totalSent = 0;
lock (_syncLock)
{
var messageBytes = WireProtocol.GetBytes(message);
while (totalSent < messageBytes.Length)
{
var sent = _clientSocket.Send(messageBytes, totalSent, messageBytes.Length - totalSent, SocketFlags.None);
if (sent <= 0)
{
throw new CommunicationException("Message could not be sent via TCP socket. Only " +
totalSent + " bytes of " + messageBytes.Length + " bytes are sent.");
}
totalSent += sent;
}
LastSentMessageTime = DateTime.Now;
OnMessageSent(message);
}
}
It serializes the message (IScsMessage) object to a byte array using the WireProtocol object and sends it to the remote application using the Socket object (_clientSocket). Sending a message is done in a lock statement. Thus, if more than one thread wants to send messages, only one of them can send at a time, others wait.
Client side
We have seen how a communication channel is created (by a connection listener in the server side) and how a message is sent and received by a communication channel. Now I will examine the SCS client objects that are used to connect to and communicate with the server. The main interface to use in the client side is IScsClient.
Client classes in the SCS framework
The IScsClient interface inherits IMessenger and IConnectableClient, and does not define any additional members. We have examined IMesenger before (in the Communication channels section). IConnectableClient is used to control the connection of the client to the server. It looks like ICommunicationChannel interface but the implementation and purpose are a little different. It defines the Connect() method to connect to a server (instead of the Start() method of channels), and the Disconnect() method to close a connection. It defines two important events: Connected and Disconnected. They can be used to monitor the connection state of the client. ConnectTimeout is used to set a timeout value while connecting to the server. Finally, it defines the CommunicationState property to check current state of the connection (same as channels).
The ScsClientBase abstract class provides a good template to implement IScsClient. Almost all implementation are made in this class. So, I will first examine the ScsClientBase class. Let's look at two properties here:
public IScsWireProtocol WireProtocol
{
get { return _wireProtocol; }
set
{
if (CommunicationState == CommunicationStates.Connected)
{
throw new ApplicationException("Wire protocol can not " +
"be changed while connected to server.");
}
_wireProtocol = value;
}
}
private IScsWireProtocol _wireProtocol;
public CommunicationStates CommunicationState
{
get
{
return _communicationChannel != null
? _communicationChannel.CommunicationState
: CommunicationStates.Disconnected;
}
}
As you can see, WireProtocol can only be changed while not connected to the server. CommunicationState directly gets the CommunicationState of the underlying channel object. The LastReceivedMessageTime, LastSentMessageTime properties, and the SendMessage(...) and Disconnect() methods are also directly mapped to the underlying channel object. The constructor of ScsClientBase is shown below:
protected ICommunicationChannel _communicationChannel;
private readonly Timer _pingTimer;
protected ScsClientBase()
{
_pingTimer = new Timer(30000);
_pingTimer.Elapsed += PingTimer_Elapsed;
ConnectTimeout = DefaultConnectionAttemptTimeout;
WireProtocol = WireProtocolManager.GetDefaultWireProtocol();
}
In the constructor, we create a Timer (this timer is defined by the SCS framework) that is used to send periodic PingMessage objects to the server (if the communication line is idle in the last minute). And we also get a reference to the default wire protocol (BinarySerializationProtocol). If the user does not set WireProtocol, then the default protocol is used. Now, let's see how the client connects to the server (how the Connect() method is implemented):
public void Connect()
{
WireProtocol.Reset();
_communicationChannel = CreateCommunicationChannel();
_communicationChannel.WireProtocol = _wireProtocol;
_communicationChannel.Disconnected += CommunicationChannel_Disconnected;
_communicationChannel.MessageReceived += CommunicationChannel_MessageReceived;
_communicationChannel.MessageSent += CommunicationChannel_MessageSent;
_communicationChannel.Start();
_pingTimer.Start();
OnConnected();
}
We first reset the wire protocol. This is needed if user re-connect to the server (disconnects and connects over same object). Then we create a communication channel. This is critical because we must create the appropriate communication channel according to the current transfer layer protocol (TCP is the default). We achieve this by defining the CreateCommunicationChannel() method as abstract. So it will be implemented by the derived classes (by the ScsTcpClient class). Then we set the WireProtocol of the channel, register to events of the channel and start it. Finally, we start the ping timer and raise the Connected event. The last method of ScsClientBase that I will examine here is the Elapsed event handler of the _pingTimer object. We are sending a Ping message just in case no communication was made in the last minute, as shown below:
private void PingTimer_Elapsed(object sender, EventArgs e)
{
if (CommunicationState != CommunicationStates.Connected)
{
return;
}
try
{
var lastMinute = DateTime.Now.AddMinutes(-1);
if (_communicationChannel.LastReceivedMessageTime > lastMinute ||
_communicationChannel.LastSentMessageTime > lastMinute)
{
return;
}
_communicationChannel.SendMessage(new PingMessage());
}
catch
{
}
}
As shown in the diagram at the beginning of this section, the only concrete client class that implements IScsClient is ScsTcpClient. It inherits ScsClientBase and overrides the CreateCommunicationChannel() method as defined below.
protected override ICommunicationChannel CreateCommunicationChannel()
{
return new TcpCommunicationChannel(
TcpHelper.ConnectToServer(
new IPEndPoint(IPAddress.Parse(_serverEndPoint.IpAddress), _serverEndPoint.TcpPort),
ConnectTimeout
));
}
It creates a .NET Socket (System.Net.Sockets.Socket) object and creates a TcpCommunicationChannel object using the socket. You can see the TcpHelper.ConnectToServer(...) method in the source code. It simply creates a Socket object and connects to a server. If can not connect to a server with a specified time (ConnectTimeout), throws an exception.
We have seen how the TCP client (ScsTcpClient) works. Now, let's see how it is created by the user of SCS framework.
public static class ScsClientFactory
{
public static IScsClient CreateClient(ScsEndPoint endpoint)
{
return endpoint.CreateClient();
}
public static IScsClient CreateClient(string endpointAddress)
{
return CreateClient(ScsEndPoint.CreateEndPoint(endpointAddress));
}
}
One of ScsClientFactory.CreateClient(...) methods is used to create a client. It gets an end point (ScsEndPoint) object and returns a client (IScsClient) object. As you can see, it just calls the CreateClient() method of the end point. Thus, we don't enter a switch or if statement, since each end point object knows what type of client will be created using the end point. So, if new protocols are added, there is no need to change the ScsClientFactory. If the endPoint arguments is a ScsTcpEndPoint (the only end point as default), then a ScsTcpClient object is created in the ScsTcpEndPoint.CreateClient() method as shown below.
internal override IScsClient CreateClient()
{
return new ScsTcpClient(this);
}
Also, user can pass a string address (like tcp://127.0.0.1:10048 for 10048 TCP port on localhos) instead of an end point. An endpoint is created from the string using ScsEndPoint.CreateEndPoint static method.
After all, user can easily create a TCP based SCS client object, connect to a server, send/receive messages, and close the connection, as shown below:
using System;
using Hik.Communication.Scs.Client;
using Hik.Communication.Scs.Communication.EndPoints.Tcp;
using Hik.Communication.Scs.Communication.Messages;
namespace ClientApp
{
class Program
{
static void Main()
{
var client = ScsClientFactory.CreateClient(new ScsTcpEndPoint("127.0.0.1", 10085));
client.MessageReceived += Client_MessageReceived;
Console.WriteLine("Press enter to connect to the server...");
Console.ReadLine();
client.Connect();
Console.Write("Write some message to be sent to server: ");
var messageText = Console.ReadLine();
client.SendMessage(new ScsTextMessage(messageText));
Console.WriteLine("Press enter to disconnect from server...");
Console.ReadLine();
client.Disconnect();
}
static void Client_MessageReceived(object sender, MessageEventArgs e)
{
var message = e.Message as ScsTextMessage;
if (message == null)
{
return;
}
Console.WriteLine("Server sent a message: " + message.Text);
}
}
}
To demonstrate, client just sends/receives ScsTextMessage objects but can send any type of objects that is derived from ScsMessage (or implements IScsMessage) as mentioned before. The server side of this application will be shown at the end of the server side section. You can find the source code in the Samples\SimpleMessaging folder in the download file.
Server side
We have seen almost everything about client side of SCS. In this section, I will examine that how the server side of SCS Messaging Layer is implemented. The main interface on the server side is IScsServer. Like other inheritance models in SCS framework, there is a template abstract implementation of it named ScsServerBase. ScsServerBase implements common members in IScsServer and provides a good base for concrete server classes. The only concrete server class is the ScsTcpServer class.
Server classes in the SCS framework
IScsServer defines WireProtocolFactory to allow the user to change the messaging protocol (Since server creates a wire protocol object for each client, a factory class is used instead of direct use of WireProtocol). It defines a kind of sorted list named Clients that is used to store the clients that are currently connected to the server (ThreadSafeSortedList is used to store clients to be thread safe, because it can be changed by more than one thread). The key of this collection is ClientId (a unique long number) and the value is a reference to the client object (IScsServerClient) to communicate with the client (will be examined later). Server class defines ClientConnected and ClientDisonnected events. Using this events, we can be informed when a client has successfully connected to and disconnected from the server. We can get a reference to a connected/disconnected clients in the event handlers using the ServerClientEventArgs event argument.
As a beginning to examining the server side, let's see some significant methods of ScsServerBase.
public virtual void Start()
{
_connectionListener = CreateConnectionListener();
_connectionListener.CommunicationChannelConnected +=
ConnectionListener_CommunicationChannelConnected;
_connectionListener.Start();
}
public virtual void Stop()
{
if (_connectionListener != null)
{
_connectionListener.Stop();
}
foreach (var client in Clients.GetAllItems())
{
client.Disconnect();
}
}
protected abstract IConnectionListener CreateConnectionListener();
private void ConnectionListener_CommunicationChannelConnected(object sender,
CommunicationChannelEventArgs e)
{
var client = new ScsServerClient(e.Channel)
{
ClientId = ScsServerManager.GetClientId(),
WireProtocol = WireProtocolFactory.CreateWireProtocol()
};
client.Disconnected += Client_Disconnected;
Clients[client.ClientId] = client;
OnClientConnected(client);
e.Channel.Start();
}
private void Client_Disconnected(object sender, EventArgs e)
{
var client = (IScsServerClient) sender;
Clients.Remove(client.ClientId);
OnClientDisconnected(client);
}
The main responsibility of the ScsServerBase class is to manage clients. In the Start() method, it creates a connection listener (IConnectionListener) using the abstract CreateConnectionListener() method. This method is implemented by ScsTcpServer to create a ScsTcpConnectionListener object (please see the ScsTcpServer class in the source code). It then registers to the CommunicationChannelConnected event and starts the connection listener. The Stop() method just stops the connection listener and disconnects all active clients.
The CommunicationChannelConnected event handler (ConnectionListener_CommunicationChannelConnected method) creates a new ScsServerClient object with the new channel, registers its Disconnect event to be informed when the client connection fails, adds it to the Clients collection, and starts communication with the new client. Finally, the Client_Disconnected event handler method is used to remove the disconnected client from the Clients collection. Note that the ClientId property of the new client is set while creating the object using ScsServerManager.GetClientId(). This method is defined below. It uses the Interlocked (System.Threading.Interlocked) class to increment _lastClientId in a thread safe manner.
internal static class ScsServerManager
{
private static long _lastClientId;
public static long GetClientId()
{
return Interlocked.Increment(ref _lastClientId);
}
}
As you have seen, a client is represented by the IScsServerClient interface in server side. This interface is used to communicate with the client from the server.
Server side client classes in the SCS framework
The IScsServerClient interface also extends IMessenger. It is implemented by the ScsServerClient class. There is no transport-layer based implementation of IScsServerClient (like ScsTcpServerClient), since ScsServerClient uses ICommunicationChannel to communicate with the client (thus, it is protocol independent). The constructor and some methods of ScsServerClient are shown below:
public long ClientId { get; set; }
protected ICommunicationChannel _communicationChannel;
public ScsServerClient(ICommunicationChannel communicationChannel)
{
_communicationChannel = communicationChannel;
_communicationChannel.MessageReceived += CommunicationChannel_MessageReceived;
_communicationChannel.MessageSent += CommunicationChannel_MessageSent;
_communicationChannel.Disconnected += CommunicationChannel_Disconnected;
}
public void Disconnect()
{
_communicationChannel.Disconnect();
}
public void SendMessage(IScsMessage message)
{
_communicationChannel.SendMessage(message);
}
As I mentioned above, ScsServerClient uses a communication channel to communicate with the client, and gets it as an argument to the constructor. It registers the events of the communication channel to be able to raise these events since it implements IMessenger and IScsServerClient and they define the MessageReceived and Disconnected events. ClientId is a unique number that is set by the server, while ScsServerClient is being created (as explained before).
The user of the SCS framework uses the ScsServerFactory class to create a server object.
public static class ScsServerFactory
{
public static IScsServer CreateServer(ScsEndPoint endPoint)
{
return endPoint.CreateServer();
}
}
Very similar to creating a client object, we get an end point object (that is used to listen to incoming client connection requests) and use the end point to create a server. Thus, the server object is created according to the end point (note that this method is internal and can not be used by the user directly). Since all end points (in fact, there is only one) know what type of server to create, the server object is successfully created. Let's see the TCP implementation of the CreateServer() method in the ScsTcpEndPoint class.
internal override IScsServer CreateServer()
{
return new ScsTcpServer(this);
}
Lastly, let's see a simple server application that listens to client connections, gets a text message from clients, and sends a response for the message. It completes the sample application that we built in the client side.
using System;
using Hik.Communication.Scs.Communication.EndPoints.Tcp;
using Hik.Communication.Scs.Communication.Messages;
using Hik.Communication.Scs.Server;
namespace ServerApp
{
class Program
{
static void Main()
{
var server = ScsServerFactory.CreateServer(new ScsTcpEndPoint(10085));
server.ClientConnected += Server_ClientConnected;
server.ClientDisconnected += Server_ClientDisconnected;
server.Start();
Console.WriteLine("Server is started successfully. Press enter to stop...");
Console.ReadLine();
server.Stop();
}
static void Server_ClientConnected(object sender, ServerClientEventArgs e)
{
Console.WriteLine("A new client is connected. Client Id = " + e.Client.ClientId);
e.Client.MessageReceived += Client_MessageReceived;
}
static void Server_ClientDisconnected(object sender, ServerClientEventArgs e)
{
Console.WriteLine("A client is disconnected! Client Id = " + e.Client.ClientId);
}
static void Client_MessageReceived(object sender, MessageEventArgs e)
{
var message = e.Message as ScsTextMessage;
if (message == null)
{
return;
}
var client = (IScsServerClient)sender;
Console.WriteLine("Client sent a message: " + message.Text +
" (Cliend Id = " + client.ClientId + ")");
client.SendMessage(
new ScsTextMessage(
"Hello client. I got your message (" + message.Text + ")",
message.MessageId
));
}
}
}
If you run the server and client applications and send a message from the client to the server, you can see a screen as shown below:
A screenshot of the running test applications
You can find the source code in the Samples\SimpleMessaging folder in the download file.
Remote Method Invocation Layer
As I mentioned before, the SCS framework allows the user to call remote methods. I have explained the Messaging Layer in detail. The Remote Method Invocation Layer is responsible for translating method calls to messages and sending them over the Messaging Layer. It also receives messages from the Messaging Layer and calls the service methods of the application. All classes in the Messaging Layer are defined in the Hik.Communication.ScsServices namespace (and its sub namespaces). I suggest you to take a look at the chat system in the first article to understand how the RMI Layer is used.
Request/Reply messaging
A method call is a type of request/reply operation. The request consists of the name of the method and the arguments, and the reply is the result value of the method (even if the method return type is void, the method can throw an exception that can be considered as a response/output). The Messaging Layer of SCS is asynchronous (because of the nature of TCP). It means the server and the client can send/receive messages independent from each other. So, we must build a mechanism to send a method call request and wait for the result of the method call.
Request/Reply style messaging is accomplished in SCS with the RequestReplyMessenger class. It is actually a decorator for any IMessenger. The class diagram of RequestReplyMessenger is shown below:
RequestReplyMessenger class diagram; just some of the members of RequestReplyMessenger is shown in diagram
RequestReplyMessenger is a generic class, and gets a class (that implements IMessenger) as its generic parameter. RequestReplyMessenger also implements IMessenger (as shown in the class diagram above). It decorates an IMessenger object with the following features:
SendMessageAndWaitForReply(...) method: Using this method, we can send a message to a remote application and get a reply message. It blocks calling a thread until a response is received or a timeout occurs. If a response is not received in a specified time, an Exception is thrown. It uses IScsMessage.RepliedMessageId property to match messages.- Queued processing of incoming messages:
RequestReplyMessenger processes message over a FIFO (first in first out) queue. If the sender application generates messages faster than processing in the receiver side, messages are queued.
First we must look at how a RequestReplyMessenger is being created and used. Assume that you have created an SCS client and wants to use RequestReplyMessenger to send request/reply style messages over this client. I will re-write the application that we built in the client side section.
using System;
using Hik.Communication.Scs.Client;
using Hik.Communication.Scs.Communication.EndPoints.Tcp;
using Hik.Communication.Scs.Communication.Messages;
using Hik.Communication.Scs.Communication.Messengers;
namespace RequestReplyStyleClient
{
class Program
{
static void Main()
{
Console.WriteLine("Press enter to connect to the server...");
Console.ReadLine();
using (var client = ScsClientFactory.CreateClient(new ScsTcpEndPoint("127.0.0.1", 10085)))
{
using (var requestReplyMessenger = new RequestReplyMessenger<IScsClient>(client))
{
requestReplyMessenger.Start();
client.Connect();
Console.Write("Write some message to be sent to server: ");
var messageText = Console.ReadLine();
var response = requestReplyMessenger.SendMessageAndWaitForResponse(new ScsTextMessage(messageText));
Console.WriteLine("Response to message: " + ((ScsTextMessage) response).Text);
Console.WriteLine("Press enter to disconnect from server...");
Console.ReadLine();
}
}
}
}
}
As you can see, we create a RequestReplyMessenger<IScsClient> object. The generic parameter is IScsClient since we are using this type of object to communicate with the server (remember that IScsClient extends IMessenger). Then we call the Start() method (it internally starts the message process queue as we will see soon). The critical line is where we call the SendMessageAndWaitForResponse(...) method. It returns a IScsMessage object. This object is the response message that is sent from the server for our outgoing message. Finally, we must Stop RequestReplyMessenger (to stop the internal messaging queue). Since it is IDisposible, stops automatically when disposed by using statement. Now, let's examine how the RequestReplyMessenger class is implemented.
Since RequestReplyMessenger can be used by more than one thread, it must block each thread, store the messaging context (as the sending message ID) and match with received messages. So, it must store some information about the request, until the response is received (or a timeout occurs). It uses WaitingMessage class objects to accomplish this task.
private sealed class WaitingMessage
{
public IScsMessage ResponseMessage { get; set; }
public ManualResetEvent WaitEvent { get; private set; }
public WaitingMessageStates State { get; set; }
public WaitingMessage()
{
WaitEvent = new ManualResetEvent(false);
State = WaitingMessageStates.WaitingForResponse;
}
}
private enum WaitingMessageStates
{
WaitingForResponse,
Cancelled,
ResponseReceived
}
WaitingMessage objects are stored in the _waitingMessages collection until a response is received.
private readonly SortedList<string, WaitingMessage> _waitingMessages;
Now, let's see how I implemented the SendMessageAndWaitForReply(...) method.
public IScsMessage SendMessageAndWaitForResponse(IScsMessage message,
int timeoutMilliseconds)
{
var waitingMessage = new WaitingMessage();
lock (_waitingMessages)
{
_waitingMessages[message.MessageId] = waitingMessage;
}
try
{
Messenger.SendMessage(message);
waitingMessage.WaitEvent.WaitOne(timeoutMilliseconds);
switch (waitingMessage.State)
{
case WaitingMessageStates.WaitingForResponse:
throw new Exception("Timeout occured. Can not received response.");
case WaitingMessageStates.Cancelled:
throw new Exception("Disconnected before response received.");
}
return waitingMessage.ResponseMessage;
}
finally
{
lock (_waitingMessages)
{
if (_waitingMessages.ContainsKey(message.MessageId))
{
_waitingMessages.Remove(message.MessageId);
}
}
}
}
First, we create a WaitingMessage object and store it in the _waitingMessages SortedList collection with the MessageId of the sending message as key (we are performing operations on the collection in lock statements for thread safety). Then we send the message by using the underlying Messenger object (that is passed as an argument into the constructor as you saw before). We must wait until a response message is received. So, we call the WaitOne(...) method of the ManualResetEvent (System.Threading.ManualResetEvent) class. It blocks the calling thread until another thread calls the Set() method of the same ManualResetEvent object or a timeout occurs. Anyway, the calling thread continues its running and checks the current State of the WaitingMessage object. If it is still in the WaitForResponse state, that means a timeout has occurred. If it is in the Canceled state, that means RequestReplyMessenger has stopped before a response was received; else (that is to say the state is in ResponseReceived), the response message is returned as a return value of the SendMessageAndWaitForResponse(...) method. Finally, we must remove the WaitingMessage object from the _waitingMessages collection since it is not being waited for anymore.
Now, let's examine that how the State of a WaitingMessage object is changed. The first case is the response message is received successfully. To do this, we must register to the MessageReceived event of the underlying IMessenger object. The event handler for MessageReceived is shown below:
private void Messenger_MessageReceived(object sender, MessageEventArgs e)
{
if (!string.IsNullOrEmpty(e.Message.RepliedMessageId))
{
WaitingMessage waitingMessage = null;
lock (_waitingMessages)
{
if (_waitingMessages.ContainsKey(e.Message.RepliedMessageId))
{
waitingMessage = _waitingMessages[e.Message.RepliedMessageId];
}
}
if (waitingMessage != null)
{
waitingMessage.ResponseMessage = e.Message;
waitingMessage.State = WaitingMessageStates.ResponseReceived;
waitingMessage.WaitEvent.Set();
return;
}
}
_incomingMessageProcessor.EnqueueMessage(e.Message);
}
This method first checks if the received message is a response message (if its RepliedMessageId is empty, then it is not a response message). If it is, check if there is a message in the _waitingMessages collection that is waiting for this response. If this condition is also true, then set the state of the message as ResponseReceived, set ResponseMessage as a new incoming message, and lastly call the Set() method of the ManualResetEvent object (WaitEvent) to notify the waiting thread to continue. If all conditions are false and the message is an ordinary incoming message, it is added to the _incomingMessageProcessor to be processed (please see the source code for processing of messages in this queue). MessageReceived event is called for non-reply messages.
The second and last method that changes the state of a WaitingMessage is the Stop() method of RequestReplyMessenger. It is defined as below:
public void Stop()
{
_incomingMessageProcessor.Stop();
lock (_waitingMessages)
{
foreach (var waitingMessage in _waitingMessages.Values)
{
waitingMessage.State = WaitingMessageStates.Cancelled;
waitingMessage.WaitEvent.Set();
}
_waitingMessages.Clear();
}
}
If RequestReplyMessenger is stopped, it sets the states of all waiting messages to Canceled and informs the waiting threads to continue by calling the Set() method of ManualResetEvent.
With the examining of the Stop method, I have finished explaining Request/Reply style messaging. This was one of the key points of the SCS framework.
Remote Method Call messages
We built a strong messaging infrastructure until this point. It's time to see how method calls are being translated into messages and vice versa. The Remote Method Invocation Layer defines two new message classes. They are ScsRemoteInvokeMessage and ScsRemoteInvokeReturnMessage.
Remote Method Invocation classes
ScsRemoteInvokeMessage stores method call information such as name of the method (MethodName), the class that has the method (ServiceClassName), and all parameters (Parameters) that are used while calling the method on a remote application. ScsRemoteInvokeReturnMessage stores the return value of the method call (ReturnValue) if no exception occurs, or the exception object (RemoteException) if an exception is thrown in the remote method. Since both of these classes are inherited from ScsMessage, they have the MessageId and RepliedMessageId properties that are used to match the request (ScsRemoteInvokeMessage) and reply (ScsRemoteInvokeReturnMessage) messages.
Service contracts and their usage
Service contracts are another key point of the SCS framework. A service contract is an interface that is used to define methods of the server that can be remotely called by clients. You can also create a service contract to define methods of the clients that can be remotely called by the server (you can think that service contracts in the SCS framework correspond to contracts in WCF).
Before going further into the implementation of the SCS Remote Method Invocation Layer, I want to demonstrate the usage of this layer in a very simple application. If you have read the first article, you can skip to the next section. If you want to see the complete usage of this layer (as the used for server-to-client method calls), please see the first article. In this simple application, I will use a calculator service and a client that uses the service to do some calculations.
First, we define a service contract interface as shown below:
[ScsService]
public interface ICalculatorService
{
int Add(int number1, int number2);
double Divide(double number1, double number2);
}
The service contract is very clear. It is an ordinary C# interface except for the ScsService attribute. Let's see a service application that implements this contract. The complete server side code is shown below:
using System;
using CalculatorCommonLib;
using Hik.Communication.Scs.Communication.EndPoints.Tcp;
using Hik.Communication.ScsServices.Service;
namespace CalculatorServer
{
class Program
{
static void Main(string[] args)
{
var serviceApplication =
ScsServiceBuilder.CreateService(new ScsTcpEndPoint(10083));
serviceApplication.AddService<ICalculatorService,
CalculatorService>(new CalculatorService());
serviceApplication.Start();
Console.WriteLine("Calculator service is started. Press enter to stop...");
Console.ReadLine();
serviceApplication.Stop();
}
}
public class CalculatorService : ScsService, ICalculatorService
{
public int Add(int number1, int number2)
{
return number1 + number2;
}
public double Divide(double number1, double number2)
{
if(number2 == 0.0)
{
throw new DivideByZeroException("number2 can not be zero!");
}
return number1 / number2;
}
}
}
The CalculatorService class implements ICalculatorService. It must be inherited from the ScsService class. In the Main method of the console application, we are first create a service application that runs on the 10083 TCP port. Then we create a CalculatorService object and add it to the service application using AddService(...). This method is generic, and gets the service contract interface type and the service implementation type. Then we start the service by using the Start() method. As you can see, it is very easy and straightforward to create a SCS service application. Now, let's write a client application that uses this service:
using System;
using CalculatorCommonLib;
using Hik.Communication.Scs.Communication.EndPoints.Tcp;
using Hik.Communication.ScsServices.Client;
namespace CalculatorClient
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Press enter to connect " +
"to server and call methods...");
Console.ReadLine();
using (var client =
ScsServiceClientBuilder.CreateClient<ICalculatorService>(
new ScsTcpEndPoint("127.0.0.1", 10083)))
{
client.Connect();
var division = client.ServiceProxy.Divide(42, 3);
Console.WriteLine("Result: " + division);
}
Console.WriteLine("Press enter to stop client application");
Console.ReadLine();
}
}
}
As you can see, we are using the ServiceProxy property of the client object (IScsServiceClient) to call the service methods remotely. It is a dynamic proxy. We did not generate a proxy class. When you add a Web Service or WCF service to your application, a proxy class is generated automatically by Visual Studio (or you can use the console utility wsdl.exe for Web Services, and scvutil.exe for WCF services). When the service contract or Web Service definition changes, you must update/regenerate your proxy code. In the SCS service, it is completely dynamic. You don't need to generate a proxy class.
Dynamic proxy
A proxy, in its most general form, is a class functioning as an interface to something else. A proxy class implements the same interface with the real class. User can call a method of the proxy object instead of directly calling the same method of the real object. The proxy object can do some actions before/after calling the real object's method, it can call more than one method or does whatever it wants.
We discussed how a proxy class can be implemented in the Mapping messages with classes and methods section. When a user calls a service method, we must get the service contract interface name, method name, and parameter list, create a ScsRemoteInvokeMessage object using these arguments, send the message, and get a response (a ScsRemoteInvokeReturnMessage object) using the RequestReplyMessenger class, and return the method result to the user. Let's examine how we implement it dynamically.
The .NET Framework allows to create dynamic transparent proxy classes by inheriting from the RealProxy (System.Runtime.Remoting.Proxies.RealProxy) class. It has a method named GetTransparentProxy() that returns a transparent proxy object. You can cast this object to any interface. Despite of the object not being an instance of a class that implements the casted interface, it does not throw an exception! It is one of the most interesting classes in the .NET Framework. So, what happens when you call a method of the object over the interface? Surely, it has no implementation (because the same TransparentProxy object can be cast to any interface). So, you must override the RealProxy.Invoke method to decide on what to do on a method call. In the Invoke method, you can get the method name that is called and the parameter list that is passed as the arguments to the method by the user. It does not matter which method is called by the user, you must handle all method calls in the same Invoke method.
The SCS framework uses the RemoteInvokeProxy class (that is derived from RealProxy) to translate method calls to messages dynamically. It is defined as below:
internal class RemoteInvokeProxy<TProxy, TMessenger> :
RealProxy where TMessenger : IMessenger
{
private readonly RequestReplyMessenger<TMessenger> _clientMessenger;
public RemoteInvokeProxy(RequestReplyMessenger<TMessenger> clientMessenger)
: base(typeof(TProxy))
{
_clientMessenger = clientMessenger;
}
public override IMessage Invoke(IMessage msg)
{
var message = msg as IMethodCallMessage;
if (message == null)
{
return null;
}
var requestMessage = new ScsRemoteInvokeMessage
{
ServiceClassName = typeof (TProxy).Name,
MethodName = message.MethodName,
Parameters = message.InArgs
};
var responseMessage =
_clientMessenger.SendMessageAndWaitForResponse(requestMessage)
as ScsRemoteInvokeReturnMessage;
if (responseMessage == null)
{
return null;
}
return responseMessage.RemoteException != null
? new ReturnMessage(responseMessage.RemoteException, message)
: new ReturnMessage(responseMessage.ReturnValue,
null, 0, message.LogicalCallContext,
message);
}
}
RemoteInvokeProxy is a generic class. The first generic parameter (TProxy) is the type of the service contract interface, the second one (TMessenger) is the type of the messenger object (this is needed to use RequestReplyMessenger). It gets a RequestReplyMessenger object that is used to send messages to and get responses from a remote application. The only method is the override of the RealProxy.Invoke() method. Don't be confused because of the IMessage, IMethodCallMessage... interfaces. They are a part of the .NET Framework, and not related to SCS.
In the Invoke method, we create a new ScsRemoteInvokeMessage object. ServiceClassName is the name of the service contract interface type. MethodName and Parameters are set using the input parameter of the Invoke method (this input parameter provides all the information about the method call). Then we send messages and receive responses using the RequestReplyMessenger.SendMessageAndWaitForResponse(...) method. Finally, we supply a return value to the method call using the response message. The implementation of the Invoke method is mostly related to the RealProxy class. We will see how we are creating and using the RemoteInvokeProxy in the following sections.
Calling methods using Reflection
Translating user method calls to messages and sending messages was the first part of remote method invocation. The second part is to call the appropriate method in the remote application according to the received ScsRemoteInvokeMessage object and sending the result of the method call in a ScsRemoteInvokeReturnMessage object. This is simply accomplished by using the Reflection mechanism in the .NET Framework.
There are two different implementations of the method call. One implementation is in the server side, the other one in the client side. They have quite different approaches to determine how to find the appropriate object to invoke its method, but calling the method part is almost the same. I'll examine the client side here (remember that the server can call the client methods remotely; for an example, see the chat system in the first article). So, let's see what happens when a message is received from the server in the client side.
private void RequestReplyMessenger_MessageReceived(object sender,
MessageEventArgs e)
{
var invokeMessage = e.Message as ScsRemoteInvokeMessage;
if (invokeMessage == null)
{
return;
}
if(_clientObject == null)
{
SendInvokeResponse(invokeMessage, null,
new ScsRemoteException("Client does not wait for " +
"method invocations by server."));
return;
}
object returnValue;
try
{
var type = _clientObject.GetType();
var method = type.GetMethod(invokeMessage.MethodName);
returnValue =
method.Invoke(_clientObject, invokeMessage.Parameters);
}
catch (TargetInvocationException ex)
{
var innerEx = ex.InnerException;
SendInvokeResponse(invokeMessage, null,
new ScsRemoteException(innerEx.Message, innerEx));
return;
}
catch (Exception ex)
{
SendInvokeResponse(invokeMessage, null,
new ScsRemoteException(ex.Message, ex));
return;
}
SendInvokeResponse(invokeMessage, returnValue, null);
}
We use the MessageReceived event to receive incoming messages from the server. As the system is running on a remote method call approach, we only accept ScsRemoteInvokeMessage messages (otherwise, we return immediately in the event handler method). We are checking if the user supplied an object to handle the method calls from the server (we did not do this in the sample calculator client in this article, but you can look at the first article for an example of this approach). We are invoking the appropriate method using Reflection, getting the return value, and finally sending a message to the server as a response to the method call. The SendInvokeResponse(...) method creates a ScsRemoteInvokeReturnMessage object using the given parameters and sends it to the server (see source code). We know what's done with this message in the server side (from the Dynamic proxy section above). If the method call throws an exception, we are also sending this exception to the server.
We have finished examining the remote method invocation mechanism over the Messaging Layer in the SCS framework. In the following sections, we will see how the client and server side are implemented using these RMI techniques.
Client side
We have seen most of the client side in the sections above. Also, we created a sample application to see the usage of an SCS service client. Now, we will examine the client side classes in SCS. A service client is created by using the ScsServiceClientBuilder.CreateClient(...) static method as we have seen before. This method is defined as below.
public static IScsServiceClient<T> CreateClient<T>(ScsEndPoint endpoint,
object clientObject = null) where T : class
{
return new ScsServiceClient<T>(endpoint.CreateClient(), clientObject);
}
It returns a IScsServiceClient interface that is implemented by the ScsServiceClient class. The class diagram is shown below.
Client side class diagram that is used to connect to a service
The IScsServiceClient interface extends IConnectableClient. The IConnectableClient interface was explained before. IScsServiceClient just adds two properties: the ServiceProxy property is a dynamic transparent proxy that is used to call remote methods of the server. It is a generic member, and the T parameter is the type of the service contract (that is passed to the ScsServiceClientBuilder.CreateClient() generic method). Timeout is used to set a timeout value on remote method calls. All members of the interfaces are implemented by the ScsServiceClient class. Its constructor is shown below:
public ScsServiceClient(IScsClient client, object clientObject)
{
_client = client;
_clientObject = clientObject;
_client.Connected += Client_Connected;
_client.Disconnected += Client_Disconnected;
_requestReplyMessenger = new RequestReplyMessenger<IScsClient>(client);
_requestReplyMessenger.MessageReceived +=
RequestReplyMessenger_MessageReceived;
_realProxy = new RemoteInvokeProxy<T, IScsClient>(_requestReplyMessenger);
ServiceProxy = (T)_realProxy.GetTransparentProxy();
}
As you have already seen in the ScsServiceClientBuilder.CreateClient() method, ScsServiceClient gets an IScsClient object (as explained before in the Messaging Layer section, this is the main interface that is used to send messages to and receive messages from the server). In the constructor, we create a RequestReplyMessenger object to send/receive messages in request/reply style. Last but not least, we create a RemoteInvokeProxy object and call its GetTransparentProxy() method to create an object that is used to call remote methods of the server. As you can see, we are casting it to T (type of the service contract). Although the T is not known at compile time of the SCS framework and it can be any interface, there is no problem with casting it by means of the special behavior of RealProxy. As we have examined how a method call is made through the client to the server, examining of the client side is finished here.
Server side
As you have seen before, a service application is created by using the ScsServiceBuilder.CreateService(...) static method that is defined as below.
public static IScsServiceApplication CreateService(ScsEndPoint endPoint)
{
return new ScsServiceApplication(ScsServerFactory.CreateServer(endPoint));
}
The CreateService(...) method gets an end point and returns an IScsServiceApplication interface. The class diagram is shown below.
Service application class diagram
The IScsServiceApplication interface defines the methods and events that can be used by the user to manage the server side. It is implemented by the ScsServiceApplication class. The constructor of this class is shown below:
internal ScsServiceApplication(IScsServer scsServer)
{
if (scsServer == null)
{
throw new ArgumentNullException("scsServer");
}
_scsServer = scsServer;
_scsServer.ClientConnected += ScsServer_ClientConnected;
_scsServer.ClientDisconnected += ScsServer_ClientDisconnected;
_serviceObjects = new ThreadSafeSortedList<string, ServiceObject>();
_serviceClients = new ThreadSafeSortedList<long, IScsServiceClient>();
}
It gets an IScsServer object as the only parameter. This object (as we have seen in the Messaging Layer) is the main object that is used to interact with the clients and manage them. Two collections are created in the constructor. The first one is used to store services that are hosted. In an SCS service application, you can run more than one service on the same end point simultaneously. The second collection is used to store the currently connected clients.
After creating a ScsServiceApplication, the AddService(...) method is called by the user to add a service that is hosted by this service application.
public void AddService<TServiceInterface, TServiceClass>(TServiceClass service)
where TServiceClass : ScsService, TServiceInterface
where TServiceInterface : class
{
if (service == null)
{
throw new ArgumentNullException("service");
}
var type = typeof(TServiceInterface);
if(_serviceObjects[type.Name] != null)
{
throw new Exception("Service '" + type.Name + "' is already added.");
}
_serviceObjects[type.Name] = new ServiceObject(type, service);
}
The AddService(...) method has two generic parameters. The first one is the service contract interface that is used by the clients. The second one is the implementation class of the interface. As you have seen in the AddService method implementation, you can add only one object for each service contract interface.
A client is represented by the IScsServiceClient interface and implemented by the ScsServiceClient class in service side (this is different from the client side IScsServiceClient interface). The class diagram is shown below:
Diagram for the class that is used to represent a client in service side
The key method in IScsServiceClient is the GetClientProxy() method. This method is generic, and used to get a reference to a dynamic transparent proxy object that is used to call client methods from the server side. IScsServiceClient also defines RemoteEndPoint that can be used to get end point address (IP address and TCP port for TCP servers) of the client application.
All services that are built on the SCS framework must be derived from ScsService class (as you have seen in the sample applications). The ScsService class has only one public property: CurrentClient. This property is used to get a reference to the client which is invoking the service method. This property is thread-safe. So, even if two clients invoke the same method concurrently, you can get the right client object using the CurrentClient property. Let's examine the ScsService class to see how it is achieved.
using System;
namespace Hik.Communication.ScsServices.Service
{
public abstract class ScsService
{
[ThreadStatic]
private static IScsServiceClient _currentClient;
protected internal IScsServiceClient CurrentClient
{
get
{
if (_currentClient == null)
{
throw new Exception("Client channel can not be obtained. CurrentClient property must be called by the thread which runs the service method.");
}
return _currentClient;
}
internal set
{
_currentClient = value;
}
}
}
}
The CurrentClient is used to get the current client which called the service method. It is used in a method to get a reference to the client object. Note that _currentClient is ThreadStatic, so it is thread safe and gets the right client object even multi clients call the same service method concurrently. Setter is internal and used by SCS system to set the client object before and after calling the service method. Related parts of this operation are shown below:
...
var serviceObject = _serviceObjects[invokeMessage.ServiceClassName];
if (serviceObject == null)
{
SendInvokeResponse(requestReplyMessenger, invokeMessage, null,
new ScsRemoteException("There is no service with name '" +
invokeMessage.ServiceClassName + "'"));
return;
}
try
{
object returnValue;
serviceObject.Service.CurrentClient = client;
try
{
returnValue = serviceObject.InvokeMethod(invokeMessage.MethodName,
invokeMessage.Parameters);
}
finally
{
serviceObject.Service.CurrentClient = null;
}
SendInvokeResponse(requestReplyMessenger, invokeMessage, returnValue, null);
...
Remember that the incoming message is a ScsRemoteInvokeMessage object (that is sent by the client). First of all, we find the service object that will handle this method call using the ServiceClassName property of the message (different clients can use different services). Then we setl the ScsService.CurrentClient property, invoking the requested method using Reflection with the given parameters in the message. Finally, we reset theScsService.CurrentClient property. Thus, user can get the client object by using the CurrentClientt property during method invocation.
User can obtain a transparent proxy object that is used to call remote methods of the client by calling the IScsServiceClient.GetClientProxy() method. This method creates a RemoteInvokeProxy and calls the GetTransparentProxy() method to create a dynamic transparent proxy to the client.
public T GetClientProxy<T>() where T : class
{
_realProxy = new RemoteInvokeProxy<T, IScsServerClient>(_requestReplyMessenger);
return (T)_realProxy.GetTransparentProxy();
}
For a sample application that uses server-to-client method calls, please see the chat system in the first article.
Performance of the SCS framework
Here, I'll show some performance results of the SCS framework. I made tests in my own PC. All communications are made between application those are running on same computer. You can download all test applications in the download file.
Test platform: Intel Core2 Duo 3,00Ghz CPU, 4GB RAM, Windows 7 64bit.
Speed
As you have seen, approximately 5500 remote method call is made with default serialization. Messaging is far more fast. 34,742 messages are sent from one application to another in a second. When I used a custom wire protocol, I could sent 62,410 messages in a second.
Scalability
In my tests, maximum 15,000 concurrent clients connected to a SCS service. Normally, Windows limits open connection count to ~5,000. I added a Windows registry value to allow more connections. To do that, open HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters registry key and add a DWord value MaxUserPort. Default value of MaxUserPort is 5,000. You can change up to 65,534. You may need to restart your computer to take affect. Also, you can find some other parameters to increase TCP connections here: http://smallvoid.com/article/winnt-tcpip-max-limit.html
Even with 15,000 clients are concurrently connected and communicating with the service, total thread count is approximately 50-60 and one client can get response for a method call in 1 milliseconds or less.
Last words
With this article, I have finished examining the SCS framework. In the first article, I focused on the usage of the framework. I examined two different sample applications: a phone book and an IRC chat system. The chat system especially demonstrates the power of the framework. In this article, I discussed how to build a framework upon TCP/IP that allows the user to invoke remote methods as simple as Web/WCF services while it is connection oriented. Then I explained almost all constructions of the framework. It's a stable, robust and scalable framework. If you find bugs, please inform me.
Changes/Updates
- 13.06.2011 (v1.1.0.1)
- BugFix: Ping messages must not be raised by messaging layer.
- 30.05.2011 (v1.1.0.0)
- Additions:
- ClientDisconnected event added to IScsServiceApplication and IScsServer classes.
- MessageSent event is added to IMessenger interface.
- SynchronizedMessenger class is added to receive messages as synchronized.
- Changes/Improvements:
- Changed background thread mechanism to provide more scalable framework. Used TPL Tasks and Async sockets instead of directly use of threads and blocking sockets (Added SequentialItemProcessor class).
- Added IScsWireProtocolFactory interface and changed IScsServer.WireProtocol to IScsServer.WireProtocolFactory. Also, IScsWireProtocol is completely changed.
- BinarySerializationProtocol class is made public to allow user to override serialization methods.
- Codes completely revised, some parts refactored and commented.
- Bugfix:
- Fixed a potential minor bug in Timer.
- Article:
- This article is completely updated according to changes in the framework.
- 11.05.2011 (v1.0.2.0)
- Added the
RemoteEndPoint property to get the addresses of clients in server side. - Download files revised.
- 10.04.2011 (v1.0.1.0)
- Added the
ConnectTimeout property to IConnectableClient to provide a way of setting the timeout value while connecting to a server.
