High Performance TCP Client Server using TCPListener and TCPClient in .NET Standard

• Download source code - 11.3 KB

Introduction

With .NET 5 on the horizon and those needing to migrate away from .NET 4.8 and .NET Core, this source code is intended to provide an example of how to establish a high-performance cross-platform Client Server messaging exchange via TCP, native to .NET Standard, without requiring specific ties to .NET Framework or .NET Core or anything else.

It additionally solves a common problem encountered with TCP sessions: The Messaging Spin Lock issue and the Asynchronous Memory Leak issue and/or the CancellationToken Exception issue that is all too common in TCP Client Server implementations.

TCP Server

For the sake of simplicity, we will use a CLI project, the project type itself could be either .NET 5 or .NET Core or .NET Framework. The client and server are coded in .NET Standard syntax and so they can interface with all three of those seamlessly.

Typical CLI Main() block for the Server host:

using System;

public static class Program
{
    public static void Main()
    {
        Console.WriteLine("Press esc key to stop");

        int i = 0;
        void PeriodicallyClearScreen()
        {
            i++;
            if (i > 15)
            {
                Console.Clear();
                Console.WriteLine("Press esc key to stop");
                i = 0;
            }
        }

        //Write the host messages to the console
        void OnHostMessage(string input)
        {
            PeriodicallyClearScreen();
            Console.WriteLine(input);
        }

        var BLL = new ServerHost.Host(OnHostMessage);
        BLL.RunServerThread();

        while (Console.ReadKey().Key != ConsoleKey.Escape)
        {
            Console.Clear();
            Console.WriteLine("Press esc key to stop");
        }

        Console.WriteLine("Attempting clean exit");
        BLL.WaitForServerThreadToStop();

        Console.WriteLine("Exiting console Main.");
    }
}

Basic CLI plumbing here, nothing unusual.

Esc Key to exit the Client window, and it clears the window every 15 messages (used only for debugging/demonstration purposes). Don't write the actual network messages to the console in your Production version.

The only key takeaway from this block of code is that in order to maintain high performance, the Client and Server are hosted in a dedicated thread. That is, separate from the thread that executes the Main block. The logic for that is contained in the RunServerThread() function.

To establish that, we will create a Host class and add it to a .NET Standard Library project type, .NET Standard Libs can be referenced by .NET 5, .NET Framework and .NET Core projects so it really is the most flexible option available at the time of writing this.

Add the following code to it:

using System;
using System.IO;
using System.Net.Sockets;
using System.Threading;

public class Host
{
    #region Public Functions
    public virtual void RunServerThread()
    {
        this.ServerThread.Start();
        this.OnHostMessages.Invoke("Server started");
    }

    public virtual void WaitForServerThreadToStop()
    {
         this.Server.ExitSignal = true;
         this.OnHostMessages.Invoke("Exit Signal sent to server thread");
         this.OnHostMessages.Invoke("Joining server thread");
         this.ServerThread.Join();
         this.OnHostMessages.Invoke("Server thread has exited gracefully");
    }
    #endregion
}

The RunServerThread() function starts the Thread that will run the server.

The WaitForServerThreadToStop() function signals to the Server Thread that it should exit gracefully at the soonest possible time, we then Join the Server Thread to the calling Thread (the Main() thread in this case), otherwise the CLI window would just terminate/abort, which we don't want to do from a cleanup/exception handling standpoint; the graceful/clean exit is preferable.

Add the supporting variables and Constructor to the Server Host class:

#region Public Delegates
public delegate void HostMessagesDelegate(string message);
#endregion

#region Variables

protected readonly StandardServer.Server Server;
protected readonly Thread ServerThread;

#region Callbacks
protected readonly HostMessagesDelegate OnHostMessages;
#endregion

#endregion

#region Constructor
public Host(HostMessagesDelegate onHostMessages)
{
    this.OnHostMessages = onHostMessages ?? 
         throw new ArgumentNullException(nameof(onHostMessages));
    this.Server = new StandardServer.Server(this.OnMessage, this.ConnectionHandler);
    this.ServerThread = new Thread(this.Server.Run);
}
#endregion

#region Protected Functions
protected virtual void OnMessage(string message)
{
    this.OnHostMessages.Invoke(message);
}
        
protected virtual void ConnectionHandler(NetworkStream connectedAutoDisposedNetStream)
{
}
#endregion 

• ServerThread: The thread that hosts the server (Server class)
• OnHostMessages and OnMessage: Used only for demo purposes, pushes messages from the TCP connector thread to the Client CLI Window. (Note: For WinForm apps, you'd have to perform an ISynchronizeInvoke across the GUI thread if you want to display messages to end users. Console.Write doesn't have this limitation.)
• ConnectionHandler: We will get back to this.
• Server: The TCP Server class code that we are about to write.

Create a class called Server and add the following code to it:

using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.Threading.Tasks;

public class Server
{
    #region Public Functions
    public virtual void Run()
    {
        if (this.IsRunning)
            return; //Already running, only one running instance allowed.

        this.IsRunning = true;
        this.ExitSignal = false;

        while (!this.ExitSignal)
            this.ConnectionLooper();

        this.IsRunning = false;
   }
   #endregion
}

Since Run() is running in a dedicated thread, you may want to add a Try Catch block to debug any unhandled exceptions that are thrown and add logging as required. For the sake of clarity, I'll leave that up to you.

The Run() function processes the ExitSignal checks and pushes the rest of the TCP logic to the ConnectionLooper() function. At this point, you are probably thinking BUT this is an infinite Spin Loop, you'd be correct but we are about to resolve that with an await.

Add the supporting variables and Constructor to the Server class:

#region Public Properties
private volatile bool _ExitSignal;
public virtual bool ExitSignal
{
    get => this._ExitSignal;
    set => this._ExitSignal = value;
}
#endregion

#region Public Delegates
public delegate void ConnectionHandlerDelegate(NetworkStream connectedAutoDisposedNetStream);
public delegate void MessageDelegate(string message);
#endregion

#region Variables

#region Init/State
protected readonly int AwaiterTimeoutInMS;
protected readonly string Host;
protected readonly int Port;
protected readonly int MaxConcurrentListeners;
protected readonly TcpListener Listener;
        
protected bool IsRunning;
protected List<Task> TcpClientTasks = new List<Task>();
#endregion

#region Callbacks
protected readonly ConnectionHandlerDelegate OnHandleConnection;
protected readonly MessageDelegate OnMessage;
#endregion

#endregion

#region Constructor
public Server(
              MessageDelegate onMessage, 
              ConnectionHandlerDelegate connectionHandler, 
              string host = "0.0.0.0",
              int port = 8080, 
              int maxConcurrentListeners = 10,
              int awaiterTimeoutInMS = 500
             )
{
     this.OnMessage = onMessage ?? throw new ArgumentNullException(nameof(onMessage));
     this.OnHandleConnection = connectionHandler ?? 
          throw new ArgumentNullException(nameof(connectionHandler));
     this.Host = host ?? throw new ArgumentNullException(nameof(host));
     this.Port = port;
     this.MaxConcurrentListeners = maxConcurrentListeners;
     this.AwaiterTimeoutInMS = awaiterTimeoutInMS;
     this.Listener = new TcpListener(IPAddress.Parse(this.Host), this.Port);
}
#endregion

Nothing fancy going on here, the only note is that the _ExitSignal member variable is of type volatile which helps prevent stale get/sets between the CLI Main() thread and the Server Host Thread. This is simpler for demo-purposes than a Lock or Mutex, and is potentially less CPU/memory intensive.

IP 0.0.0.0 is IPAddress Any. You can change or remove defaults as-desired.

Here, we maintain a List of TcpClient connection Tasks (async tasks), this could be an array of size(maxConcurrentListeners) instead of a List. It might run a few microseconds faster if you do.

OnMessage is only used for demo purposes, to display the messages in the CLI window.

OnHandleConnection is the callback where the consumer of this class will code their network logic specific to their business case.

Add the following code to implement the ConnectionLooper() function:

#region Protected Functions
protected virtual void ConnectionLooper()
{
     while (this.TcpClientTasks.Count < this.MaxConcurrentListeners)
     {
        var AwaiterTask = Task.Run(async () =>
        {
             this.ProcessMessagesFromClient(await this.Listener.AcceptTcpClientAsync());
        });
        this.TcpClientTasks.Add(AwaiterTask);
     }

     int RemoveAtIndex = Task.WaitAny(this.TcpClientTasks.ToArray(), this.AwaiterTimeoutInMS);
     
     if (RemoveAtIndex > 0) 
        this.TcpClientTasks.RemoveAt(RemoveAtIndex);
}
#endregion

Here, our server is asynchronously listening to a quantity of TCP connections (limited to int MaxConcurrentListeners), this keeps the internal .NET ThreadPool threads from being exhausted. This is dependent on the number of CPU cores on the vm/server/host. The beefier your host, the more concurrent listeners you'll be able to support.

The Task await establishes a thread continuation which proceeds into ProcessMessagesFromClient when a client successfully connects, that function is where the actual processing of the network communications occur.

We then WaitAny() on the list of these awaitable Tasks, but only for a given amount of milliseconds, this part is key. Without the timeout, we wouldn't be detecting changes to the ExitSignal variable which is key for a graceful exit in a multi-threaded environment. The WaitAny prevents a spin lock while also detecting exited TCPListener Tasks (.NET internal ThreadPool threads).

It also avoids memory leaking an infinite/excessive number of awaits or threads or tasks, while still asynchronously processing connections.

AcceptTcpClientAsync throws an Exception if the Task is aborted, you may desire to provide each Task with a CancellationToken and gracefully exit each Task in the Task List if your requirements require that. Since we never call Listener.Stop(), when the host Thread gracefully exits, that cleanup is being handled internally by the .NET GC in this example code.

When WaitAny detects that a Task has completed the RemoveAt Index is returned so that we can simply remove that Task from our List, a new one is re-added on the next pass after an ExitSignal check is performed.

Add the following code to implement the ProcessMessagesFromClient() function:

protected virtual void ProcessMessagesFromClient(TcpClient Connection)
{
    using (Connection)
    {
         if (!Connection.Connected)
            return;

         using (var netstream = Connection.GetStream())
         {
            this.OnHandleConnection.Invoke(netstream);
         }
    }
}

This function runs in a continuation ThreadPool thread when successfully connected, generally it won't be the same thread as the ServerThread or Main() thread. ConfigureAwait(false) can be added if you need that strictly-enforced.

The TCPClient and NetworkStream is automatically closed & disposed with the Using syntax, in this way, the consumer of this class doesn't need to concern itself with that cleanup.

The OnHandleConnection.Invoke essentially calls the ConnectionHandler() function from the Host class, which we will now write:

protected virtual void ConnectionHandler(NetworkStream connectedAutoDisposedNetStream)
{
     if (!connectedAutoDisposedNetStream.CanRead && !connectedAutoDisposedNetStream.CanWrite)
        return; //We need to be able to read and write

     var writer = new StreamWriter(connectedAutoDisposedNetStream) { AutoFlush = true };
     var reader = new StreamReader(connectedAutoDisposedNetStream);

     var StartTime = DateTime.Now;
     int i = 0;
     while (!this.Server.ExitSignal) //Tight network message-loop (optional)
     {
          var JSON_Helper = new Helper.JSON();
          string JSON = JSON_Helper.JSONstring();

          string Response;
          try //Communication block
          {
              //Synchronously send some JSON to the connected client
              writer.WriteLine(JSON);
              //Synchronously wait for a response from the connected client
              Response = reader.ReadLine();
          }
          catch (IOException ex)
          {
              _ = ex; //Add Debug breakpoint and logging here
             return; //Swallow exception and Exit function on network error
          }

          //Put breakpoint here to inspect the JSON string return by the connected client
          Helper.SomeDataObject Data = JSON_Helper.DeserializeFromJSON(Response);
          _ = Data;

          //Update stats
          i++;
          var ElapsedTime = DateTime.Now - StartTime;
          if (ElapsedTime.TotalMilliseconds >= 1000)
          {
              this.OnHostMessages.Invoke("Messages per second: " + i);
              i = 0;
              StartTime = DateTime.Now;
          }
     }
}

This is mostly boiler plate network messaging code that you will code yourself. If you only need to push 1 message, then remove the while-loop, keeping the ExitSignal check as an if-statement doesn't hurt, the volatile bool can be checked as often as needed.

The network stream supports both string lines and binary byte array transmissions.

In this particular demo, we are serializing and de-serializing an object into JSON XML as a string and sending back and forth via Write/Read-Line (.NET converts it automatically into an array of ASCII char bytes[], if you desire other charsets .NET has several conversion helpers.)

One keynote here is that Writes and Reads to the stream may throw an IOException if the connection is lost for any reason, you will need to handle that in Try...Catch block. If you need to handle or bubble exceptions, you can use throw; instead of return;

As noted, the Usings block will cleanup the network connection when the Thread or Task Continuation completes or aborts/throws.

Generally, I never use throw ex; as that discards the top-most exception info, where as throw; maintains the full stack trace/inner exceptions.

TCP Client

The CLI and Host classes for the TCP Client is practically identical to the TCP Server, the full version is attached and downloadable.

The only difference illustrated below is that the clients use TCP Client rather than TCP Listener, and the ConnectionLooper() function is simpler since we aren't dealing with multiple concurrent inbound connections.

using System;
using System.Net.Sockets;
using System.Threading;

#region Public Functions
public virtual void Run()
{
    if (this.IsRunning)
       return; //Already running, only one running instance allowed.

    this.IsRunning = true;
    this.ExitSignal = false;

    while (!this.ExitSignal)
        this.ConnectionLooper();

    this.IsRunning = false;
}
#endregion

#region Protected Functions
protected virtual void ConnectionLooper()
{
     this.OnMessage.Invoke("Attempting server connection... ");
     using (var Client = new TcpClient())
     {
         try
         {
            Client.Connect(this.Host, this.Port);
         }
         catch(SocketException ex)
         {
             this.OnMessage.Invoke(ex.Message);
             //Server is unavailable, wait before re-trying
             Thread.Sleep(this.ConnectionAttemptDelayInMS); 
             return; //Swallow exception
         }

         if (!Client.Connected) //exit function if not connected
             return;

         using (var netstream = Client.GetStream())
         {
             //Process the connection
             this.OnHandleConnection.Invoke(netstream); 
         }
    }
}
#endregion

Here, we only use a Thread Sleep, if you host the client in a Task (ThreadPool manager), then Task.Delay is preferred.

If you don't need an external ExitSignal for graceful exit, by all means remove it as that will simplify the implementation, i.e., your process would exit on its own when it is ready to do so. I only added it to the demo example to show how it could be done for those that need to have that feature.

The containing While loop is not required if you are only making a singular connection or intermittent connections. In this example case, we are waiting infinitely for the server to come online, you will need to customize that in accordance to your requirements if they differ.

Additionally, if you need to make multiple concurrent client connections from the same process, you can do that too, and in that case, you will definitely want to use a comparable design to the Server class, which utilizes an awaitable WaitAny() asynchronous TaskList pattern rather than just a simple Synchronous Connect() loop shown in this example code.

Conclusion

Hopefully, someone finds this code useful.

I was able to achieve up to 74000 bi-directional JSON exchanges with it using just a single core VM so performance should be decent. Server in .NET4.8 and Client in Core3.1 for hosts in this example case.

If you aren't serializing object, it should run even faster, likely bottlenecked only by the network card bandwidth or CPU speed.

History

• 10^th June, 2020 - Version 1.0