Determining Threading Limits For Multi-Threaded Network Applications

Disclaimer

This article will explain a method of benchmarking a W9X/W2K/XP based computer to determine the maximum safe level of thread usage while keeping the system able to process other input without entering into a freeze condition. Please be aware that the concepts and methods contained in this article are simply the author's interpretation of available material on the subject and should not be considered the only way to perform this type of test. Additionally, the author will accept no claims of liability arising from system damage, loss of data, or any other adverse effects caused as a result from the readers' implementation of any of the methods described within this article.

Introduction

As a programmer, it is often important to know the system/resource needs of your software applications. This is especially important if you are writing a multithreaded network application. Often you need to know how many client threads your typical target system is capable of supporting. The reason this is important is that most, if not all of the time, our application should be well behaved and not claim too many of the systems resources. We all know that there is nothing worse than an unresponsive application that uses up so many system resources processing information, that we're unable to do any actions until the program is finished. The problem is that too often, we can't be sure what the target system is capable of supporting. Fortunately, I have developed a method to give a reasonable estimate of the thread amount that can safely be run on a target machine without bogging down the system. With a reasonably small amount of math, we can determine this value.

The Concept and Methodology

First, we need to obtain the following values:

1. The CPU speed in MHZ
2. CPU usage with OS loaded
3. Total available memory
4. Total memory with OS loaded
5. Typical CPU usage of a single thread
6. Typical memory usage of a single thread

With these values, we have a template on which to base our calculations. We will start by subtracting the CPU Usage with the OS loaded from the CPU Default Speed. We then subtract 25% from this value. The reason for this operation is to give us some indication of the total amount of CPU time slices that we have to work with, and provide some latitude to allow other applications to run in concert with the server without impacting the target machine to an extreme degree. The operation would be notated like this:

(CPU_TotalAvailable - CPU_UsageWithOS) - 25% = CPU_TotalAvailableForThreads

CPU_TotalAvailableForThreads is the maximum amount that can be used to execute threads for our application. It assumes that the program will want to use no more than 25% of the total remaining CPU to run our process. This is a reasonable assumption since it's generally considered rude for an application to use all available resources. Nobody likes a hog :-)

In order to determine the amount of memory we have to work with, we perform the same procedure that we used to determine the available CPU. Ideally, we would like to run entirely in memory to avoid paging, which can also cause system slowdown. The operation would notate as this:

(Mem_TotalAvailable - Mem_UsageWithOS) - 25% = Mem_TotalAvailableForThreads

Mem_TotalAvailableForThreads is the maximum amount that can be used to execute threads for our application. This also assumes that the program will want to use no more than 25% of the total remaining memory to run our process.

Now comes the fun part. Create your thread and profile it in such a way as to determine how much CPU and memory it uses in a single instance. This will give you a baseline value to work with. For purposes of this example, let's say that one thread uses 512 KB of memory and .5% of the CPU. Now using these values against the values we obtained earlier, you can divide the memory and CPU usage values into the Mem_TotalAvailableForThreads and CPU_TotalAvailableForThreads to determine two new values CPU_ThreadsMaxAllowable, like this:

CPUTotalAvailable / .5% = CPU_ThreadsMaxAllowable

and

MemTotalAvailableForThreads / 512KB = Mem_ThreadsMaxAllowable

Alone, neither of these two values will do us much good as they are fairly ambiguous, so we add them together and divide by two, to give us MaxAllowableThreads which is a balance between the CPU and Mem.

In Closing

All said and done, I think this is a reasonable way to determine a self level for thread usage in multithreaded network applications. This could be done from inside the application, to determine it's thread usage at runtime, or on a test system that closely mimics the intended installation platform. Good luck, and I hope you find this article useful.