Creating Threads based on processors/cores

Question

5.00/5 (2 votes)

See more:

I'm not clear on how the whole threading process works with regards to the optimal number of threads to create. I've heard/read that I should only create a number of threads that corresponds to the number of processors my machine is running. Example: 4 threads for a quad core processor, 2 threads for a dual core processor, etc...

So what happens if I'm running a quad core machine and I create 10 threads to handle some really intensive calculations and database operations?

The reason I ask this is because I'm actually running a quad core machine and doing intensive math calcs and database operations (inserts and updates) in realTime and the only way that my app runs efficiently is with 10 threads. If I run anything less performance degrades.

Can someone please clarify to me what's going on?

Thanks in advance,

-Donald

Posted 18-Apr-11 9:35am

d.allen101

Updated 18-Apr-11 9:36am

v2

Add a Solution

Comments

AspDotNetDev 18-Apr-11 18:17pm

Are each of your cores hyperthreaded? If so, that could be why you are seeing an improvement around 10 threads (4 cores x 2 = 8 virtual cores). Could also be that some of your threads are spawning on the same core (in which case, it would be somewhat random and would not really be related to the number of threads, though more threads might be more likely to end up on more cores).

3 solutions

Solution 1

The number of viable runnning threads really has nothing to do with how many cores you have since .Net uses the available cores as it sees fit. It also depends pretty much on what the threads are doing, and whether they're in a waiting state or actively processing something.

There is a break-even point somewhere, but I'm not sure if there's any reliable way to predict it short of having the app monitor it's own overall performance (of course, that would be handled in its own thread - grin), and adjust the maximum number of concurrently running threads on based on soe hard-coded threshold.

If ten is a good number in your environment, then ten is a good number.

.Net 4 introduced parallel tasks, which may (or may not) be more efficient.

Posted 18-Apr-11 9:45am

#realJSOP

Comments

d.allen101 18-Apr-11 15:51pm

thanks for the reply john. just to give you a little more info, I'm basically running a blocking queue thats threaded and I'm running 10 threads to perform calculations on data then insert and update my database with the calc data. So what the correct way to determine the number of threads need? I found 10 by trial and error with my app? Also, how do I find out the "breaking point" for too many threads?

Sergey Alexandrovich Kryukov 18-Apr-11 17:43pm

I agree with you. My 5. I pointed out some cases when you can get benefit of thread affinity, pretty marginal in my opinion.
Please see my Solution.
--SA

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Sergey Alexandrovich Kryukov · Accepted Answer · 2011-04-18T11:41:00

Solution 2

In addition to what John said: You can control of the affinity of some threads to some cores: use System.Threading.Thread.BeginThreadAffinity, System.Threading.Thread.EndThreadAffinity. At the level of the process, you can use System.Diagnostics.Process.ProcessorAffinity.

Doing it a normal situation is not recommended and would be usually pointless. One reason for such trick I can see is serving some "bad" hardware requiring heavy regular polling; it will make the load of other cores/CPUs more predictable.

—SA

Posted 18-Apr-11 11:41am

Sergey Alexandrovich Kryukov

Comments

Espen Harlinn 18-Apr-11 18:22pm

Good points, my 5

Sergey Alexandrovich Kryukov 18-Apr-11 21:52pm

Thank you, Espen.
--SA

El_Codero 5-Mar-12 18:16pm

Thanks for sharing this trick. Best Regards

Sergey Alexandrovich Kryukov 5-Mar-12 18:28pm

Thank you, Björn, but this is not a trick, and it is usually not needed.
--SA

El_Codero 5-Mar-12 18:33pm

But it could be a trick if non standard hardware is used? Do you maybe know approximately how many percent it could be increased (generally)?

Sergey Alexandrovich Kryukov 5-Mar-12 19:11pm

I'll explain what did I mean. It's not when it is non-standard, it's when it's lousy. :-)

Good hardware with good API uses inversion of control and thus works by interrupts. The example is the network adapter. When you read the network stream in a thread, this is a blocking call and the thread waste zero CPU time being frozen in a wait state. When data comes, a thread is waken up and works immediately. How? Because the adapter itself generates the hardware interrupt, driver interrupt procedure wakes up the thread or whatever through IPC subscribed to the hardware event -- stuff like that. Due to such architecture, you don't need any polling of the device.

For widely used types of devices, everything is pretty good. But it hardware and hardware drivers are "professional" (read, manufactured for industry, expensive due to much smaller market), the stuff is often pretty ugly. Due to much rigid market relationships and reduced freedom of competition and also cultural barriers, many "professional" company produce bad stuff. On mass production marker such stuff is more easily wiped by competition. And their programmers often have no idea how to program. As a result, the devices which require periodic and frequent polling are way to usual. My work around is: to dedicate (sacrifice) a CPU code for them through affixing.

Gradually, the situation improves, but mostly by moving processing to the other end of Ethernet cable, through getting rid of everything except network compatibility. In this cases, it's good to create a machine-wide network with a separate switch and be happy, especially if the network adapter it is separate. Many machines can be build like that. But event, directly inserted PCI card or even RS-232 are pretty common or occasional. And such devices often require frequent polling at the expense of the PC's CPU, which is sad.

--SA

Sergey Alexandrovich Kryukov 5-Mar-12 19:14pm

It's totally impossible to evaluate any percent or something like that, because it depends. The point is not even improving any throughput. The point is to make responsiveness of applications and operation of other processes and thread alive and not dependent on the work load in one of the cores. This is often more important. After all, who would tolerate is some unrelated UI stalls or delays just because some hardware does... who knows what?
--SA

El_Codero 6-Mar-12 7:11am

Thanks SAKryukov for your explanation. I made the same experience with some ugly drivers and they're often annoying and really bad programmed (both for business and end user devices). I'm trying to summarize your message, please correct me one more time if I mixed something wrong.
I think you mean if you have performance issues you should check both sides of the architecture and structure of the system. This means it could be a bottleneck (or I should say you can increase performance) through optimizing / extension of network hardware/architecture. I.e if you not happy with integrated NIC because of polling and let the CPU working, you can/should optimize it through a quality network card (i.e. accoring to my expierence Intel-Network Cards doing a great job and do the work well without polling the cpu). But normally it mostly depends how efficient your code is working. I'm right? Thanks for your time and explanation, Best Regards

Sergey Alexandrovich Kryukov 7-Mar-12 0:35am

Yes, but a network card is just one example. The root problem is the polling itself and the way to tackle it if it is unavoidable because you use some less-then-perfect architecture. From the performance and throughput point of view, polling is always flawed, because there is no optimal solution for polling: if you poll too fast, you waste performance for nothing as nothing will be changed in the system during many polls; ultimately, you reduce responsiveness due to overuse of the CPU and communication channel resources. But if you poll less frequently, you loose events (respond too late) and reduce responsiveness any way. And if you try to optimize it, you don't reach the goal, because the system is stochastic, so you never can guarantee it provide required responsiveness.

At the same time, inversion of control and moving from pull approach (like polling) to push approach is the ideal solution. This is explained here:

http://en.wikipedia.org/wiki/Push_technology
http://en.wikipedia.org/wiki/Inversion_of_control

Now, the problem is: how to solve the problem is push technology is impossible simply because the subsystem you are polling is not accessible for improvements. (Proprietary, closed-source, unfeasible to improve due to limited resource or involved risks, etc.). What to do in this case?

It's also good to understand that the polling is maybe unavoidable in some lower level of some subsystem, such as some external device. But, the important difference here is this: 1) polling is done in some limited and more regularly-timed environment, in contrast to OS; 2) even more importantly, it does not happen at the expense of a CPU, or, more exactly, for some CPU-enabled machines (a typical example: a real-time host playing the role of servo system controller), -- it happens not at the expense of your CPU.

That understanding leads us to the idea: when we have to do polling at the expense of our CPU, you can dedicate one code to one or more polling tasks where timing is more regular and predictable. We kind of simulate external device which would do polling on its own at the expense of the separate CPU.

Does this reasoning make sense to you?

Thank you,
--SA

El_Codero 8-Mar-12 19:52pm

Just added this discussion to my bookmarks (and my brain). Thanks for sharing your expierences, it really makes sense to me!

With Best Regards
Björn Ranft

Sergey Alexandrovich Kryukov 8-Mar-12 19:58pm

You are very welcome.
Thank you for this interesting discussion, too. This is a cycle of problems we often discuss in the scope of hardware control and some other similar fields.

--SA

Espen Harlinn 9-Mar-12 4:26am

I think you will find that this paper is a good reference for the design you are trying to explain:
http://www.cs.wustl.edu/~schmidt/PDF/PLoP-95.pdf

Sergey Alexandrovich Kryukov 9-Mar-12 23:44pm

Thank you for the reference, it looks interesting; I'll read it later.
--SA

Espen Harlinn · Accepted Answer · 2011-04-18T12:36:00

Solution 3

In your case the usual approach is to create one calculation thread for each core, offload IO to separate threads - possibly using separate computers for staging (including storing) of incomming data, and similarly try to offload storing of results to separate computers - often using a message queing solution.

Staging computer 1 ---
                        \
Staging computer 2 ---   \
                           calculating computer enques results to ---> storage solution  
Staging computer 3 ---   /

Staging computer 4 --- /

Moving IO out of the way is often the only real solution when you're doing heavy calculations. Database IO is *much* more expensive then enquing results to a message quing solution, especially if you're using something that handles load balancing in an intelligent way.

Regards
Espen Harlinn

Posted 18-Apr-11 12:36pm

Espen Harlinn

Comments

Sergey Alexandrovich Kryukov 18-Apr-11 21:55pm

Very good advices, my 5. However, tasks also should be considered and the effect or it... depends, I agree with John.
--SA

Espen Harlinn 19-Apr-11 8:09am

Take a look at OPs' previous postings, as he is trying to deal with a fair amount of incomming data the calculations probably needs all available CPU, setting thread affinity and doing as little as possible, apart from the computations, is usually the only real option when you're dealing with quants in "real-time". In this scenario database access is often way too expensive to execute on the calculating/match searching computer. Spreading the load between several computers for a single instrument is often not an option due to IO overhead.

JSOPs' reply doesn't really show anything about how to get the most CPU cycles for the calculation as his approach often will degrade the computation by as much as 50%, maybe more.

Sergey Alexandrovich Kryukov 19-Apr-11 13:22pm

Well, John does't try to show that, it's just a general considerations, one valid point is the possibility to use Task to make threading... well, implicit and declarative...
Your notes are very good. How I/O is arranged in threading model is really a key.
--SA

d.allen101 21-Apr-11 1:05am

hey harlinn, can you point me to 'specific' threading topics that I need to understand in order to accomplish and understand exactly what you and SAKryukov are talking about? The two of you are talking over my head but I NEED/WANT to understand where you guys are coming from!

Espen Harlinn 21-Apr-11 9:24am

I'll try to get back to you on monday, it's easter time :)

Sergey Alexandrovich Kryukov 8-Mar-12 20:12pm

Hi Espen,

I don't know why Allen addresses to you on the topic I tried to explain in my post, but he refers to my answer (please see) where I warn against touching thread affinity or other "system" characteristics of threads.

In most cases the structure of thread use should be based on logical behavior and thread communications without touching any "system" characteristics. You explains one of the aspect of such approach.

At the same time, I explain some cases where thread affinity can be used. In a nutshell, it could be used when we deal with some flawed architectural elements and need some workaround. In fact, since the time of last Donald's post, we had very interesting discussion with Björn Ranft, where we discussed the problem and the approach in much more detail.

Please see.

@Donald:

Please take a look at the discussion I just mentioned. Perhaps the problem would be more clear to you. Most likely, it would be not related to your problems, because in most cases, the best approach in not touching the thread affinity (association between threads and CPU cores).

Perhaps you could formulate some particular questions, because right now I don't understand how else could I help.

--SA

Espen Harlinn 9-Mar-12 4:20am

I made this http://www.codeproject.com/Articles/168688/Microsoft-Message-Queuing-Log-Trade-Information-us in an effort to show how easily it can be done. The design is pretty simplistic, and improving the performance should be pretty easy: do your own serialization and post multiple records to the database during each call to command.ExecuteNonQuery();

Nish Nishant 19-Apr-11 16:17pm

Voted 5!

Espen Harlinn 19-Apr-11 16:24pm

Thank you Nishant!