if work received can be performed independent of each other, we could always spawn a thread for processing that piece of work. the problem here is that an operating system like windows has severe performance problems when a large number of threads are created or running at the same time, waiting to have access to the cpu. the windows operating system needs to manage all of these threads, and compared to the unix operating system, it just doesn’t hold up. if large amounts of work are issued to the server, this model will most likely cause the windows operating system to become overloaded. system performance will degrade drastically.
this article is a case study comparing thread performance between windows nt and solaris.
in the .net framework, the “system.threading” namespace has a threadpool class. unfortunately, it is a static class and therefore our server can only have a single thread pool. this isn’t the only issue. the threadpool class does not allow us to set the concurrency level of the thread pool. the concurrency level is the most important setting when configuring a thread pool. the concurrency level defines how many threads in the pool may be in an “active state” at the same time. if we set this parameter correctly, we will have the most efficient, performance enhanced thread pool for the work being processed.
imagine we have a thread pool with 4 threads and a concurrency level of 1. then, three pieces of work are queued up for processing in the pool. since the concurrency level for the thread pool is 1, only a single thread from the pool is activated and given work from the queue. even though there are two pieces of work queued up, no other threads are activated. this is because the concurrency level is set to 1. if the concurrency level was set to 2, then another thread would have been activated immediately and given work from the queue. in diagram b we have thread 1 running and all of the other threads sleeping with two pieces of work queued.
so the question exists, why have more than 1 thread in the pool if the concurrency level is set to 1? if thread 1 in diagram b ever goes to sleep before it completes its work, another thread from the pool will be activated. when thread 1 goes to sleep, there are 0 threads “active” in the pool and it is ok to activate a new thread based on the concurrency level. in diagram c, we now have thread 1 sleeping and thread 4 running with one piece of work queued.
eventually, thread 1 will wake up, and it is possible for thread 4 to still be active. we have 2 threads active in the pool, even though the concurrency level is set to 1. in diagram d, we now have thread 1 and thread 4 running and one piece of work still queued.
