http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660I was under the impression that the TPL was for taking advantage of multiple cores for *processor-intense* operations. The processor is not your limiting factor here, it's the inherent latency in the PostTo operation which requires thread management. I would thing you'd be better off with something like ThreadPool to do that. http://ayende.com/4476/unbounded-concurrency#comment20http://ayende.com/4476/unbounded-concurrency#comment20Sat, 01 May 2010 16:38:47 GMTThe thing is your client is working as hard as it can but your server may not be. Even if you remove that limit you would want to set the MaxDegreeOfParallelism to approximate the number of cores on your system. Better yet use ASYNC IO instead of Parallel.ForEach avoid all the blocking and use fewer threads. I use Jeffrey Richter's AsyncEnumerator for this and it's great. http://ayende.com/4476/unbounded-concurrency#comment19http://ayende.com/4476/unbounded-concurrency#comment19Thu, 29 Apr 2010 17:28:40 GMTDan, I removed the 2 connection limit, as mentioned in the post http://ayende.com/4476/unbounded-concurrency#comment18http://ayende.com/4476/unbounded-concurrency#comment18Thu, 29 Apr 2010 17:20:09 GMTIf you want to see how fast RavenDB can take documents then this test won't do that well. You want your client to do at most 2 posts concurrently perhaps using MaxDegreeOfParallelism as suggested by James. 2 is the default max number of connections to one remote host that WININET will allow. When you allow more than the max WININET limit all the threads above 2 will block as soon as they try to connect. In order to get a real stress test of RavenDB you will need to increase the WININET connection limit to match the number of cores in your system or you will need multiple client machines all doing 2 at a time. I think I would go for the multiple machine approach and use 4 or 5 all running two threads concurrently posting documents. http://ayende.com/4476/unbounded-concurrency#comment17http://ayende.com/4476/unbounded-concurrency#comment17Thu, 29 Apr 2010 17:14:51 GMT@James, what value did you use for MaxDegreeOfParallelism? http://ayende.com/4476/unbounded-concurrency#comment16http://ayende.com/4476/unbounded-concurrency#comment16Thu, 29 Apr 2010 16:43:54 GMTI ran into a very similar issue the other day despite setting MaxDegreeOfParallelism to a reasonable number. I wonder if there is something else at play here? http://ayende.com/4476/unbounded-concurrency#comment15http://ayende.com/4476/unbounded-concurrency#comment15Thu, 29 Apr 2010 16:38:59 GMTIt seems to me that the PostTo operation on the client is IO bound* and not CPU bound. Therefore I think you may be better using the APM model instead of allocating threads to each task. From what I understand the APM model will leverage the Completion Port technology and therefore not steal so many threads....this leaves the threads available for the server (localhost) to the actually process the requests. *I imagine the client code would spend most of its CPU time blocked while reading from disk and then blocked again while performing the network IO/waiting for a response. http://ayende.com/4476/unbounded-concurrency#comment14http://ayende.com/4476/unbounded-concurrency#comment14Thu, 29 Apr 2010 16:04:30 GMTRahien, thanks for the info. http://ayende.com/4476/unbounded-concurrency#comment13http://ayende.com/4476/unbounded-concurrency#comment13Thu, 29 Apr 2010 15:42:38 GMTCopenhas, Index updates in Couch are blocking, only one thread can do it at a single time. Couch concurrency really shines for reads, for writes, it is limited to a single thread for documents and one thread for each index (on first read) http://ayende.com/4476/unbounded-concurrency#comment12http://ayende.com/4476/unbounded-concurrency#comment12Thu, 29 Apr 2010 15:27:19 GMTDemis, The 1 request handling works nicely if you don't do IO. As you noted, if you do need it, you have a hard limit on what you can do. Even if you use async IO, it is pretty easy to get into situations where you can handle requests fast enough but you blow out the async IO queues. http://ayende.com/4476/unbounded-concurrency#comment11http://ayende.com/4476/unbounded-concurrency#comment11Thu, 29 Apr 2010 15:26:02 GMTYeah that would make sense that CouchDB has a single write thread. I'll have to take a look to see how CouchDB really takes advantage of Erlang. I suppose when updating indexes or on reads. Hmm..but is that a write thread in Erlang's sense or does CouchDB actually create and manage an OS level thread for writing... meh, this is getting off the point of the post. http://ayende.com/4476/unbounded-concurrency#comment10http://ayende.com/4476/unbounded-concurrency#comment10Thu, 29 Apr 2010 15:12:48 GMTCouchDB share the same number of threads that other high performance servers like Redis and Nginx has: 1. In this way, the way they achieve concurrency is just by being really really fast. For maximum perf and scalability this is the correct number to have, although with the advent of multi-core computing the meaning has now changed to: "The correct number of threads is, of course, equal to the number of cores on the box" -- The Moth [www.danielmoth.com/.../...arallel-Programming.aspx](http://www.danielmoth.com/Blog/Do-NOT-Explicitly-Use-Threads-For-Parallel-Programming.aspx) Achieving this in say Redis is easy just run a new instance on a different port. Although as @Anders Lybecker implies this just lets you achieve CPU efficiency, this does not solve the problem if the actual request itself is expensive. Maybe the request needs to write a lot of data to disk in which case IO now becomes your bottleneck and your new hard limit will be reached. Which is why I would suggest either partitioning / load balancing or to defer execution. http://ayende.com/4476/unbounded-concurrency#comment9http://ayende.com/4476/unbounded-concurrency#comment9Thu, 29 Apr 2010 14:49:37 GMTIt sounds like you should be using the I/O Thread Pool threading - check out [blogs.msdn.com/.../9990424.aspx](http://blogs.msdn.com/pfxteam/archive/2010/04/09/9990424.aspx) for a drop-in scheduler that will work with PFX. http://ayende.com/4476/unbounded-concurrency#comment8http://ayende.com/4476/unbounded-concurrency#comment8Thu, 29 Apr 2010 14:46:44 GMTCopenhas, Couch has exactly one write thread. The issue isn't with thread management, the issue is with any locks that you have to deal with. http://ayende.com/4476/unbounded-concurrency#comment7http://ayende.com/4476/unbounded-concurrency#comment7Thu, 29 Apr 2010 14:08:51 GMTI do wonder what the upper limit is for CouchDB with Erlang. Due to Erlang's concurrency model it can handle thousands of it's internal processes. Of course behind the scenes it is actually managing the queues, scheduling, and OS threads that make everything run. I remembered reading about some Erlang webserver that didn't die until 80,000 concurrent connections while Apache died around 4,000. Of course questions of that webserver's sophistication and performance at those levels certainly comes up, but I believe it still demonstrates the abilities of Erlang. Regardless I'm very excited about RavenDB and yes each request has a fair amount of overhead to it. It sounds like the problem is really just needing to find the maximum number of threads that can be handled concurrently. Even though Erlang is running thousands of "processes" it still limits the number of concurrent OS threads it's using. I would assume this is based on system specs when the runtime starts up. http://ayende.com/4476/unbounded-concurrency#comment6http://ayende.com/4476/unbounded-concurrency#comment6Thu, 29 Apr 2010 13:46:31 GMTYou optimize one place to find the new bottleneck! The Task Parallel Library tries to keep your CPU busy, at the expense of the overall performance. http://ayende.com/4476/unbounded-concurrency#comment5http://ayende.com/4476/unbounded-concurrency#comment5Thu, 29 Apr 2010 12:13:26 GMTNo need for a custom partitioner. Just set ParallelOption.MaxDegreeOfParallelism to some reasonable value. Parallel.ForEach(Directory.GetFiles("Docs","*.json"), new ParallelOptions { MaxDegreeOfParallelism = 10 }, file => ...); http://ayende.com/4476/unbounded-concurrency#comment4http://ayende.com/4476/unbounded-concurrency#comment4Thu, 29 Apr 2010 11:54:46 GMTHi, as far as I unterstand Parallel.For (or ForEach in this case), each single iteration could be executed in parallel to each other iteration. That's why you could end up with this huge chunk of blocking threads. Maybe using a Partitioner could help. In that case you would not support executing every iteration in parallel, but only batches of for e.g. 100. I believe .Net 4.0 has some support for creating partions in Parallel.ForEach ... look at [msdn.microsoft.com/en-us/library/dd560853.aspx](http://msdn.microsoft.com/en-us/library/dd560853.aspx)http://ayende.com/4476/unbounded-concurrency#comment3http://ayende.com/4476/unbounded-concurrency#comment3Thu, 29 Apr 2010 11:19:21 GMT@junior programmer This is not a problem that a node.js or an erlang solution is going to solve. The problem here is that the request itself is an expensive operation and as ayende has suggested there *will always be* a celling on the number of requests it can process at the same time. ASP.NET already has a 'request queue' behind the scenes which improves the situation but this has a limit as well and there is so many requests you can queue before it will to start throwing Timeout exceptions. The easiest solution to develop is to throttle the requests on the client and perform them in batches. However I believe the most ideal solution in the long term is to convert the requests into async operations (if not already done so) which will remove the time-coupling on the client. This will allow you to store the requests into a persistent message quene which will allow you to process the request *offline* and at a load that the server can handle efficiently. http://ayende.com/4476/unbounded-concurrency#comment2http://ayende.com/4476/unbounded-concurrency#comment2Thu, 29 Apr 2010 10:39:51 GMTperhaps erlang or node.js is more suited than the c# threading model for massive concurrency http://ayende.com/4476/unbounded-concurrency#comment1http://ayende.com/4476/unbounded-concurrency#comment1Thu, 29 Apr 2010 10:11:04 GMT