Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,128 | Comments: 45,549

filter by tags archive

NH ProfGetting big, and bigger

time to read 2 min | 201 words

Well, at least now I have proof positive that all the architecture work that we did in order to ensure that NH Prof will scale better actually works. The following screenshot shows an example that used to crash NH Prof, it loading an 80 MB file with over 200,000 individual events to scan and process.


The fun part is that this is run through the entire pipeline, but without involving the UI, so we get the ability to process all those events without having to wait for the UI. If you remember, we had to invest a lot of time making sure that this sort of scenario will work properly.

Now, it takes 85 seconds for this to load, which I consider pretty reasonable, it means about 2,350 events per second, which isn’t a high number, but is plenty fast enough overall, considering the amount of work that I have to do in order to do things like calculate alerts, format the SQL, and do all the other things that NH Prof has to do.

More posts in "NH Prof" series:

  1. (09 Dec 2010) Alert on bad ‘like’ query
  2. (10 Dec 2009) Filter static files
  3. (16 Nov 2009) Exporting Reports
  4. (08 Oct 2009) NHibernate Search Integration
  5. (19 Aug 2009) Multiple Session Factory Support
  6. (07 Aug 2009) Diffing Sessions
  7. (06 Aug 2009) Capturing DDL
  8. (05 Aug 2009) Detect Cross Thread Session Usage
  9. (22 May 2009) Detecting 2nd cache collection loads
  10. (15 May 2009) Error Detection
  11. (12 May 2009) Queries by Url
  12. (04 Feb 2009) View Query Results
  13. (18 Jan 2009) Superfluous <many-to-one> update
  14. (18 Jan 2009) URL tracking
  15. (10 Jan 2009) Detecting distributed transactions (System.Transactions)
  16. (06 Jan 2009) The Query Cache
  17. (05 Jan 2009) Query Duration
  18. (24 Dec 2008) Unbounded result sets
  19. (24 Dec 2008) Row Counts



Over 2000 events per second? Impressive, is your queue running on Rhino Queues or sth else? (I'm asking because some time ago I had problems with RQ performance, it was nowhere near 2000 messages/second)

Ayende Rahien


No, it is an in memory queue.

And it is actually about the processing time & IO than anything related to the queue.

In memory queue, naively implemented, can easily do millions per second.

I intent to sit on the RQ perf issue when I have free time, mind you.

Thomas Krause

How well does the performance scale with the number of cpu cores?

Do you use multi threading at all for processing the events and if yes how do you partition the work between multiple threads?

Ayende Rahien


No, processing events is done in a single background thread, that significantly simplify what I need to do to bring it all into a single cohesive model.

Fabio Maulo

The "format the SQL" is a responsibility of which tear of NHProf ?

Ayende Rahien


This is done on the backend portion of NH Prof

Fabio Maulo

If you think that the SQL formatting is a possible point to improve performances perhaps you should move it up, if the SQL formatting is really only to show a fashion SQL to the user.

Ayende Rahien

Unfortunately, I actually do need to process all / most of them.

The SQL Formatter part is also responsible for creating the nice listing in the headers, which is almost always needed anyway

Comment preview

Comments have been closed on this topic.


  1. The worker pattern - 2 days from now

There are posts all the way to May 30, 2016


  1. The design of RavenDB 4.0 (14):
    26 May 2016 - The client side
  2. RavenDB 3.5 whirl wind tour (14):
    25 May 2016 - Got anything to declare, ya smuggler?
  3. Tasks for the new comer (2):
    15 Apr 2016 - Quartz.NET with RavenDB
  4. Code through the looking glass (5):
    18 Mar 2016 - And a linear search to rule them
  5. Find the bug (8):
    29 Feb 2016 - When you can't rely on your own identity
View all series


Main feed Feed Stats
Comments feed   Comments Feed Stats