Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 18 | Comments: 73

filter by tags archive

The smallest bugs, the biggest problems – Part II

time to read 2 min | 237 words

Originally posted at 11/22/2010

In a previous post, I talked about how I found the following (really nasty) bug in RavenDB’s managed storage (which is still considered unstable, btw):

When deleting documents in a database that contains more than 2 documents, and  the document(s) deleted are deleted in a certain order, RavenDB would go into 100% CPU. The server would still function, but it would always think that it had work to do, even if it didn’t have any.

Now, I want to talk about the actual bug.


What I did wrong here is to reuse the removed and value parameters in the second call to TryRemove. That call is internal, and is only needed to properly balance the tree, but what it ended up doing is always return the removed/value from the right side of the tree.

Compounding the problem is that I only actually used the TryRemove value in a single location, and even then, it is a mistake. Take a look:


That meant that I actually looked for the problem in the secondary indexes for a while, before realizing that the actual problem was elsewhere.


Rob Kent

When I worked in the mainframe world, one of the support guys, whose name was Ken Russell, came up with what we called 'Russell's Law':

'The length of time it takes to find a bug is inversely proportional to the size of the fix'.

A misplaced byte could take days to find and minute to fix, whereas an obvious bug required a page of code to patch.


I assume it's nothing more than a typo... but where does theValue come from?


Ade - A private field?


out arguments shit.. That's why i prefer to create a special class to return more values.

Wesner Moise

Since this is an functional algorithm, you should use a different strategy for detecting actual removal.

1) Check before and after counts


2) Compare to see if the new tree is identical to the previous one after the operation

Comment preview

Comments have been closed on this topic.


  1. Production postmortem: The industry at large - one day from now
  2. The insidious cost of allocations - about one day from now
  3. Buffer allocation strategies: A possible solution - 5 days from now
  4. Buffer allocation strategies: Explaining the solution - 6 days from now
  5. Buffer allocation strategies: Bad usage patterns - 7 days from now

And 2 more posts are pending...

There are posts all the way to Sep 11, 2015


  1. Find the bug (5):
    20 Apr 2011 - Why do I get a Null Reference Exception?
  2. Production postmortem (10):
    01 Sep 2015 - The case of the lying configuration file
  3. What is new in RavenDB 3.5 (7):
    12 Aug 2015 - Monitoring support
  4. Career planning (6):
    24 Jul 2015 - The immortal choices aren't
View all series



Main feed Feed Stats
Comments feed   Comments Feed Stats