Ayende @ Rahien

It's a girl

The smallest bugs, the biggest problems – Part II

Originally posted at 11/22/2010

In a previous post, I talked about how I found the following (really nasty) bug in RavenDB’s managed storage (which is still considered unstable, btw):

When deleting documents in a database that contains more than 2 documents, and  the document(s) deleted are deleted in a certain order, RavenDB would go into 100% CPU. The server would still function, but it would always think that it had work to do, even if it didn’t have any.

Now, I want to talk about the actual bug.

image

What I did wrong here is to reuse the removed and value parameters in the second call to TryRemove. That call is internal, and is only needed to properly balance the tree, but what it ended up doing is always return the removed/value from the right side of the tree.

Compounding the problem is that I only actually used the TryRemove value in a single location, and even then, it is a mistake. Take a look:

image

That meant that I actually looked for the problem in the secondary indexes for a while, before realizing that the actual problem was elsewhere.

Tags:

Posted By: Ayende Rahien

Published at

Originally posted at

Comments

Rob Kent
11/26/2010 02:04 PM by
Rob Kent

When I worked in the mainframe world, one of the support guys, whose name was Ken Russell, came up with what we called 'Russell's Law':

'The length of time it takes to find a bug is inversely proportional to the size of the fix'.

A misplaced byte could take days to find and minute to fix, whereas an obvious bug required a page of code to patch.

Ade
11/27/2010 01:34 AM by
Ade

I assume it's nothing more than a typo... but where does theValue come from?

J
11/27/2010 11:56 AM by
J

Ade - A private field?

Jeronimo
11/30/2010 06:33 AM by
Jeronimo

out arguments shit.. That's why i prefer to create a special class to return more values.

Wesner Moise
12/01/2010 10:04 AM by
Wesner Moise

Since this is an functional algorithm, you should use a different strategy for detecting actual removal.

1) Check before and after counts

or

2) Compare to see if the new tree is identical to the previous one after the operation

Comments have been closed on this topic.