Ayende @ Rahien

Hi!
My name is Ayende Rahien
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

@

Posts: 5,947 | Comments: 44,539

filter by tags archive

The smallest bugs, the biggest problems – Part II


Originally posted at 11/22/2010

In a previous post, I talked about how I found the following (really nasty) bug in RavenDB’s managed storage (which is still considered unstable, btw):

When deleting documents in a database that contains more than 2 documents, and  the document(s) deleted are deleted in a certain order, RavenDB would go into 100% CPU. The server would still function, but it would always think that it had work to do, even if it didn’t have any.

Now, I want to talk about the actual bug.

image

What I did wrong here is to reuse the removed and value parameters in the second call to TryRemove. That call is internal, and is only needed to properly balance the tree, but what it ended up doing is always return the removed/value from the right side of the tree.

Compounding the problem is that I only actually used the TryRemove value in a single location, and even then, it is a mistake. Take a look:

image

That meant that I actually looked for the problem in the secondary indexes for a while, before realizing that the actual problem was elsewhere.


Comments

Rob Kent

When I worked in the mainframe world, one of the support guys, whose name was Ken Russell, came up with what we called 'Russell's Law':

'The length of time it takes to find a bug is inversely proportional to the size of the fix'.

A misplaced byte could take days to find and minute to fix, whereas an obvious bug required a page of code to patch.

Ade
Ade

I assume it's nothing more than a typo... but where does theValue come from?

J
J

Ade - A private field?

Jeronimo

out arguments shit.. That's why i prefer to create a special class to return more values.

Wesner Moise

Since this is an functional algorithm, you should use a different strategy for detecting actual removal.

1) Check before and after counts

or

2) Compare to see if the new tree is identical to the previous one after the operation

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

  1. RavenDB Sharding - Adding a new shard to an existing cluster, splitting the shard - 30 minutes from now

There are posts all the way to May 22, 2015

RECENT SERIES

  1. RavenDB Sharding (2):
    21 May 2015 - Adding a new shard to an existing cluster, the easy way
  2. The RavenDB Comic Strip (2):
    20 May 2015 - Part II – a team in trouble!
  3. Challenge (45):
    28 Apr 2015 - What is the meaning of this change?
  4. Interview question (2):
    30 Mar 2015 - fix the index
  5. Excerpts from the RavenDB Performance team report (20):
    20 Feb 2015 - Optimizing Compare – The circle of life (a post-mortem)
View all series

RECENT COMMENTS

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats