Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,128 | Comments: 45,549

filter by tags archive

ChallengeFind the bug

time to read 1 min | 54 words

This piece of code caused a crashing bug on a QA server ( and we are lucky it didn’t go to production ).


Can you spot the bug?

I will give you a hint, it is the code that isn’t there.

More posts in "Challenge" series:

  1. (28 Apr 2015) What is the meaning of this change?
  2. (26 Sep 2013) Spot the bug
  3. (27 May 2013) The problem of locking down tasks…
  4. (17 Oct 2011) Minimum number of round trips
  5. (23 Aug 2011) Recent Comments with Future Posts
  6. (02 Aug 2011) Modifying execution approaches
  7. (29 Apr 2011) Stop the leaks
  8. (23 Dec 2010) This code should never hit production
  9. (17 Dec 2010) Your own ThreadLocal
  10. (03 Dec 2010) Querying relative information with RavenDB
  11. (29 Jun 2010) Find the bug
  12. (23 Jun 2010) Dynamically dynamic
  13. (28 Apr 2010) What killed the application?
  14. (19 Mar 2010) What does this code do?
  15. (04 Mar 2010) Robust enumeration over external code
  16. (16 Feb 2010) Premature optimization, and all of that…
  17. (12 Feb 2010) Efficient querying
  18. (10 Feb 2010) Find the resource leak
  19. (21 Oct 2009) Can you spot the bug?
  20. (18 Oct 2009) Why is this wrong?
  21. (17 Oct 2009) Write the check in comment
  22. (15 Sep 2009) NH Prof Exporting Reports
  23. (02 Sep 2009) The lazy loaded inheritance many to one association OR/M conundrum
  24. (01 Sep 2009) Why isn’t select broken?
  25. (06 Aug 2009) Find the bug fixes
  26. (26 May 2009) Find the bug
  27. (14 May 2009) multi threaded test failure
  28. (11 May 2009) The regex that doesn’t match
  29. (24 Mar 2009) probability based selection
  30. (13 Mar 2009) C# Rewriting
  31. (18 Feb 2009) write a self extracting program
  32. (04 Sep 2008) Don't stop with the first DSL abstraction
  33. (02 Aug 2008) What is the problem?
  34. (28 Jul 2008) What does this code do?
  35. (26 Jul 2008) Find the bug fix
  36. (05 Jul 2008) Find the deadlock
  37. (03 Jul 2008) Find the bug
  38. (02 Jul 2008) What is wrong with this code
  39. (05 Jun 2008) why did the tests fail?
  40. (27 May 2008) Striving for better syntax
  41. (13 Apr 2008) calling generics without the generic type
  42. (12 Apr 2008) The directory tree
  43. (24 Mar 2008) Find the version
  44. (21 Jan 2008) Strongly typing weakly typed code
  45. (28 Jun 2007) Windsor Null Object Dependency Facility



Is a transaction or some other synchronization method missing here? That is, it seeks the values, then deletes them. The data could change during the deletions.


It's in the loop... ?


I think, you're missing a check before deleting it ...


Where does 'data' comes from? (The ApplyToKeyAndActiveVersions line).


Ayende, where's your sweet spot in the Esent database? Do you miss good ol' days of WINAPI programming, or is Esent exceptionally performant/stable/powerful/whatever? Maybe there was a reason Microsoft has hidden it?

Ayende Rahien

No, that is handled elsewhere

Ayende Rahien


Another table that we join against.

Ayende Rahien


You are close...


No, that is not it.


I use it when I need a local DB. It is really easy to use when you get how it is working.

It helps that it is the only local DB with true threading support.


maybe issue is related with closure of the session and data variables here: v=>Api.JetDelete(session, data)

Ayende Rahien


No, that is not it


Maybe "v => Api.JetDelete(session, data)" can't be run thru delegate for some reason?

Ayende Rahien


No, that is not it. It works just fine.


I take it the crash is a stack overflow from an infinite loop and that you are either missing a breaking condition or not doing something that would should effect the 'position'..

Basically no idea, is this something that will probably only make sense when illustrated and explained?


What happens to the table cursor after Api.JetDelete()?

And does Api.JetDelete() immediately remove the table row or only mark it as deleted?

Otherwise scenario could be:

  • Api.JetDelete moves cursor forward

  • Api.TryMovePrevious moves cursor backwards (to same row)

=> infinite loop


Stephen, That would hang, not crash...


Ah, true ;)

(btw ayende, having problems submitting comments from ie7/8- not sure if its just me, hitting submit doesn't do anything it just sits there).

Ayende Rahien


No stack overflow, no, and not an infinite loop, but you should concentrate on the loop

Ayende Rahien


It is marking that for deletion in the end of the current transaction, TryMovePrevious is not the issue.

The loop will terminate at the expected time.


I guess that you are trying to clean expired values (stuff we don't care about anymore).. thus having a big result set would pose a problem... am I right?


do we need to know the contract of the jet api to find the problem?


If the Api.RetrieveColumnAsString gets the first one then the Api.TryMovePrevious would never go to the rest but stop when the first one is deleted.

Ayende Rahien


Tada, you are very close.


No, you don't.

Ayende Rahien


No, that is not the behavior we have


How often do values expire?

Michael Morton

You're calling the function on a fixed timer and it's being executed again, before the previous execution has finished. Considering it results in a crash, it's probably stacking a good number of times.

Ayende Rahien

That is a user defined value. In the system that we are talking abut, 24 hours

Ayende Rahien



What timer? There isn't any external code involved that affect the bug in this function

Sander Rijken

The call to Api.JetDelete should probably be called with session and key, not session and keyS (deleting everything)?

Ayende Rahien


keys is the variable holding the table name, not a collection. It is instruction to delete the current row.

Ayende Rahien


Notice what I said, what is the code that IS NOT THERE?!

Neal Blomfield

You get the version but do not check it has expired?

TBH I am guessing based on what I would expect to see rather than understanding the code / api that this is using but it seems like the most logical thing.

Ayende Rahien


No, that is not that.

And you don't need to understand anything about the API to see the problem.

The problem is that until you see the problem, you don't know what it is.

I am actually encouraged that no one managed to find it.

It means that I am not that stupid


Commit size will become huge in this case... Since you say that it is about code that isn't there, all I am able to come with is fixing a maximum of expired values to be processed in the loop and process the remaining later. That's similar to ORA-01555 snapshot too old!

Ayende Rahien


TADA! You got it!

When you hit it exactly right, you may get a LOT of expired values.

Those expired values can be big enough to hit the commit limit, and cause an error!

So I was stupid, I thought so! :-)


Wow... gloubidou I am impress.

Oren, how did you uncover the bug? If the application throw an exception somewhere then it's probably not that hard to trace down the culprit. Did you catch it through stress testing?


Silly question: why is there a commit limit?

Ayende Rahien

As I said, we run into this error in QA.

Once I had the stack trace, and the error, it was really obvious what was wrong

Comment preview

Comments have been closed on this topic.


  1. The worker pattern - 3 days from now

There are posts all the way to May 30, 2016


  1. The design of RavenDB 4.0 (14):
    26 May 2016 - The client side
  2. RavenDB 3.5 whirl wind tour (14):
    25 May 2016 - Got anything to declare, ya smuggler?
  3. Tasks for the new comer (2):
    15 Apr 2016 - Quartz.NET with RavenDB
  4. Code through the looking glass (5):
    18 Mar 2016 - And a linear search to rule them
  5. Find the bug (8):
    29 Feb 2016 - When you can't rely on your own identity
View all series


Main feed Feed Stats
Comments feed   Comments Feed Stats