Ayende @ Rahien

Hi!
My name is Ayende Rahien
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

@

Posts: 5,947 | Comments: 44,544

filter by tags archive

The “features” that no one talks about makes all the difference


You might have noticed that I am talking a lot about support and operations recently.  This is because we have been doing a lot of work around that area. Making sure that RavenDB is even more forthcoming and open about what is going on.

This week it has been about making sure that the shutdown phase of RavenDB is as transparent as it could be. Debugging those sort of issues is a PITA, because you very rarely really stop to consider them. But we got some feedback from customers about a common set of issues there.

Under some circumstances, shutting down RavenDB might take a while. We identified several things that can cause this, mostly indexing in progress.

The first thing we did is change the way we are doing indexing to allow aborting them during shutdown, but there are still a set of operations that might take a while that we have to complete even in shutdown scenario.

For example, we might be just in the middle of flushing to disk, and we really want that to complete successfully (otherwise we would need to run an expensive check on startup).

Therefor, we added this:

image

You’ll still have to wait, sure. But now if you watch the logs you can see why, and have some understanding about how long this is going to take.


Comments

Yves Reynhout

You mean like the same expensive check at startup when the process crashes or got killed?

Yves Reynhout

Well, you've shown us the graceful shutdown process. What happens - when the process dies (for whatever reason) - to those things you "really" want to complete successfully? I assume you recover from such things upon restart by checking integrity, no? Granted, it's a bit off topic.

Ayende Rahien

Yves, Oh, sure, we had that for a long while.

Yves Reynhout

I guess my question - albeit a rhetoric one - was, beyond the fact you're being proactive in communicating why shut down is taking so long, the check happens at start-up, regardless (although I assume it to be fairly cheap if all is well) of a successful or faulty shutdown.

Ayende Rahien

Yves, It is cheap to tell if we had a good shutdown, expensive if we had a bad one.

Gene Hughson

Paying attention to operations and support requirements is the software equivalent of the old saying that amateurs study tactics and professionals study logistics.

Karep

what's the reason for localReason variable?

Ayende Rahien

Karep, The value of the original string might change during this method.

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

No future posts left, oh my!

RECENT SERIES

  1. RavenDB Sharding (3):
    22 May 2015 - Adding a new shard to an existing cluster, splitting the shard
  2. The RavenDB Comic Strip (2):
    20 May 2015 - Part II – a team in trouble!
  3. Challenge (45):
    28 Apr 2015 - What is the meaning of this change?
  4. Interview question (2):
    30 Mar 2015 - fix the index
  5. Excerpts from the RavenDB Performance team report (20):
    20 Feb 2015 - Optimizing Compare – The circle of life (a post-mortem)
View all series

RECENT COMMENTS

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats