Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,007 | Comments: 44,762

filter by tags archive

Things you never want to hear from a support person dealing with a production server crash

time to read 1 min | 84 words

I got all of the following during a recent support call to deal with a hard server crash:

  • I have never seen something like that before.
  • Our internal tools crash when they connect to your servers.
  • We are probably not going to recover this server.
  • You do have a backup, right?

Thankfully, it turned out to be something stupid that was fixed very quickly. But it sure was scary.



you do have a backup, right?

Alex Simkin

So, what was that stupid thing?


You left out the most interesting part! What was the problem?


I always start those conversations with, "No sweat, we'll just restore your backup." It's more fun to watch them puke sooner than later.

I figure if I waste 1/3 of my life doing backups everyone else can too.

Comment preview

Comments have been closed on this topic.


No future posts left, oh my!


  1. Speaking (3):
    23 Sep 2015 - Build Stuff 2015 (Lithuania & Ukraine), Nov 18 - 24
  2. Production postmortem (11):
    22 Sep 2015 - The case of the Unicode Poo
  3. Technical observations from my wife (2):
    15 Sep 2015 - Disk speeds
  4. Find the bug (5):
    11 Sep 2015 - The concurrent memory buster
  5. Buffer allocation strategies (3):
    09 Sep 2015 - Bad usage patterns
View all series



Main feed Feed Stats
Comments feed   Comments Feed Stats