Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,131 | Comments: 45,565

filter by tags archive

Rule out the stupid stuff first

time to read 3 min | 567 words

A customer reported that they have problems downloading large (10+ MB) attachments from RavenDB over the network. The problem wasn’t present when running locally.

The first thing I did was go and check how we are loading and sending the data to the user. But it seems fine, we were doing buffered reads from disk and streaming the results to the network.

I then wrote the simplest thing that I could think of to reproduce this issue:

class Program
    static void Main(string[] args)
        var listener = new HttpListener
                Prefixes = {"http://+:8080/"}
        while (true)
            var context = listener.GetContext();
            var sp = Stopwatch.StartNew();
            var chunks = int.Parse(context.Request.QueryString["chunks"]);
            var _4kb = new byte[4*1024];
            for (int i = 0; i < chunks; i++)
                context.Response.OutputStream.Write(_4kb, 0, _4kb.Length);
            var totalSize = (double) (_4kb.Length*chunks);
            Console.WriteLine("{0:#,#.##;;0} mb in {1:#,#}", Math.Round((totalSize / 1024) / 1024, 2), sp.ElapsedMilliseconds);

This is a simple HTTP server, which will stream to you as much data as you request, as fast as it can. The nice thing about it is that it is a pure in memory device, no IO / waiting required. So it should be able to saturate the network easily.

Using this, we can determine whatever the problem is really with RavenDB or something else.

The customer came back with a report that on their network, a 2MB request was taking over a minute. Considering that we are talking about internal networks, that sounded like we have identified the problem. There is something fishy going on with the network. Now it is in the hands of the sys admin, I just hope he is not this guy.


Paul Hatcher

I had something similar on my home network - took ages to track down and turned out to be a hub that was on its way out.

Performance was fine with 1-2 ports occupied but degraded rapidly as more devices were added.

Jim Baltika

I think the idea it self to store and server binary files in Raven DB for any enterprise level application is bad from beginning. We are using RavenDb with amazon S3 without any problems. Amazon S3 provides access security plus ability to access file with specific TTL (time to live). The best of all worlds :)

Gene Hughson

These kinds of things are why I've always tried to remain a generalist at heart. Focusing intently on one single aspect makes it extremely likely that you'll run down blind alleys, not because they're probable but because they're what you understand.

Johannes Gustafsson

Was it really that simple? I admit, I actually felt quite naive to complain about it in the first place. The thing was that it was reproducible. I had 2 completely different environments, at different locations, set up by different people on different hardware, where I got the same behaviour. And it actually turned out (thanks to your sample) that there probably is some problem in HttpListener (or http.sys) when using 4k buffers.

Here is the original conversation: https://groups.google.com/d/topic/ravendb/EnI_2tjAwLg/discussion

Btw, the fix you made in the latest build made the problem go away, so thanks :-)

PS. A few people on the mailing list have complained about very slow download speeds for the silverlight UI. Since the code in Raven uses 4k buffers for this, my money is on that they have the same problem as I have. DS

Ayende Rahien

Johannes, It wasn't so simple. But wait for the next post in which I am talking about that.

Comment preview

Comments have been closed on this topic.


  1. RavenDB Conference 2016–Slides - 15 hours from now
  2. Proposed solution to the low level interview question - about one day from now

There are posts all the way to Jun 02, 2016


  1. The design of RavenDB 4.0 (14):
    26 May 2016 - The client side
  2. RavenDB 3.5 whirl wind tour (14):
    25 May 2016 - Got anything to declare, ya smuggler?
  3. Tasks for the new comer (2):
    15 Apr 2016 - Quartz.NET with RavenDB
  4. Code through the looking glass (5):
    18 Mar 2016 - And a linear search to rule them
  5. Find the bug (8):
    29 Feb 2016 - When you can't rely on your own identity
View all series



Main feed Feed Stats
Comments feed   Comments Feed Stats