Rule out the stupid stuff first

time to read 3 min | 567 words

A customer reported that they have problems downloading large (10+ MB) attachments from RavenDB over the network. The problem wasn’t present when running locally.

The first thing I did was go and check how we are loading and sending the data to the user. But it seems fine, we were doing buffered reads from disk and streaming the results to the network.

I then wrote the simplest thing that I could think of to reproduce this issue:

class Program
{
    static void Main(string[] args)
    {
        var listener = new HttpListener
            {
                Prefixes = {"http://+:8080/"}
            };
        listener.Start();
        while (true)
        {
            var context = listener.GetContext();
            var sp = Stopwatch.StartNew();
            var chunks = int.Parse(context.Request.QueryString["chunks"]);
            var _4kb = new byte[4*1024];
            for (int i = 0; i < chunks; i++)
            {
                context.Response.OutputStream.Write(_4kb, 0, _4kb.Length);
            }
            context.Response.Close();
            var totalSize = (double) (_4kb.Length*chunks);
            Console.WriteLine("{0:#,#.##;;0} mb in {1:#,#}", Math.Round((totalSize / 1024) / 1024, 2), sp.ElapsedMilliseconds);
        }
    }
}

This is a simple HTTP server, which will stream to you as much data as you request, as fast as it can. The nice thing about it is that it is a pure in memory device, no IO / waiting required. So it should be able to saturate the network easily.

Using this, we can determine whatever the problem is really with RavenDB or something else.

The customer came back with a report that on their network, a 2MB request was taking over a minute. Considering that we are talking about internal networks, that sounded like we have identified the problem. There is something fishy going on with the network. Now it is in the hands of the sys admin, I just hope he is not this guy.