Ayende @ Rahien

Refunds available at head office

Voron perf testing, part II

After the previous round of perf testing, I decided to compare just Voron and Esent. The machine is the same as before, and I am testing this on a 300GB HDD. One of the reasons that I choose to use an HDD here is that it is drastically slower, so it shows problematic I/O patterns more easily.

I also decided that it would be interested to see exactly how it scale with regards to concurrency.

image

And here you can see the latencies involved in each 100 items transaction.

image

Now, to be fair to the poor thing, it is running on an old HDD. Here are the stats from running this. Notice the actual latencies that are involved:

image

But interestingly, we can see that the more concurrent writes we use, the slower we become. At a guess, I would assume that this is related to concurrent threads issuing different writes and causing a lot of seeks.

But after investigating a bit deeper, I think that the problem is actually different. Concurrent writes are inherently random. This happens because as we read from the sequential stream in multiple threads, and each thread writing them to Esent, what the db actually gets is a stream of non sequential ranges.

Those can be roughly in the same place, but they aren’t sequential, negating a lot of potential optimizations.

Tags:

Posted By: Ayende Rahien

Published at

Originally posted at

Comments

alex
12/16/2013 10:00 PM by
alex

Just out of curiosity: have you done any testing where the journal (and/or backing memmap files for scratch pages memory) are located on a different disk than the data file?

Comments have been closed on this topic.