Yesterday the perf team let me know that they managed to get ~18% improvement on Voron by utilizing another on disk data structure. I’ll post more on that when we have it merged and working.
Talking about this, we decided to run a few benchmarks on our current status.
Nitpicker – this post talks about the performance of low level storage engines, the benchmark used was storing sequential values with 8 bytes key and 8 bytes value.
Here are the results for writing a billion values (16 bytes total) in ten thousands transactions.
- 1,000,000,000 values (1 billion)
- 9.13 minutes
- 1.824 million writes / sec
- 31 GB final size
Then we spiked a small optimization that should allow us to defer major I/O costs, and we got:
- 100,000,000 values (hundred million)
- 40 seconds
- 2.449 million writes / sec
- 4.1 GB final size
We pretty much cheat here, because we defer the I/O cost under load to a later time, by not purging the journals, but that is something that I’ll post in detail once we have this merged.
Oh, and those are the number before the additional 18% optimization . And they were run on a single commodity hardware node over SSD drive.