Scratching an itch: NMemcached
Last night I found myself up at 1 AM, feeling restless. I decided that I need a new project, something that is very simple and will allow me to channel some energy. I decided to pursue something that I have long been curious about, build a Memcached server in .Net. The Memcached server is a distributed cache server that is very popular in the Ruby, PHP & Perl worlds, with some following on both Java & .Net side.
It is written in C, and it is has some fairly interesting characteristics. Chief among them, it is a straightforward technical challenge. I spend about a day on this, but I have a working Memcached server, which can work with the existing clients.
Currently I support all the standard commands:
The only thing that I do not support is the stats command. The project has over a hundred unit tests and 34 integration tests and currently stand at 94% test coverage.
All of that said, what will probably interest most people is the performance comparison. I do hope that some people will stop to actually look at the design of the code, but here are the numbers, for reading / writing of 10,000 (small) values (over 10 iterations, using 20 connections):
Native Memcached finish in: 1709.6 ms
NMemcached completes in: 5768.5 ms
Based on profiling, I decided to make a tiny change to the part of the application that handles reading the commands from the user. Now it is reading them in continuous fashion, instead of one byte at a time. This brought the speed of the NMemcached version to 3144.5 ms, which is a huge benefit for such a small change.
Which put a project that was written in a day at just over 3 times slower than a heavily optimized piece of mature C code. Not bad, if I say so myself. I took a look with dot Trace, to see what was taking most of the time, and it looks like a significant amount goes in the networking section (specifically the LineReader class). There are a lot of tests, so it is not a problem to go and fix this if someone wants to.
You can access the code here: https://rhino-tools.svn.sourceforge.net/svnroot/rhino-tools/experiments/NMemcached