I started a project like that a while ago, but eventually I decided that it would be much easier to just use Memcached. Some of the tricks that Memcached is using is explicit memory management of the layout of the cache. I assume that the GC will take care of much of that for us.
Sriram has managed to get to 16,000 requests / second with his current code. Which is certainly impressive.
I find it very interesting because I am much more comfortable with CLR based tools. I know their failure scenarios and how to deal with them. Native tools... not so much.
I have used Memcached for production, and it works. But assuming that Cacheman would be on par with the Memcached implementation, I believe that I will choose it.
I am reading the code now (well, reflecting over it) and it is interesting read.