http://ayende.comAyende @ RahienCopyright (C) Ayende Rahien 2004 - 2021 (c) 202660Rob, I am going to respond to that post in a separate post http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment12http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment12Fri, 09 Apr 2010 07:14:02 GMTDemis, Raven's does that, yep. It provide two things, immediate background indexing and explicit notification about whatever the indexed results are stale or not. http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment11http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment11Fri, 09 Apr 2010 07:13:28 GMTErik, Yes and no. You would need some way to update the lucene index, and if you need to do multi key access, that is likely to be expensive. For things like that, I would rather use a doc db, yes. http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment10http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment10Fri, 09 Apr 2010 07:12:44 GMT@Erik yeah that would be golden, and it looks like Oren's heading down that path with his Divan Db. The problem is that if Lucene walks your data in a background task than you could potentially have stale data in the Search Index. The ideal solution would be something along the lines of Oren's solution where the Search Index is updated in real-time (or close to real-time i think he quoted 25ms) preferably using queue technology. http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment9http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment9Fri, 09 Apr 2010 01:42:54 GMTHmmm, if I put two and two together, I'd say that a Key-Value store combined with something like Lucene is a really good option for a lot of things. I'm probably wrong here, but as far as I understand, it works a bit like this right? - Lucene walks your data, generating indexes (by key), and allows very quick searches by using these indexes to find the key. - A key-value store uses keys, obviously. So to search for something, you use Lucene to find the keys, and fetch them from your KV store. So searching shouldn't be a problem like that, right? You're probably going to prove me wrong with the next one in the series about document databases showing a far easier way though ;-) http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment8http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment8Thu, 08 Apr 2010 23:21:17 GMT@Mark Rogers I think like any technology there is a risk to over use it. Devs might think hey here's this shiny new tech that's x times faster than our RDBMS and will solve all of our performance and scalability issues only to realize half-implemented way down the line that they have a suite of new problems and actually need that lovely query language in-built into their RDBMS. Personally I don't view NoSQL data stores as a replacement technology. Rather I believe they make a good complementary technology and are great when they have in-built support for your Use-Case. As Oren has pointed out most key value stores are a good fit when you want to 'just access a bucket of bits using a key'. Personally we've had great success at mflow.com and have provided a much better and more responsive user experience as a result of utilizing Redis whenever it made sense to. Most of our web services end up hitting optimized 'in-memory data views' which mirrors data that ultimately resides in our sharded RDBMS databases. If you're interested in checking out the user experience said technology and you live in the *UK*, you can access our closed-beta and as a bonus 'download any free track of your choice' with my limited invite codes attached to this url (we're kinda like social music service that's kinda like itunes+twitter+spotify): [http://www.mflow.com/download/index/mythz007](http://www.mflow.com/download/index/mythz007)http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment7http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment7Thu, 08 Apr 2010 16:08:50 GMT@Mark Rogers [Citation Needed] http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment6http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment6Thu, 08 Apr 2010 15:36:36 GMTWhen I read about how people mention that they actually used a non-RDMS in production, the story usually starts off hopeful and then descends into regret and despair. http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment5http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment5Thu, 08 Apr 2010 15:00:48 GMTI would say that only useful for accessing 'a bucket of bits using a key' is only true for primitive key value stores like Memcached. There are new NoSQL databases (like Redis and MongoDB) with a lot more advanced features opening up a new suite of use-cases: At mflow (apart from maintaining intelligent in-memory data views) we are using Redis to store rolling error logs (important for maintaining a fast, combined chronological ordered list of error logs for our load-balanced SOA web services) and as a fast in-memory Message Queue for persisting and load-balancing async web service requests. There are a number of other use-cases for where it is the best solution, like being the optimal data store for high-performance 'web streaming' real-time notification comet servers, that when used in conjunction with a technology like nodejs.org solves the 10k connection problem. A good example of advanced software that utilizes Redis as a backend that I just noticed on the twitter-sphere today is: [http://www.ohmstudio.com/](http://www.ohmstudio.com/) Who provide software allowing you to collaboratively compose music online. Their demo video is impressive. http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment4http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment4Thu, 08 Apr 2010 12:11:45 GMTAhh, awesome. I'm starting to look into MongoDB now. Looks very promising, although it requires a bit of a brainshift in the way I think about storage. http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment3http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment3Thu, 08 Apr 2010 11:01:03 GMTHenning, Yes, that is possible, but the fun part about most document dbs is that they are already key/value stores :-) http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment2http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment2Thu, 08 Apr 2010 09:26:10 GMTInteresting ideas. Could I use a combination of Key / Value store and for example document db, where you store user session data, shopping cart etc in the K/V store, and your "domain model" in a document db? http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment1http://ayende.com/4456/that-no-sql-thing-key-value-stores-usages#comment1Thu, 08 Apr 2010 09:24:55 GMT