reNoSQL, meh

time to read 4 min | 604 words

I was pointed to this blog post, it is written by the guy who wrote ZODB (a python object database) who isn’t excited about NoSQL:

But for me there was a moment of pure joy one morning when an absolutely awesome colleague I worked with at the time said to me something like: "There's a problem with this invoice, I traced it down to this table in the database which has errors in these columns. I've written a SQL statement to correct, or should it be done at the model level?". Not only had he found and analyzed the problem, he was offering to fix it.

Praise the gods. To do similar in Plone, he would have had to learn Python, read up on all the classes. Write a script to get those objects from the ZODB and examine them. Not a small undertaking by any means.

What was going on

The tools for the ZODB just weren't there and it wasn't transparent enough. If there was a good object browser for the ZODB (and yes a few simple projects showed up that tried to do that) that did all the work and exposed things that would really help. But setting up and configuring such a tool is hard and I never saw one that allowed you to do large scale changes.

I also got the following comment, on an unrelated post:

Not directly related, but I'm curious why you don't use Rhino Queues? The tooling with msmq?

Tooling are incredibly important! In fact, it is the tooling that can make or break a system. Rhino Queues is a really nice queuing system, it offer several benefits over MSMQ, but it has no tooling support, and as such, it has a huge disadvantage in comparison to MSMQ (or other Queuing technologies).

With databases, this is even more important. I can (usually) live without having direct access to queued messages, but for databases and data stores, having the ability to access the data in an easy manner is mandatory. Some data is unimportant (I couldn’t care less what the values are in user #12094’s session are), but for the most part, you really want the ability to read that data and look at it in a human readable fashion.

The problems that Andy run into with ZODB are related more to the fact that ZODB didn’t have any good tooling and that the ZODB storage format was tied directly to Python’s pickle.

With Raven, I flat out refuse to release it publically until we have a good tooling story. Raven comes with its own internal UI (accessible via any web browser), which allows you to define indexes, view/create/edit the documents, browse the database, etc.

I consider such abilities crucial, because without those, a database is basically a black hole, which requires special tooling to work with. By providing such tooling out of the box, you reduce the barrier to entry by a huge margin.

This image was sent to me by a lot of people:

This is funny, and true. There is another layer here, you don’t query a key/value store, that is like asking how to feed a car, because you are used to riding horses. If you need to perform queries on a key/value store, you are using the wrong tool, or perhaps you are trying to solve the problem in a non idiomatic way.

More posts in "re" series:

  1. (26 Apr 2017) Writing a Time Series Database from Scratch
  2. (28 Jul 2016) Why Uber Engineering Switched from Postgres to MySQL
  3. (15 Jun 2016) Why you can't be a good .NET developer
  4. (12 Nov 2013) Why You Should Never Use MongoDB
  5. (21 Aug 2013) How memory mapped files, filesystems and cloud storage works
  6. (15 Apr 2012) Kiip’s MongoDB’s experience
  7. (18 Oct 2010) Diverse.NET
  8. (10 Apr 2010) NoSQL, meh
  9. (30 Sep 2009) Are you smart enough to do without TDD
  10. (17 Aug 2008) MVC Storefront Part 19
  11. (24 Mar 2008) How to create fully encapsulated Domain Models
  12. (21 Feb 2008) Versioning Issues With Abstract Base Classes and Interfaces
  13. (18 Aug 2007) Saving to Blob
  14. (27 Jul 2007) SSIS - 15 Faults Rebuttal
  15. (29 May 2007) The OR/M Smackdown
  16. (06 Mar 2007) IoC and Average Programmers
  17. (19 Sep 2005) DLinq Mapping