reNoSQL, meh

time to read 4 min | 604 words

I was pointed to this blog post, it is written by the guy who wrote ZODB (a python object database) who isn’t excited about NoSQL:

But for me there was a moment of pure joy one morning when an absolutely awesome colleague I worked with at the time said to me something like: "There's a problem with this invoice, I traced it down to this table in the database which has errors in these columns. I've written a SQL statement to correct, or should it be done at the model level?". Not only had he found and analyzed the problem, he was offering to fix it.

Praise the gods. To do similar in Plone, he would have had to learn Python, read up on all the classes. Write a script to get those objects from the ZODB and examine them. Not a small undertaking by any means.

What was going on

The tools for the ZODB just weren't there and it wasn't transparent enough. If there was a good object browser for the ZODB (and yes a few simple projects showed up that tried to do that) that did all the work and exposed things that would really help. But setting up and configuring such a tool is hard and I never saw one that allowed you to do large scale changes.

I also got the following comment, on an unrelated post:

Not directly related, but I'm curious why you don't use Rhino Queues? The tooling with msmq?

Tooling are incredibly important! In fact, it is the tooling that can make or break a system. Rhino Queues is a really nice queuing system, it offer several benefits over MSMQ, but it has no tooling support, and as such, it has a huge disadvantage in comparison to MSMQ (or other Queuing technologies).

With databases, this is even more important. I can (usually) live without having direct access to queued messages, but for databases and data stores, having the ability to access the data in an easy manner is mandatory. Some data is unimportant (I couldn’t care less what the values are in user #12094’s session are), but for the most part, you really want the ability to read that data and look at it in a human readable fashion.

The problems that Andy run into with ZODB are related more to the fact that ZODB didn’t have any good tooling and that the ZODB storage format was tied directly to Python’s pickle.

With Raven, I flat out refuse to release it publically until we have a good tooling story. Raven comes with its own internal UI (accessible via any web browser), which allows you to define indexes, view/create/edit the documents, browse the database, etc.

I consider such abilities crucial, because without those, a database is basically a black hole, which requires special tooling to work with. By providing such tooling out of the box, you reduce the barrier to entry by a huge margin.

This image was sent to me by a lot of people:

This is funny, and true. There is another layer here, you don’t query a key/value store, that is like asking how to feed a car, because you are used to riding horses. If you need to perform queries on a key/value store, you are using the wrong tool, or perhaps you are trying to solve the problem in a non idiomatic way.

More posts in "re" series:

  1. (23 Jun 2021) The performance regression odyssey
  2. (27 Oct 2020) Investigating query performance issue in RavenDB
  3. (27 Dec 2019) Writing a very fast cache service with millions of entries
  4. (26 Dec 2019) Why databases use ordered indexes but programming uses hash tables
  5. (12 Nov 2019) Document-Level Optimistic Concurrency in MongoDB
  6. (25 Oct 2019) RavenDB. Two years of pain and joy
  7. (19 Aug 2019) The Order of the JSON, AKA–irresponsible assumptions and blind spots
  8. (10 Oct 2017) Entity Framework Core performance tuning–Part III
  9. (09 Oct 2017) Different I/O Access Methods for Linux
  10. (06 Oct 2017) Entity Framework Core performance tuning–Part II
  11. (04 Oct 2017) Entity Framework Core performance tuning–part I
  12. (26 Apr 2017) Writing a Time Series Database from Scratch
  13. (28 Jul 2016) Why Uber Engineering Switched from Postgres to MySQL
  14. (15 Jun 2016) Why you can't be a good .NET developer
  15. (12 Nov 2013) Why You Should Never Use MongoDB
  16. (21 Aug 2013) How memory mapped files, filesystems and cloud storage works
  17. (15 Apr 2012) Kiip’s MongoDB’s experience
  18. (18 Oct 2010) Diverse.NET
  19. (10 Apr 2010) NoSQL, meh
  20. (30 Sep 2009) Are you smart enough to do without TDD
  21. (17 Aug 2008) MVC Storefront Part 19
  22. (24 Mar 2008) How to create fully encapsulated Domain Models
  23. (21 Feb 2008) Versioning Issues With Abstract Base Classes and Interfaces
  24. (18 Aug 2007) Saving to Blob
  25. (27 Jul 2007) SSIS - 15 Faults Rebuttal
  26. (29 May 2007) The OR/M Smackdown
  27. (06 Mar 2007) IoC and Average Programmers
  28. (19 Sep 2005) DLinq Mapping