But for me there was a moment of pure joy one morning when an absolutely awesome colleague I worked with at the time said to me something like: "There's a problem with this invoice, I traced it down to this table in the database which has errors in these columns. I've written a SQL statement to correct, or should it be done at the model level?". Not only had he found and analyzed the problem, he was offering to fix it.
Praise the gods. To do similar in Plone, he would have had to learn Python, read up on all the classes. Write a script to get those objects from the ZODB and examine them. Not a small undertaking by any means.
What was going on
The tools for the ZODB just weren't there and it wasn't transparent enough. If there was a good object browser for the ZODB (and yes a few simple projects showed up that tried to do that) that did all the work and exposed things that would really help. But setting up and configuring such a tool is hard and I never saw one that allowed you to do large scale changes.
I also got the following comment, on an unrelated post:
Not directly related, but I'm curious why you don't use Rhino Queues? The tooling with msmq?
Tooling are incredibly important! In fact, it is the tooling that can make or break a system. Rhino Queues is a really nice queuing system, it offer several benefits over MSMQ, but it has no tooling support, and as such, it has a huge disadvantage in comparison to MSMQ (or other Queuing technologies).
With databases, this is even more important. I can (usually) live without having direct access to queued messages, but for databases and data stores, having the ability to access the data in an easy manner is mandatory. Some data is unimportant (I couldn’t care less what the values are in user #12094’s session are), but for the most part, you really want the ability to read that data and look at it in a human readable fashion.
The problems that Andy run into with ZODB are related more to the fact that ZODB didn’t have any good tooling and that the ZODB storage format was tied directly to Python’s pickle.
With Raven, I flat out refuse to release it publically until we have a good tooling story. Raven comes with its own internal UI (accessible via any web browser), which allows you to define indexes, view/create/edit the documents, browse the database, etc.
I consider such abilities crucial, because without those, a database is basically a black hole, which requires special tooling to work with. By providing such tooling out of the box, you reduce the barrier to entry by a huge margin.
This image was sent to me by a lot of people:
This is funny, and true. There is another layer here, you don’t query a key/value store, that is like asking how to feed a car, because you are used to riding horses. If you need to perform queries on a key/value store, you are using the wrong tool, or perhaps you are trying to solve the problem in a non idiomatic way.
More posts in "re" series:
- (06 Sep 2018) Summary
- (05 Sep 2018) When the data hits the disk
- (04 Sep 2018) Reading data from disk
- (03 Sep 2018) The hash structure
- (31 Aug 2018) Working with the file system
- (30 Aug 2018) Digging into the C++ impl
- (29 Aug 2018) Let’s check these numbers
- (28 Aug 2018) Let’s start with managed code
- (27 Aug 2018) Reading the paper