One of the things that I wanted to do with RDB is to create an explicit actor model inside the codebase. I have been using a similar structure inside NH Prof, and it has been quite successful. The design goals for RDB is:

Assumptions for the database cosntruction

Get / Put / Delete semantics for Json documents.

All those operations can access batches of documents to work on. Those operations fully implement ACID. Which means that if you got a successful response for a document Put, you can rely on the document always being there.

Those operations should be considered cheap.

Reboot / crash resistant

The DB can crash / restart, but no lose of functionality may occur, but as soon as it restarts, everything goes on as usual. There can be no in memory data structures / work that cannot be recovered from persistent structure.

Views for searching

The DB use views, defined using linq expressions, for supporting search capabilities. Those views are background indexed (so no holding up request processing for views). When you get a result from a queue you always know if the result is stale or not.

Adding a view to an existing database is a cheap operation, regardless of the database size. During view construction, the view can be queried (but its results will be considered stale). Reboot during view construction will not impact the construction process.

Indexing a document twice is a stable operation, which means that a view can always choose to re-index things if it so choose.

Overall design

RDB stores two major pieces of information in transactional storage.

Documents, obviously, which are stored in a format that allows to send the document content to the user quickly, and tasks.

Tasks are how RDB maintains state over crashes / reboots, and they also form the base of async work of the database. Any work that is going to take some time for the database to perform is written to transactional storage as a task. Those tasks are things like: “View ‘peopleByName’ should index documents 1 – 42'”.

There are background threads working of off this tasks queue, performing the work and removing the task when they are completed.

The results of each view is written to a Lucene index (one per view).

So far i have the entire structure done, I need to some polishing, and I have a different OSS strategy to go with, but thinks are looking good.