In a previous post, I asked about designing a document DB, and brought up the issue of storage, along with a set of questions that needs to be handled:

• How do we physically store things?

There are several options, from building our own persistent format, to using an RDMBS. I think that the most effective option would be to use Esent. It is small, highly efficient, require no installation and very simple to use. It also neatly resolve a lot of the questions that we have to ask in addition to that.

• How do we do backups?

Esent already has the facilities to do that, so we have very little to worry about it here.

• How do we handle corrupted state?

See above, Esent is also pretty good in doing auto recovery, which is a nice plus.

• Where do we store the views?
• Should we store them in the same file as the actual data? 

I think not, I think that the best alternative is to have a file per view. That should make things such backing up just the DB easier, not to mention that it will reduce contention internally. Esent is built to handle that, but it is better to make it this way than not. All the data (include logs & temp dirs) should reside inside the same directory.

Crash recovery on startup should be enabled. Transactions should probably avoid crossing file boundaries.It is important the the files will include a version table, which will allow to detect invalid versions (caused a whole bunch of problems with RDHT until we fixed it).

• Are we transactional?

Yes, we are transactional. But only for document writes. We are not transactional for document + views, for example, since view generation is done as a background service.

• Do we allow multi document operation to be transactional?

Yes, depending on the operation. We allow submittal of several document writes / deletes at the same time, and they would succeed or fail as a single unit. Beyond that, no.