+972 52-548-6969
Design exerciseFile system abstraction in RavenDB
The question recently came up in a discussion with a customer. They have an existing binary storage solution that they want to migrate to RavenDB. No problems, right? RavenDB has attachments support for just this reason, after all.
The key from their perspective is that their current solution gives them a file system abstraction, and they want to keep the same concept when moving the data to RavenDB. In RavenDB, we tend to think about the data as binaries attached to documents, not as raw files. But the actual solution ended up being quite elegant.
We start by creating the following “document”.
At the moment, it is pretty bare bones, but you can add additional items here, such as owner, collaborators, etc. That depends entirely on your system and needs and doesn’t impact how we will be using this. With just this class, we can now build the API. I’ll first show the code, and then discuss it.
As you can see, there really isn’t much here. The idea is that the Folder is used to store the child folders, and the files inside the folder are attachments on the folder document. We expose the typical file system operations (put, list, get file, etc) in a simple interface.
This allows you to build transactional file system on top of RavenDB and expose a natural looking file system format. More advanced usages can be to get multiple levels of the folder tree, implementing permissions, ownership, etc.
More posts in "Design exercise" series:
- (01 Aug 2019) Complex data aggregation with RavenDB
- (31 Jul 2019) Arbitrary range aggregations in RavenDB
- (30 Jul 2019) File system abstraction in RavenDB
- (19 Dec 2018) Distributing (consistent) data at scale, answer
- (18 Dec 2018) Distributing (consistent) data at scale
- (26 Nov 2018) A generic network protocol
Comments
But what if there are many files in a folder and/or you want to sort by size, date, etc.? How would you model many files inside a Folder?
Marco, We have test a single document with > 100,000 files in it. It works. The metadata also includes the size of the object, so it is trivial to do things like sort by size. For more complex stuff, you can simply also hold more data about the files in the
Folderdocument directlyComment preview