RavenDB Security ReviewEncrypting data on disk
Continuing our discussion on nonce reuse issues that were raised in the security report, I want to talk about the way we encrypt the most important thing, your data.
RavenDB uses an algorithm called XChaCha20Poly1305 with 256 bits key to encrypt the data. But as we have learned, just using a key is not good enough, we need to use a nonce as well. This is easy when you need to encrypt a message in one go, but the way encryption in RavenDB works, we need to encrypt pieces of the data (randomly, depending on the way users are accessing the system).
In order to do that, RavenDB encrypt each page (usually 8KB in size) independently of each other. We actually use a different key for each page, derived from the master key, but I’ll touch on that in a different post. Here, I want to talk about nonce.
Encryption today is not just about hiding data, it is also about being able to detect if the value has been tampered with, typically called authenticated encryption (AEAD). The algorithm we use requires 16 bytes for the message authentication code (MAC). The problem is that we need to store that MAC somewhere, and the nonce as well. And that value cannot be in the page itself, since that is encrypted.
Luckily for us, we have the page header, a 64 bytes that are reserved at the beginning of each page. And we planned things accordingly to ensure that RavenDB will use only 32 bytes out of the header, giving us 32 bytes free for the encryption to use. The problem is that the XChaCha20Poly1305 algorithm uses a 16 bytes MAC and a 24 bytes nonce. And that is a bit too much to fit in 32 bytes, as you can imagine. Here is the kind of space allocation we have:
Increasing the size of the page header will have repercussions throughout our system, very late in the game, so we didn’t want to do that. Instead, we cheated. The 16 bytes of the nonce are generated using a cryptographic random number generator, but we pass a pointer to the page header 8 bytes before the nonce, so the encryption algorithm also takes the last 8 bytes of the page header itself into account in the nonce. We are guaranteed at least 128 bits of strong randomness there, and the page header itself will change from time to time, obviously, but we rely on the nonce random bytes to ensure uniqueness.
In this manner, we are able to fit the encryption requirements into our existing file structure and have strong encryption without uprooting everything.
More posts in "RavenDB Security Review" series:
- (27 Mar 2018) Non-Constant Time Secret Comparison
- (26 Mar 2018) Encrypt, don’t obfuscate
- (23 Mar 2018) Encrypting data on disk
- (22 Mar 2018) Nonce reuse
- (20 Mar 2018) Finding and details
So, you take 8 bytes from miscellaneous data as part of the nonce, right? Why don't you take the 8 bytes page number instead? Page numbers are supposed to be unique, aren't they? Preventing nonce reuse.
Jesus, The page number is fixed. The key here is that we don't want to use the same nonce value twice for the same page.
What is the performance impact of using encryption?
Also, how do you approach this w.r.t. journal writes / data syncs. I could imagine a few approaches with different pros/cons:
Alex, Depending on the exact usecase, it case by 5% - 20% cost (most CPU). As for journals, we are going through the normal transaction writing, including diffing and compression, then encrypt the transaction when writing it to the journal.
The data sync code just moves encrypted pages from the temporary storage to the data file, without bother to encrypt / decrypt them.
Regarding the size data. Transactions are always rounded to 4KB boundaries, and we are writing to preallocated files. In order to see what size a particular transaction is, you are going to be able to monitor system calls usage from RavenDB. If you have permissions to do that, you can probably already have permissions to talk to RavenDB as an admin, in which case, you can just ask it to give you the encryption key.