Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,128 | Comments: 45,548

filter by tags archive

Maximum addressable virtual memory

time to read 2 min | 259 words

I wonder what is says about what I am doing right now that I really wish that I could have the OS give me more control over virtual memory allocation. At any rate, the point of this post is to point out something quite important to people writing databases, especially databases that make use of virtual memory a lot.

There isn’t quite as much of it as I thought it would be. Oh, on 32 bits the 4GB limits is really hard to swallow. But on 64 bits, the situation is much better, but still constrained.

On Windows, using x64, you are actually limited to merely 8TB of address space. In Windows 8.1, (and I assume, but couldn’t verify, Windows 2012 R2) you can use up to 128TB of virtual address space. With Linux, at least since 2.6.32, and probably earlier, the limit is 128TB per process.

Implications for Voron, by the way, is that the total size of of all databases in a single process can be up to 8TB (probably somewhat less than that, the other stuff will also need memory). Currently the biggest RavenDB database that I am aware of was approaching the 1.5 – 2.0 TB mark last I checked (several months ago), and Esent, our current technology, is limited to 16TB per database.

So it isn’t great news, but it is probably something that I can live with. And at least I can give proper recommendations. In practice, I don’t think that this would be an issue. But that is good to know.



Is 32 bit even relevant for you?

You can tell big customers to just use Server 2012. That should solve it for 5-10 years until the next OS version comes out.

Howard Chu

Windows has always been lame as a server platform. I'm impressed that they've finally stretched to 128TB in 8.1, it's about time they released an OS that actually supported the full range of the underlying processor. Still, Windows 8 has been so poorly received it hardly matters.

Bartosz Adamczewski

@tobi, Many clients are very resistant to change so they will not switch just because you want it and still ignoring those clients is foolish as this is potential revenue.

I'm not very familiar with Voron (meaning I know 0) but as far as I know you can control virtual memory allocation using VirtualAllocEx this is also known as off heap allocation. The one thing beyond control is the maximum addressable space. In such cases where you are running out you need to move some of those allocated resources to local addressable file that mimics VM, that will hit performance if not done right (you need to know what goes and what stays based on scoring, metrics, requirements etc).

Ayende Rahien

Tobi, 32 bits is relevant pretty much because of you. As far as I am aware, you are the one running RavenDB on the most outdated hardware. And yes, you can say that you need Server 2012 for very big DBs, it is just interesting to note those limitations

Ayende Rahien

Bartosz, Voron relies very heavily on memory mapped files. In practice, I expect most 32 bits systems to hit issue when they go beyond 768 MB - 1 GB, just because of the address space limit.

Carsten Hansen

I wonder whether the bottleneck of ESENT is the size of Version Store and transaction size.

See http://managedesent.codeplex.com/discussions/248471 https://blogs.technet.com/b/exchange/archive/2006/04/19/425722.aspx https://groups.google.com/forum/#!topic/ravendb/389Q2D_owbM

The version store for this instance (2) has reached its maximum size of 1023Mb


Ayende, I honestly did not get that comment. I'm not a RavenDB customer. Maybe a confusion with another customer.

Kijana Woodard

@tobi - Wow. There is a "tobi" who posts to the raven forum regularly who runs on very constrained hardware. All this time I thought "blog tobi" was the same as "forum tobi". Lol.

Ayende Rahien

Carsten, No, this has nothing to do about it.

Ayende Rahien

Tobi, See Kijanna's comment. I always assumed that you were the same person from the forum.


One of the benefits of such a banal user name is internet anonymity. If your name is John Smith you are completely anonymous.


Ayende, didn't it seem like a split personality to you? I must have behaved differently than him. My opinions are not exactly mainstream.

Ryan Heath

@Tobi Not to offend any Tobi ;) But you "sounded" like the "forum Tobi". Ie smart and often a different pov ;)

// Ryan


Just for the record: I'm the other tobi :-)

I'm using RavenDB as an embedded DB for a POS cash register application. The hardware this is running on often only comes with an Atom CPU (D525 and similar). And the OS that is shipped with the hardware mostly is only 32bit.

The worst case I've evere needed to run RavenDB on was a 800Mhz Celeron with 512MB RAM. - it works, but I wouldn't recommend this :-)


@e-tobi, It is good to know that I'm not the only one going down this path. I'm using RavenDB in the same use case (embedded DB for a POS application, with very low-end hardware)... :-)


@Rodirigo: Argh! Another competitor:-) But good to know, I'm not the only one. I think you're only the 3'rd person I'm aware of, that uses RavenDB for a "plain old desktop application" :-) Would be nice to know, what other interesting projects RavenDB is used out there. I'm only aware of some web apps. If you ever want to share experience about using RavenDB in the POS environment, feel free to drop me a mail.

Comment preview

Comments have been closed on this topic.


  1. The low level interview question - 3 hours from now
  2. The worker pattern - 3 days from now

There are posts all the way to May 30, 2016


  1. The design of RavenDB 4.0 (14):
    26 May 2016 - The client side
  2. RavenDB 3.5 whirl wind tour (14):
    25 May 2016 - Got anything to declare, ya smuggler?
  3. Tasks for the new comer (2):
    15 Apr 2016 - Quartz.NET with RavenDB
  4. Code through the looking glass (5):
    18 Mar 2016 - And a linear search to rule them
  5. Find the bug (8):
    29 Feb 2016 - When you can't rely on your own identity
View all series



Main feed Feed Stats
Comments feed   Comments Feed Stats