Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,026 | Comments: 44,842

filter by tags archive

RavenDB Sharding – Blind sharding

time to read 6 min | 1148 words

From the get go, RavenDB was designed with sharding in mind. We had a sharding client inside RavenDB when we shipped, and it made for some really cool demos.

It also wasn’t really popular, we didn’t implement some things for sharding. We always intended to, but we had other things to do and no one was asking for it much.

That was strange. I decided that we needed to do two major things.

  • First, to make sure that the experience for writing in a sharded environment was as close as we could get to the one you get with a non sharded environment.
  • Second, we had to make it simple to use sharding.

Before our changes, in order to use sharding you had to do the following:

  • Setup multiple RavenDB server.
  • Create a list of those servers urls.
  • Implement IShardStrategy, which exposes
    • IShardAccesStrategy – determine how we call to the servers.
    • IShardSelectionStrategy – determine how we select which server a new instance will go to, and what server an existing instance belongs on.
    • IShardResolutionStrategy – determine which servers we should query when we are querying for data (allow to optimize which servers we are actually hitting for particular queries)

All in all, you would need to write a minimum of 3 classes, and have to write some sharding code that can be… somewhat tricky.

Oh, it works, and it is a great design. It is also complex, and it makes it harder to use sharding.

Instead, we now have the following scenario:


As you can see, here we have three different servers, each running in a different port. Let us see what we need to do to get us working with this from the client code:


First, we need to define the servers (and their names), then we create a shard strategy and use that to create a sharded document store. Once that is done, we are home free, and can do pretty much whatever we want:

string asianId, middleEasternId, americanId;

using (var session = documentStore.OpenSession())
    var asian = new Company { Name = "Company 1", Region = "Asia" };
    var middleEastern = new Company { Name = "Company 2", Region = "Middle-East" };
    var american = new Company { Name = "Company 3", Region = "America" };

    asianId = asian.Id;
    americanId = american.Id;
    middleEasternId = middleEastern.Id;

    session.Store(new Invoice { CompanyId = american.Id, Amount = 3 });
    session.Store(new Invoice { CompanyId = asian.Id, Amount = 5 });
    session.Store(new Invoice { CompanyId = middleEastern.Id, Amount = 12 });


using (var session = documentStore.OpenSession())
        .Where(x => x.Region == "America")


        .Where(x => x.CompanyId == asianId)

What you see here is the code that saves both companies and invoices, and does this over multiple servers. Let us see the log output for this code:


You can see a few interesting things here:

  • The first four requests are to manage the document ids (hilos). By default, we use the first server as the one that will store all the hilo information.
  • Next (request #5) we are saving two documents, note that the shard id is now part of the document id.
  • Request 6 and 7 here are actually queries, we returned 1 results for the first query, and none for the second.

Let us look at another shard now:


This is much shorter, since we don’t have the hilo requests. The first request is there to store two documents, and then we see two queries, both of which return no results.

And the last shard:


Here we again don’t see the hilo requests (since they are all on the first server). We do see putting of the two docs, and request #2 is a query that returns no results.

Request #3 is interesting, because we did not see that anywhere else. Since we did a load by id, and since by default we store the shard id in the document id, we were able to optimize this operation and go directly to the relevant shard, bypassing the need to query anything other server.

The last request is a query, for which we have a result.

So what did we have so far?

We were able to easily configure RavenDB to use 3 ways sharding in a few lines of code. It automatically distributed writes and reads for us, and when it could, it optimized the data access so it would only access the relevant shards. Writes are distributed on a round robin basis, so it is pretty fair. And reads are optimized on whatever we can figure out a minimal number of shards to query. For example, when we do a load by id, we can figure out what the shard id is, and query that server directly, rather than all of them.

Pretty cool, if you ask me.

Now, you might have noticed that I called this post Blind Sharding. The reason this is called this name is that this is pretty much the lowest rung in the sharding ladder. It is good, it split your data and it tries to optimize things, but it isn’t the best solution. I’ll discuss a better solution in my next post.


Itay Sagui

What about moving from a non-sharded environment to a sharded environment? Is there a magic solution, or do we need to write or own migration code to pull data from the old environment, and put it in the new environment?

Ayende Rahien

Itay, You would need to do some migration in order to go there, yes.


In the first 3 screenshots - I spent far too long trying to work out why you were censoring the available commands - before realising you were underlining the URLs instead.


@Damien, hahaha, me too!!!


Can you tell when it's time to switch from single database to sharded? Is it number of records or database size? And another one: how does Ravendb perform queries across shards? I assume it glues the results at the client - is this true? What about paging and sorting then?


this just makes me drool.

Daniel Lang

This is a very nice feature, obviously. However, I'm not sure if it was worth the effort:

If someone needs to have sharded instances, then he plays the big data game, which means he will have to put a serious amount of effort into his application and will certainly need to get a very good understanding of RavenDB anyway. Because this simple round-robin won't be sufficient in these situations, he would definitely want to implement his own sharding strategy to have full control over where the documents go. All the others that aren't comfortable writing their own sharding-strategy won't have such demanding applications and therefore only want replication and not sharding if any clustering at all.

Altogether, very good job, but if find it unlikely that someone will use it that way.

Ayende Rahien

Rafal, Wait for the next post, it discusses how we handle merging queries. :-) And you usually shard based on expected load, rather than actual data size.

Ayende Rahien

Daniel, You are correct with regards to that, sure. Except that you are missing tomorrow's post, which deals with that.

But even so, you are still not considering an important fact. Blind sharding is super important to let you start playing with sharding, without a big hurdle of the initial cost

Matthew Shapiro

How easy is it to go from blind sharding to a specialized sharding strategy later? Seems like it would be easier to figure out your sharding strategy right away based on the load points of your single RavenDB instance and do sharding properly from the beginning.

Hendry Luk

One of RavenDb's selling points is cross-document transactions. Would this still work in a sharded setup? And likewise with data constraints (such as unique-constraint). Also, will it be possible to configure your sharding in a routing server (like mongos) rather than within each client?

Ayende Rahien

Hendry, Cross docs transactions till work, yes. As you'll see today in the next post, we handle things in such a fashion that we have great locality of reference, so most of those features just work.

Things like Unique Constraints cannot work in this fashion, because they assume they have access to the entire data set, but there are other things that can be doe to make this work

Bob Horn

What happens when when of the shards goes down, especially the first one? Will another shard take over the hilo duties?

Ayende Rahien

Bob, that depends on how you setup the system. You can do sharding AND replication that means that you have a second node ready to work if the first one is down.

Comment preview

Comments have been closed on this topic.


No future posts left, oh my!


  1. Technical observations from my wife (3):
    13 Nov 2015 - Production issues
  2. Production postmortem (13):
    13 Nov 2015 - The case of the “it is slow on that machine (only)”
  3. Speaking (5):
    09 Nov 2015 - Community talk in Kiev, Ukraine–What does it take to be a good developer
  4. Find the bug (5):
    11 Sep 2015 - The concurrent memory buster
  5. Buffer allocation strategies (3):
    09 Sep 2015 - Bad usage patterns
View all series


Main feed Feed Stats
Comments feed   Comments Feed Stats