RavenDB replication had a load balancing mode for quite some time, using round robin balancing between all the nodes.

With RavenDB 3.5, we have added support for Service Level Agreements with the load balancing. You can configure it like so:

What this does is instruct the client to define a policy, and if a server starts responding too slowly for the provided policy, we are going to reduce the number of requests that we are going to send its way. For example, let us say that we use the default 100ms per request, with three nodes.

We start with all read requests being spread across all three servers. All of them respond pretty much the same, so we have no reason to change the load allocation. Now, and administrator is running a full backup on one of the nodes, which can consume quite a lot of I/O, so requests from this node suffer and become slower. The moment that those requests start going over the 100ms limit, RavenDB wakes up, and say: "It isn't nice to violate the SLA, maybe I won't visit as often", and start changing the request allocation away from this server.

Note that we aren't going to stop talking to it directly, the RavenDB client will just fewer requests its way, until either the server is able to keep up with the load in the given SLA. As long as it can't keep up with the SLA, we will further reduce the number of requests. This whole approach uses decaying semantics, so new responses are more important than old responses, which means that as soon as the backup operation completes and we have free I/O, RavenDB will detect that and ramp up the number of requests until we are balanced again.

We actually have two modes here. One of them (shown above) talks only to the primary, and if the SLA is violated, it will then start directing queries to other servers. The second mode is to spread the load along all servers, and only use the servers that are under the provided SLA. I think that the second mode is more interesting, but the first one is easier to reason about, because it is a simple overflow model.

As a reminder, we have the RavenDB Conference in Texas in a few months, which would be an excellent opportunity to see RavenDB 3.5 in all its glory.