API DesignSmall modifications over a network
In RavenDB 4.0 (yes, that is quite a bit away), we are working on additional data types and storage engines. One of the things that we’ll add, for example, is the notion of gossiping distributed counters. That doesn’t actually matter for our purposes here, however.
What I wanted to talk about today is the problem in making small updates over a network. Consider the counters example. By far the most common operations are some variant of:
counters.Increment(name, 1);
Since the database is remote, we need to send that over the network. But that is a very small operation. Going all the way to the remote database just for that is a big waste of time. Consider that in most systems, we don’t have just a single thread of execution, we have many. Each of them performing their own operations. Allowing each operation to go on its own is a big waste, but what other options can we offer?
There are other considerations as well. While most of the time we’ll be making small operations, it is very common to need to do bulk operations as well. Maybe you are setting up the system for the first time, or importing data from a file, etc. Making large number of individual requests will kill any hope of doing this fast. The obvious answer is to use batching. send a lot of values to the server all at once. This reduce the network overhead significantly.
The API for that would be something like:
using(var batch = counters.Advanced.NewBatch()) { foreach(var line in File.EnumerateAllLines("counters.csv")) { var parts = line.Split(); batch.Increment(parts[0], long.Parse(line[1])); } }
So that is great for big things, but not very helpful for individual operations. We still have to go to the server for each of those.
Of course, we could also use the batch API, but making use of that for an individual operation is… a big waste.
What to do?
The end result we arrived at was that we have three levels of API:
- counters.Increment(name, 1); – single operation, executed immediately over the network. Guaranteed to succeed or fail and give you the result.
- counters.Advanced.NewBatch() – batch operation, executed over all of the items in the batch (but not as a single transaction), will let you know if the whole operation succeeded, or if there was an issue with something.
- counters.Batch.Increment() – the default batch, thread safe, can be utilized by individual requests. This is a fire & forget operation. We increment, and behind the scene we’ll merge all the increment from all the threads and send them in batches to the server.
Note that the last option means that we’ll only do batches, so only when enough time has lapsed or we have enough items to send will we send the data to the server. The idea is that you get to choose, based on the importance of what you are doing.
If you need confirmation that something was successful, use the individual operation. If you just want us to make best effort, and if something bad really happened you don’t care about it, use the batch option.
Note that I’m using the counters example here because it is simple, but the same applies for things like time series, and other stuff that we are currently building.
More posts in "API Design" series:
- (04 Dec 2017) The lack of a method was intentional forethought
- (27 Jul 2016) robust error handling and recovery
- (20 Jul 2015) We’ll let the users sort it out
- (17 Jul 2015) Small modifications over a network
- (01 Jun 2012) Sharding Status for failure scenarios–Solving at the right granularity
- (31 May 2012) Sharding Status for failure scenarios–explicit failure management doesn’t work
- (30 May 2012) Sharding Status for failure scenarios–explicit failure management
- (29 May 2012) Sharding Status for failure scenarios–ignore and move on
- (28 May 2012) Sharding Status for failure scenarios
Comments
Great improvement :) A new lucene version and new Json would be also a big performance boost
Drill down sideways queries, auto complete features (ngram), auto sharding and Linux support will make ravendb the best product in the market
Atomic counters/sequences would be a very useful feature. The batch feature is a nice touch.
Trying to understand why this isn't just a scripting case. In your example you loop over CSV and propose an increment call per part which gets batched up and committed outside the user's control. Why not define a JavaScript on the server that takes an array ("a batch") of parts sent over the wire and iterates over it on the server instead? This way the user gets fine grained control over the batch. I'm guessing the answer has something to do with your distributed counter architecture...
Eli, Here is my answer: http://ayende.com/blog/171233/api-design-well-let-the-users-sort-it-out?key=f6c2813b568f4eadbf3a2b6b77fdd4cb
Comment preview