+972 52-548-6969
API DesignWe’ll let the users sort it out
In my previous post, I explained an API design that give the user the option to perform an immediate operation, use the default fire and forget or use an explicit bulk mechanism. The idea is that most operations are small, and that the cost of actually going over the network is going to dominate the cost of the entire operation. In this case, we want to give the user the option of selecting the “I want to know the operation completed” or “I just want to try the best, I’m fine if there is a failure” modes.
Eli asks:
Trying to understand why this isn't just a scripting case. In your example you loop over CSV and propose an increment call per part which gets batched up and committed outside the user's control. Why not define a JavaScript on the server that takes an array ("a batch") of parts sent over the wire and iterates over it on the server instead? This way the user gets fine grained control over the batch. I'm guessing the answer has something to do with your distributed counter architecture...
If I understand Eli properly, the idea is that we’ll just expose an API like this:
Increment(“users/1/visits”, 1);
And provide an endpoint where a user can POST a JavaScript code that will call this. The idea is that the user will be able to decide whatever to call this once, or send a whole batch of updates in one go.
This is certainly an option, but in my considered opinion, it is a pretty bad one. It has nothing to do with the distributed architecture, it has to do with the burden we put on the user. The actual semantics of “go all the way to the server and confirm the operation” vs “let us do a bulk insert kind of thing” are pretty easy. Each of them has a pretty well defined behavior.
But what happens when you want to do an operation per page view? From the point of view of your code, you are making a single operation (incrementing the counters for a particular entity). From the point of view of the system as a whole, you are generating a whole lot of individual requests that would be much better off as a single bulk request.
Having a scripting endpoint gives the user the option of doing that, sure, but then they need to handle:
- Error recovery
- Multi threading
- Flushing on batch size / time
- Failover
And many more that I’m probably forgetting. By providing the users with the option of making an informed choice about speed vs. safety, we avoid putting the onus of the actual implementation on them.
More posts in "API Design" series:
- (04 Dec 2017) The lack of a method was intentional forethought
- (27 Jul 2016) robust error handling and recovery
- (20 Jul 2015) We’ll let the users sort it out
- (17 Jul 2015) Small modifications over a network
- (01 Jun 2012) Sharding Status for failure scenarios–Solving at the right granularity
- (31 May 2012) Sharding Status for failure scenarios–explicit failure management doesn’t work
- (30 May 2012) Sharding Status for failure scenarios–explicit failure management
- (29 May 2012) Sharding Status for failure scenarios–ignore and move on
- (28 May 2012) Sharding Status for failure scenarios
Comments
What I mean is to expose an API like this: Item[] batch = {"item1", "item2"...}; JSEngineResponse response = JSEngine.exec(function(batch) { // where function is the user's implementation or a name of a function on db for (var i = 0; i < batch.length; i++) { var item = batch[i]; Increment(item, 1); // where Increment is part of a scriptable interface in the JSEngine. could expose other common tasks for users to call into } }, batch);
The user would be responsible for collecting up the batch and setting some threshold batch size or timeout to invoke JSEngine.exec(), but that should be the user's prerogative anyway. They should be able to define the responsiveness of whatever thing (eg counter) they are watching and making tradeoffs between # of requests versus freshness. Multithreading, error recovery, failover, other complexities could be handled by a combination of smarts in the JSEngine executor and using your original concept of a Batch where instead of calling Increment directly you're simply putting items in buckets in a threadsafe manner to be processed later via the function passed during exec().
I guess what I'm getting at is that I think your solution isn't general enough by defining a Batch with an Increment method directly. As a user, I'd rather call that when the entire payload lands on the server and define what happens with the payload myself.
Eli, "user would be responsible for collecting up the batch and setting some threshold batch size or timeout to invoke" - that is a big thing to put on the user.
Also, I think we need to define the context here. In our terminology, a server is the RavenDB server (the db server) and the client is the calling code (usually a web app). Asking the user to correlate multiple requests to create batches and send them properly is a decidedly non trivial task.
As a simple example, creating a large batch and sending them to the db server as a big string will have the unfortunate side effect of creating large strings on the web app client, which can cause large object heap fragmentation. I guarantee that the dev writing the web app code will not think about that up front, but over time, that has a big effect on his system.
By making sure that we are handling that, we are avoiding the whole issue
Comment preview