Using the .NET JIT to reduce abstraction overhead

time to read 2 min | 349 words

I ran into this recently and I thought that this technique would make a great post. We are using that extensively inside of RavenDB to reduce the overhead of abstractions while not limiting our capabilities. It is probably best that I’ll start with an example. We have a need to perform some action, which needs to be specialized by the caller.

For example, let’s imagine that we want to aggregate the result of calling multiple services for a certain task. Consider the following code:

As you can see, the code above sends a single request to multiple locations and aggregates the results. The point is that we can separate the creation of the request (and all that this entails) from the actual logic for aggregating the results.

Here is a typical usage for this sort of code:

You can notice that the code is fairly simple, and uses lambdas for injecting the specialized behavior into the process.

That leads to a bunch of problems:

  • Delegate / lambda invocation is more expensive.
  • Lambdas need to be allocated.
  • They capture state (and may capture more and for a lot longer than you would expect).

In short, when I look at this, I see performance issues down the road. But it turns out that I can write very similar code, without any of those issues, like this:

Here, instead of passing lambdas, we pass an interface. That has the same exact cost as lambda, in fact. However, in this case we also specify that this interface must be implemented by a struct (value type). That leads to really interesting behavior, since at JIT time, the system knows that there is no abstraction here, it can do optimizations such as inlining or calling the method directly (with no abstraction overhead). It also means that any state that we capture is done so explicitly (and we won’t be tainted by other lambdas in the method).

We still have good a separation between the process we run and the way we specialize that, but without any runtime overhead on this. The code itself is a bit more verbose, but not too onerous.