We an investigation into locking cost we have when we noticed this:

And then we realized that we also need to have hold the reader lock across async/await calls (which is not safe to do with ReaderWriterLockSlim. And as long as we need to do this, we might as well customize it for our needs.

We need something that has as little cost as possible, allows us to support entering the read lock from one thread while exiting from another and need to handle contention well. Our scenario in this case is producers holding the lock while the generate work, and then a flusher that goes and clean this up while holding the write lock.

This means that we also need to consider that we have only two states (readers & single writer), and that the writer will likely hold the lock for a significant amount of time.

Reading the code for RederWriterLockSlim is quite instructive. And I got some really cool ideas from it.

Here is the most important piece in my implementation

The _waiters variable holds the key state for the lock. Here is how we take a reader lock:

We increment the _waiters using interlocked operation, and then check if we have a waiter pending, if not, we are done and we have the read lock.

This means that the cost of taking a write lock is simply a single interlocked instruction, a mask and comparison. That is pretty cheap.

To take a write lock is just a bit more complex:

We declare the fact that we want a writer using the Interlocked.Add, which will block future readers from taking the lock, then we try to get the lock for the writer. If we fail with either, we just wait to do this again after the next lock release.

In other words, the cost for uncontended read is 1 interlocked operation, and the cost for uncontended write is two interlocked operations. Which is pretty cool, all around.

The actual implementation can be found here, and it handle cases such as too many reader locks, and all the rest of the actual details.