In a previous post, I showed how to get 10x performance boost by batching remote calls. The code included the following method:
This code has a subtle bug in it. Take a look and see if you can find it.
Yes, I’ll wait, honestly.
Go read the code and think about interesting ways to break it.
Okay, found it? Awesome, now let me explain anyway .
This code suffer from the slow arrival syndrome. If we have a new request coming in every 100 ms, then the first request here will languish in the queue for over 25 seconds!
Luckily, the fix is simple:
We just need to make sure that we aren’t waiting for too long. In this case, we will wait a maximum of less than half a second for the messages to go over the wire.