That example seems weird, it didn't achieve what its goal.

• He put MaximumRequestLimit and SlidingTimeWindowInSeconds inside model, my guess that treat as setting, which is already weird place to begin with. He never use it at end. He use hard coded variables. REQUEST_LIMIT and SLIDING_TIME_WINDOW_IN_SECONDS.
• The counter is not aware of time. This is not 30 request every 30 seconds limit. It can as far as goes to 30 request every 900 seconds. e.g. every request have 29~30 seconds apart. That document will constantly sliding whenever the count increment.

One way to implement would be time ranged document. ```json5 { id: 'throttling/counter/{purpose}/2020-02-27T02:35:30Z' } // document has counter ~~~ The ID of document should be 15 seconds apart. e.g. * throttling/counter/{purpose}/2020-02-27T02:35:30Z * throttling/counter/{purpose}/2020-02-27T02:35:45Z * throttling/counter/{purpose}/2020-02-27T02:36:00Z * throttling/counter/{purpose}/2020-02-27T02:36:15Z

Whenever request happen, get current range and last range, add together it's current allocated request count. If total exceed 30, then stop. The document can also have expire time, such as expire after 2 hours, so database can clean up the junks.

e.g. If current time is 2020-02-27T02:36:29Z, I will grab 2020-02-27T02:36:15Z and 2020-02-27T02:36:00Ztime frame document. If first time range had 10 count, second had 5 count, its safe. If first range had 30 and second range had 0, means it is currently on throttling.

The time range can vary, 15 seconds is more accurate than 30 seconds, time range too small means too many documents to maintain.

On the other hand, throttling normally happen at API level, if client level, it can implement in memory. Such as Interlocked.Increment. But that's not the main problem. I know the article is an example of use expiration, counter, locker, but it is not the solution for its initial goal.

The example I give didn't explain the issue well. The better example would be small pock to cause constantly slide, then sudden spike will hit limit.

1. On 0 mark, requested 28 requests. Expire on 30 mark. (count is 28)
2. On 29 mark, did another request, which slide to expire on 59 mark. (count is now 29)
3. On 58 mark, I want to do another 29 requests, now the count is 29, which only allow me to execute one request. The other 28 request have to wait 30 more seconds.

Hi @Jason,

Thanks for the feedback, let me see if I can help clarify a few things.

He put MaximumRequestLimit and SlidingTimeWindowInSeconds inside model, my guess that treat as setting, which is already weird place to begin with. He never use it at end. He use hard coded variables. REQUEST_LIMIT and SLIDING_TIME_WINDOW_IN_SECONDS.

The example model with MaximumRequestLimit and SlidingTimeWindowInSeconds is meant to be an illustration, the demo code doesn't actually use that to store the settings. I wanted to just call out that you could load settings from the database instead of hardcoding, if that's what you wished to do. You may want to store them separately, in a MyServiceConfig document, for example.

The counter is not aware of time. This is not 30 request every 30 seconds limit. It can as far as goes to 30 request every 900 seconds. e.g. every request have 29~30 seconds apart. That document will constantly sliding whenever the count increment.

I am not sure I understand. In the demo, the counter will increment to 30 and will stop making requests until the window expires (which at a minimum will be 30 seconds and at a maximum, 90 seconds due to the RavenDB setting for 'delete frequency interval'). The window doesn't reset when the counter is incremented, they are two separate mechanisms, it only resets when RavenDB deletes the expired document.

If the first request took 30 seconds, that's OK, because we're throttling to 30 requests per 30s interval, we won't hit the external API rate limit if that's the case. We just don't want to go over it, we're rate limiting ourselves. We are not trying to "evenly distribute" requests, I think that's a different story altogether.

To give a solid example, I use this in production to throttle outgoing requests to a web service that will throw an exception if you exceed 200 requests within 30 minutes. This throttling mechanism ensures I don't exceed it (and in the rare case it allows an additional request, it's a message queue, so the message will be put back onto the queue and be throttled next time).

On the other hand, throttling normally happen at API level, if client level, it can implement in memory. Such as Interlocked.Increment.

The premise of the article is that you can't do it in-memory when you have multiple processes running on a cluster, so other solutions using Interlocked.Increment don't quite work. You need some kind of backing store to coordinate the throttling and so I present one way to do it using RavenDB alone.

The better example would be small pock to cause constantly slide, then sudden spike will hit limit.

I don't think this is quite correct. Remember as long as the counter is at or above maximum, the code must wait until the document expires and RavenDB deletes it, to resume again. Incrementing the counter has no bearing on the expiration time (it is set at document creation time). Only when the document is deleted will the counter reset back to 0.

I encourage you to run the sample application and check it out, I think it does what you expect and I've been running essentially the same code in production for some weeks now to help proactively manage requests to an external API without much issue. I could be misunderstanding what you mean, though, so let me know if it still is confusing.

Cheers, Kamran