We face numerous challenges when scaling the rate limiter to support multiple servers and concurrent threads.
Race Conditions
Synchronisation issue
- We need multiple rate limiters to handle millions of users
To demonstrate, in the image below, both the rate limiters must share the counters for both clients, otherwise the rate limiters would fail their purpose
- In such cases, we need synchronization between rate limiters to keep the counters in sync
- One possible solution is to use centralized data stores like Redis.
