Reddit API Ratelimiting Explained
As a purveyor of a potentially usage-intensive tool, Reddit's rate limits have long been a source of some consternation. The primary reason for this is that the term "Ratelimit" as used by Reddit's API metadata in various places is not as well-defined as one might like—especially if one is engaged in writing a Reddit post scheduler, and especially of one has decided to charge per-post, incentivising one to encourage users to post freely (while maintaining a close eye on Reddiquitte, of course).
If you are looking for advice on how Reddit's rate-limiting works, or just want to know how Later deals with it, read on!
There are TWO rate limits!
This is the key point for developers here. When you submit a successful API request to Reddit, you will receive some ratelimit headers with the response. They are, as described by Reddit's API Rules, as follows:
X-Ratelimit-Used: Approximate number of requests used in this periodX-Ratelimit-Remaining: Approximate number of requests left to useX-Ratelimit-Reset: Approximate number of seconds to end of period
This is pretty straightforward. As a baseline Reddit gives you 600 requests every 10 minutes. After each request your client can inspect these headers and possibly implement a delay as appropriate.
However, when you try to submit posts via API, you will sometimes end up with response bodies that look like this:
{"ratelimit": 512.2, "errors": ["RATELIMIT", "You are doing that too much. Please try again in 9 minutes", "ratelimit"]}
Something that stymied me a bit when developing ratelimit-handling logic is that this is not the same ratelimit as above. The response-body ratelimit is instead a limit on the ability of a particular user to perform certain actions (usually submissions or comments). This ratelimit is not especially well-documented, perhaps to allow the Reddit devs to employ it as needed without having to justify themselves.
How does Later handle this?
Later's approach to this problem is to use a system of queues to move items along the posting pipeline—and delay appropriately where required. The pipeline is composed of 3 main parts:
The scheduler is actually just a polling database job. It queries the database every X (presently 20) seconds to find which posts are due, then sends them to the post queue (maintaining a list of post ids that have been processed so that it doesn't double-post).
The post queue is in fact many queues. It maintains a set of per-user queues and routes posts accordingly. It takes posts that are due, looks up their information one last time in the database, then builds appropriate API requests and adds them to the request queue. It also monitors responses for the per-user ratelimit. When it finds one, it parks that queue for as many seconds as reddit requested before it will process any more posts from that user.
The request queue is the final piece of the puzzle. It's just got a single queue from which it reads requests, performs the HTTP request, checks the response for rate-limit headers, returns the response, and then has a wee nap as appropriate.
As a technical detail for those interested, the above system of queues is implemented in Clojure (as is all of Later's backend) using core.async.
How does that affect me, a Later user?
Let's say you accidentally schedule 20 posts for the same time to 20 different subreddits. After some number of those (how many exactly depends on your account's standing with Reddit) Reddit will complain and your queue will become blocked for about 9 or 10 minutes. During this time all your posts will still be in there, waiting to post when the time is up. If you choose to delete or reschedule a post during this time your changes will still take effect.
This doesn't mean that you shouldn't worry about Reddit's rate limits. Reddit's API servers don't seem to mind if you butt up against the rate limits regularly, but the same cannot be said of Reddit's admins and moderators. If you have a lot of posts to make, try to space them out throughout the day, or at very least put about 10 minutes between them.
If you're ever curious about what's happening to your post, head to the Posts page in Later. All completed posts will have a "Show Logs" link below them, which will let you see if they succeeded or if errors were encountered. You'll even have direct access to the response body (if one exists).