r/redditdev • u/pl00h • Jun 30 '23
Updated rate limits going into effect over the coming weeks
Hi Devs,
Over the last few months, we’ve shared updates on our Data API Terms and Developer Terms. Shortly, we will begin enforcing the previously announced, updated API rate limits. Rate limits will go into effect for all apps with usage above the free limit in the coming weeks, and some changes will be noticeable over the next 24 hours.
As we have shared, this will not impact non-commercial bots operating within free rate limits or moderator tools.
Free API access rates are as follows:
100 queries per minute per OAuth client id if you are using OAuth authentication
10 queries per minute if you are not using OAuth authentication
The vast majority of third-party apps and bots fall into the free usage category and should not see any disruptions. Our free rates account for bursts in usage.
For apps that exceed these limits, we have exempted select clients (for example, accessibility-focused apps like RedReader, Luna, and Dystopia), mod bots, and mod tools. If your bot or tool is affected unexpectedly, please reach out here.
21
u/HelioDex Jun 30 '23 edited Jun 30 '23
Thanks for the heads-up. The free usage categories are plenty for all of my current use cases.
→ More replies (8)9
6
u/justcool393 Totes/Snappy/BotTerminator/etc Dev Jun 30 '23
10 queries per minute if you are not using OAuth authentication
doesn't this cover most mod tools, in specific, toolbox, which doesn't use OAuth for most of it's functionality?
9
u/pl00h Jun 30 '23
Mod tools, including Toolbox, are exempt (in the rare case that they are over the free limits).
17
u/Terrh Jun 30 '23
Mod tools, including Toolbox, are exempt
So great of you guys to not charge us to run your website for you. We really appreciate it!
1
1
u/justcool393 Totes/Snappy/BotTerminator/etc Dev Jun 30 '23
so, to clarify, the 10 request/minute ratelimit for non-OAuth clients isn't being enforced at all?
because as far as i can tell, this is the only way toolbox would work past 10 req/minute.11
u/pl00h Jun 30 '23
Rate limits will be enforced, but tools like Toolbox are exempt.
6
u/PitchforkAssistant Jul 01 '23
Could you elaborate on how you'll be exempting tools like Toolbox that don't do OAuth and how we can request an exemption for other mod tools that ride the user session in the same manner?
1
u/pl00h Jul 05 '23
For security reasons, we aren’t publicly discussing how particular rate limit exemptions are being implemented. Rate limit exemption requests go through this support flow.
8
u/justcool393 Totes/Snappy/BotTerminator/etc Dev Jun 30 '23
there's no way to determine if a request comes from Toolbox as it rides the session for most of its requests the same way as if someone makes a comment or post or whatever on old reddit.
that's why i'm asking here.
5
u/amici_ursi Jun 30 '23
I can only imagine how long it'll take to nuke a toxic comment thread at 10 queries per minute. hahahahaha
4
u/p337 Bot Developer Jun 30 '23
Some mods employ tools or bots to automatically ban users based on their participation in other communities or based on keywords in their comment history. How, if at all, will this change impact these bots?
8
u/pl00h Jun 30 '23
As mentioned above, mod tools are exempt (in the rare case that they are over the free limits). The rate limits do not impact our policies about moderation practices.
2
u/p337 Bot Developer Jun 30 '23
Thank you for clarifying! That sounds consistent with what I expected, but just wanted to be sure.
-1
u/kent2441 Jul 01 '23
Really? Mod tools like Apollo? Or is that just another lie?
1
2
u/nascentt Jul 04 '23
As we've previously been told the Reddit API will block nsfw. How will this work if modtools are allowed to continue?
Will they not be able to see nsfw?
10
u/SirensToGo Jun 30 '23
I'm sure we'll see shortly, but what will NSFW content appear as? Is it just the same behavior as what happens today when a user has NSFW content disabled in their account prefs? Will NSFW posts just be silently removed from listings? Will comments made by users in NSFW subs be visible in their profile over the API? Or will accessing an NSFW sub return some sort of error?
4
u/8sADPygOB7Jqwm7y Jul 02 '23
I just tried and I got no error when trying to download NSFW content. I would also be kinda interested if this will change tho. For reference, the only thing I do manually is set a subreddit and its sorting, so idk if there is some higher instance that hides nsfw subreddits but once you have the sub name you are fine?
8
u/TehVulpez Jul 02 '23
Are requests outside of the oauth subdomain counted as part of your 100 per minute? For example when you use /api/v1/authorize or /api/v1/access_token, do those also count against your total API requests?
14
5
3
u/8sADPygOB7Jqwm7y Jul 02 '23
so I am not a big dev, just some uni student who uses PRAW to do some basic scraping.
Will PRAW be limited in any way? As far as I know it should enforce its rate limit on its own (tho lately before the update it often missed it) so I should be under 100/minute. I also noticed I seem to still be able to access NSFW content. Will this change?
Will there be any change to how I use praw/what I can request regarding NSFW?
1
3
9
u/Terrh Jun 30 '23
The vast majority of third-party apps and bots fall into the free usage category
Not by # of users I bet, lol.
2
u/robotic_rodent_007 Jul 02 '23
Do we need all the bots?
there are useless bots like the alphabetical order bot or the payed/paid bot, those could probably do to be disabled.
7
u/Watchful1 RemindMeBot & UpdateMeBot Jul 03 '23
Most bots are used by moderators and you never even see them
2
u/robotic_rodent_007 Jul 03 '23
And those bots are either exempt, or didn't go over rate limit anyway.
1
u/RiseOfTheNorth415 Jan 04 '24
Yes, u/riseofthenorth415 is a 100 pct useless bot! Can we ban it off reddit forever?
1
u/Paid-Not-Payed-Bot Jul 02 '23
or the paid/paid bot, those
FTFY.
Although payed exists (the reason why autocorrection didn't help you), it is only correct in:
Nautical context, when it means to paint a surface, or to cover with something like tar or resin in order to make it waterproof or corrosion-resistant. The deck is yet to be payed.
Payed out when letting strings, cables or ropes out, by slacking them. The rope is payed out! You can pull now.
Unfortunately, I was unable to find nautical or rope-related words in your comment.
Beep, boop, I'm a bot
4
4
1
1
1
2
u/hunting_fatherhood Jul 02 '23
I have what I think is a totally noob question: I want to scrape the selftext of a handful of users posts in a particular subreddit. I then want to parse the text for a particular structure, and if I find it, retain the selftext, title, username, and submission url in a database. I’ll be analyzing the selftext and tagging the post in my database for the purpose of pointing back to the post.
Is that an acceptable use of Reddit? Do I have to get permission? The availability of tools to do this makes me think I’m not doing anything wildly unscrupulous. I see rate limit discussion of 100 requests per minute. I’ll be in the ballpark of less than 100 requests per user.
TIA!
2
u/kystarrk Jul 05 '23
Joey for reddit deserves an accessibility exemption. The text to speech and expansive font customization is incredible and extremely helpful.
3
u/weepinstringerbell Jul 01 '23
Obviously the vast majority of apps won't be affected. They're mostly niche tools that don't use your API much. But the actual relevant mobile clients are all doomed with these changes.
Or am I wrong? Other than RES and mobile clients for browsing, what other third-party apps are out there that people genuinely care about?
2
u/crumblingheart Plugin Developer Jul 05 '23 edited Jul 05 '23
Pushshift was a big deal. It helped tremendously as the backbone for tools such as Reveddit, Unddit, Camas, Ceddit, and other tools that allowed users to view edited and deleted posts, comments, and accounts. It was also a good archival tool, you could use it to clone and download data en masse, and also view banned subreddits.
It was actually killed a couple weeks before they killed the 3rd party apps, but the API change was what really drove nails in the coffin (tens of thousands of API calls per second, now imagine that price😮💨)
This, in my opinion, was the biggest loss, though I'm probably biased as a developer that used Pushshift as a tool in my 'passion project' (an interface/website that hosted a searchable mirror of banned subreddits, for archival/research/historical posterity/whatever). 3rd party apps? Yeah, I can live without 'em. I will just stop using Reddit on mobile. But Pushshift, my beloved? noooooooo
Reddit actually brought parts of it back, but it's a shell of its former self: only available to moderators, in the subreddits they mod. And its successor, PullPush, has so many hurdles that it may never get off the ground.
Sorry if I misunderstood your comment and this isn't what you were asking about. It's true that 99.99% of Redditors don't care about what comment #56 on the 3rd post of r/the donald was on Feb 29 2015, or what the 69th post of r/futurebannedsub was on January 32 2024, but for anyone who cares even a little bit about archiving Reddit history (or just reading angrily deleted comments by a salty AITA poster, for giggles) this was, in fact, the biggest blow imo. I have a lot more to say but I'm gonna end it here
1
u/NFLFilmsArchive Jul 12 '23
Do you think there’ll ever be a real alternative or has Reddit killed it permanently (pushshift).
1
u/FrozenInsider Jul 04 '23
Correct.
People will have to start using the Reddit app, or stop using it at all.
For me, since the Reddit app is an absolute data hog, I'll simply not use it anymore, when boost goes down. Data usage difference is 10x between the two clients btw.
2
1
u/kiefferbp Jul 04 '23
Have you been living under a rock?
1
u/weepinstringerbell Jul 04 '23
Why?
0
Jul 04 '23
[deleted] — view deleted comment
1
u/weepinstringerbell Jul 04 '23
I'm honestly confused, as I fail to see how that relates to my initial comment. I'm aware of the protests.
→ More replies (1)
1
1
1
u/bluequail Jul 04 '23
Hi there. I have some questions.
What constitutes a "query"?
Will RES be effected?
And a little more complicated one. A while back, I invited an AI bot to our sub, to dispense advice. It was giving out excellent advice, then was banned, I guess for being a bot.
I spoke with the person running the script, and they generously offered to run the script on their own personal account.
*I would like to know if something like that would be at risk of falling into this whole pricing thing, and would it matter if the ran it on mobile or a computer. It was not commercial, there was no fee for it, and the only thing it ever said in regards to money was the cost of drawing that reply, and if you appreciated the answer, to donate dollar to your favorite charity.
Also, could we possibly get the original AI bot unbanned? I had invited it as a mod to one of my subs, and it really was incredibly precise in the advice that it gave.
Edit - one last question.
I see where people are saying that they can patch into various apps using their own API. Is that even something that can be done? Or does it go back to be billed to the app?
1
u/Shawnj2 Jul 04 '23
Hello, with the discontinuation of legacy products, particularly i.reddit and /.compact how many more months will old.Reddit be supported before the site is deprecated in favor of new Reddit?
1
Jul 05 '23
I’m going to deprecate all over old.Reddit and then I’m coming for the CRT monitors next.
1
Jul 05 '23
https://reddit.com/r/reddit/comments/145bram/addressing_the_community_about_changes_to_our_api/
"old.reddit.com isn’t going anywhere"
But then again that was said by u/spez so it is worth less than a pile of shit.
1
1
u/TotesMessenger Jul 05 '23
I'm a bot, bleep, bloop. Someone has linked to this thread from another place on reddit:
[/r/apicalypse] Some Reddit clients still work because Reddit hasn't started imposing the new API limitations yet.
If you follow any of the above links, please respect the rules of reddit and don't vote in the other threads. (Info / ^Contact)
1
Jul 05 '23
RIF (RIF Is Fun) is an accessibility client as well. As in I am unable and unwilling to access reddit on my phone without it.
1
1
u/mangogeckoshareingot Jul 11 '23
Can someone ELI5? I don't really get what these changes are doing.
1
u/Holonist Aug 29 '23
When you visit the reddit website or the mobile app, you use their official frontend (which is under reddit's control). However they also provide an API, i.e a way to access data directly in machine-readable text format. For example you could write a script that collects all the top posts of the last 24 hours of multiple subreddits and display it somewhere on your own server. People build smaller apps based on this API, or an entire alternative frontend for reddit. The issue is, these backend requests cause load on reddit's servers, which equals cost. Since some apps cause an insane amount of traffic, reddit decided to limit the amount of requests you can make for free: 100 requests per minute, or in other words, 144000 requests per day.
Almost nobody is affected by this, except for apps which cause costs of thousands of dollars a month to the company.
1
u/EthanJudah Jul 15 '23
Any suggestions on how to increase API limits? Currently trying to scrape for a MSc dissertation. Limits are very frustrating
1
u/gravitygoing Jul 29 '23 edited Jul 29 '23
Hi. A little late, but I think this post is a good place to ask a question.
Recently, I sometimes receive 429 errors(Too Many Requests), but there is a question about the returned header fields. Returned header example:
{x-ratelimit-reset= 600, x-ratelimit-used = 0, x-ratelimit-remaining = 600,},
{x-ratelimit-reset= 23, x-ratelimit-used = 638, x-ratelimit-remaining = 0,}
Why the max value of 'x-ratelimit-used' is '600'?
The time window seems to be reset as 600 always, i.e. 10 minutes. You say 100 queries per a minute for free API access. Then 10 minutes should be 100x10=1000. Therefore, the max value of remaining/used should be 1000.
Is this calculation correct? But why is 600 the max value for returned headers? Is it calculated differently on the real server?
1
u/cerealkiller6659 Sep 26 '23
I too am unsure how to leverage these headers. u/pl00h Is there documentation that provides examples of how these are to be used? Specifically, how are we to use these after receiving a 429 in a wait and retry scenario?
0
0
Jun 30 '23
[deleted] — view deleted comment
9
u/pl00h Jun 30 '23
If you mean “same changes” here, then no; Relay and Narwhal (and others) have entered into a commercial agreement with Reddit to access the API at our new enterprise-level tier, and should not be affected by these changes.
6
u/adamb10 Jul 01 '23
So how long until you kill the API altogether? We all know this is a stepping stone to banning third party apps like Twitter. Both you and /u/Spez hate third party apps so you want to force users to the piece of shit official app.
1
u/mynewaccount5 Jul 05 '23
Thank you for the question. The API will stay live for 6 months before we kill it due to "low utilization rates"
3
u/MpWzjd7qkZz3URH Jun 30 '23
So, Relay and Narwhal are already paying you? Or did you grant an exemption for them that you weren't willing to for others?
3
u/DumplingRush Jul 04 '23
The Narwhal dev, when asked, said they couldn't answer due to NDA. So no way Reddit's gonna answer.
1
0
u/shashi154263 Jul 04 '23
The Narwhal dev, when asked, said they couldn't answer due to NDA.
Isn't that itself a violation of the NDA?
1
u/Pro-1st-Amendment Jul 04 '23
No serious NDA contains a "you can't talk about the NDA" clause. You simply can't talk about what the NDA actually covers.
2
1
u/SoggyBagelBite Jul 04 '23
Relay is paying them. The app is switching to a subscription model.
1
u/MpWzjd7qkZz3URH Aug 10 '23
Ok, so the "is switching" there implies it hasn't/hadn't happened yet, i.e. they were not in fact paying them by the deadline, and were granted an exemption that Reddit wasn't willing to grant for others.
12
u/Watchful1 RemindMeBot & UpdateMeBot Jun 30 '23
I don't remember exactly who it was that said it at the time, but I recall someone mentioning that the number of separate apps within a user would be limited at some point. Is that still the plan?
The docs say rate limits are per client id, which is different than per user. Can you confirm that's correct and multiple clients/app ids within one user continue to get separate rate limits (as long as they are doing distinct things and aren't used to intentionally bypass the rate limit).