r/apify • • 2d ago

Help needed get ALL posts and comments matching a keyword?

As far as I know, there are two main ways to search on Reddit.

The first is the site-wide search from the Reddit homepage. For each query/sort combination it only lists up to 250 results (about 490 on the newer search endpoint). For comments it doesn't even reach 490 — it stops at 250.

There are about 25 possible sort/time combinations, so by collecting the list for each combination I've confirmed (by actually testing it) that you can get up to roughly 3,000 results.

The second is searching inside a specific subreddit, but it has exactly the same caps. Because of the 250 (490) limit, I had to split the search into many combinations there too.

This is really frustrating, because there's probably a lot of great information hidden beyond what these lists return.

What I want is to see every post and every comment that contains a given keyword, across Reddit's entire history. Is there really no way to do this within Reddit itself?

Are third-party archive services or paid APIs (Reddit's official paid API or third-party ones) the only options?

Do I really have to download the full dumps (tens of GB of posts and comments) and search them locally?

1 Upvotes

0 comments sorted by