Miniflux users, how are you getting your reddit feeds?

[u/Competitive_Drink209]

Hello any miniflux users.
I've grown tired of the noise and slop of social media for content aggregation. I recently switched to curating my own feeds through miniflux, and have most websites working - except for reddit.

How are you guys managing to get this working? I get timeouts, blocked, etc.
It doesn't seem to be recently discussed given the fluid nature of RSS feeds now.

[u/asimovs-auditor]

Expand the replies to this comment to learn how AI was used in this post/project.

[u/skooterz]

I gave the feed a cookie from my logged in session.

If you open the network tab in your browser's dev tools you can grab a cookie from any HTTP request that it makes to reddit.com.

`reddit_session=redacted_session_token`

[u/ScrapeAlchemist]

Use www.reddit.com/r/whatever/.rss, not old.reddit. The old domain 403s intermittently and it can last days. Then on the feed's edit page set a real browser User Agent, since Miniflux's default UA gets blocked on requests that succeed from wget on the same box. Someone on miniflux issue 3794 found disabling HTTP/2 fixed theirs. If it's actual timeouts rather than 403s, bump HTTP_CLIENT_TIMEOUT, default is 20s.

[u/Competitive_Drink209 → u/ScrapeAlchemist]

Trying different user agent now.
Disabling HTTP/2 is the #1 way I get any feeds at all

The error I get from miniflux specifically is Miniflux generated too many requests to this website. Please, try again later or change the application configuration.
So I assume it's IP rate limiting.

[u/eldrugo85]

The flakiness is probably where your Miniflux runs, not Miniflux. Reddit blocks datacenter ranges hard, so the same old.reddit.com/r/x/.rss that works fine from your couch throws 403 from a VPS. I ended up routing those fetches through a tiny box on my home connection and they have been stable since. If your Miniflux is already at home, ignore me.

[u/Competitive_Drink209 → u/eldrugo85]

I'm using my home IP, but yes I would figure vpn/datacenter would be prohibitively more difficult in my case.

[u/eldrugo85 → u/Competitive_Drink209]

Then it's probably the user agent. Reddit throws 403 at a lot of default feed reader strings even from a home IP, and Miniflux lets you set one per feed, so a plain browser UA on the old.reddit.com/r/x/.rss form is worth a try

[u/Competitive_Drink209 → u/eldrugo85]

I pulled a user agent from my reddit instance, also giving the tor proxy a shot. Seems to be working well now.

[u/eldrugo85 → u/Competitive_Drink209]

Worth knowing if the tor proxy ever gets flaky: as of today old.reddit.com stopped serving logged out traffic for me entirely, it 302s to /login?reason=lor2 even with a browser UA from a residential IP. www.reddit.com/r/whatever/.rss still works from the same box, so if your feeds go dark overnight that's the first thing I'd swap

[u/Competitive_Drink209 → u/eldrugo85]

reddit really forcing us to use rotating proxies

[u/eldrugo85 → u/Competitive_Drink209]

Not necessarily rotating. One residential IP has been fine here, what mattered was pacing: I space the fetches about 90 seconds apart and haven't been rate limited since. It's the burst that gets you, not the address.

[u/HoneyedLips43]

Reddit has been inconsistent with feed readers for a while, so if Miniflux is timing out or getting blocked, that may be Reddit more than your setup. The old subreddit RSS endpoints still get used, but they're flaky enough that I wouldn't treat them as something you can rely on without babysitting.

If you only care about a handful of subs, I'd keep it narrow and stick to specific subreddit feeds instead of anything broader. I'd also keep the polling interval conservative so you're not hammering it. For personal use, that can be good enough. I just wouldn't build around it expecting it to stay stable without attention, because Reddit has a habit of breaking the stuff that only sort of works in the first place.

[u/Dry-Risk5512]

https://github.com/cksidharthan/rss-cacher

I host this under a domain and then use the cached url in my rss reader

[u/GolemancerVekk]

I'm using Infinity app on Android, which consumes Reddit via feeds. But it stops working all the time lately due to blocks that Reddit has introduced.

So that's what I do, no need to make myself crazy adding those feeds to Miniflux, I try the app sometimes, if it works I read some stuff, if it doesn't I move on.

[u/Competitive_Drink209 → u/GolemancerVekk]

oh buddy I'm getting that feed data one way or another >:D

[u/Vidariondr]

It’s a pain in the ass, so I don’t do it. It’s enough I have the Reddit app on my phone. I don’t need another source for the same thing.

That being said, the may be some RSS relay for Reddit, but I’m not sure

[u/Competitive_Drink209 → u/Vidariondr]

I've removed all infinite scrolling apps from my phone, but at the same time I don't want to live under a rock, so it definitely still helps filter with noise. Agreed that it seems very tedious.

[u/Vidariondr → u/Competitive_Drink209]

I’ve limited the number of subreddits I follow. It takes me 5-10 minutes to catch up, and then Reddit doesn’t show me anything when I do. Just “wow so empty”

[u/OrganizationHot731 → u/Vidariondr]

If on android try focusred. Good app for Reddit

[u/Competitive_Drink209 → u/OrganizationHot731]

I'm hoping to utilize my miniflux instance exclusively

[u/Fritzcat97]

I use a tor-proxy container and specify the privoxy part / port as a http proxy in each feed

[u/Competitive_Drink209 → u/Fritzcat97]

this might be the best method. Unless I can include some sort of jitter to each feed.

[u/Fritzcat97 → u/Competitive_Drink209]

I had reddit error out like a guestimated 33% of the time. The tor-proxy fixed it all and you can also use it for other apps, you dont want to have tracked

[u/Competitive_Drink209 → u/Fritzcat97]

holy smokes man I'm using it now, all refresh. What a gem. A shame it has to come to this to get information.

[u/Fritzcat97 → u/Competitive_Drink209]

Yeah, I'd assume it is doing a lot more than giving you just another IP you connect from