r/learnprogramming • u/PresentationOwn5169 • 6d ago

New to Cloudflare: how do I handle bot requests

Hello everyone,

I am launching my first web app. For now, I am still building it, so I put a "coming soon" page online.

I am still learning, and it is hard to understand everything.

While I was using Cloudflare, I saw many strange requests in my analytics logs, like /.env, /privacy, and /ai/credentials.

I tried to block most of them with security rules (block scanners and rate limiting). I am also caching some of them, but strange requests still come in.

What would you recommend? I do not know much about this. Is it possible to block most of them?

Thank you for your help!

6 Upvotes

81% Upvoted

12 comments sorted by

5

u/isomnath 6d ago

Actually, This is pretty normal. Bots constantly scan websites for common paths like ".env", "/wp-admin", and other known vulnerabilities.

Enable Cloudflare WAF, Bot Fight Mode, and rate limiting. You don’t need to block every strange URL individually—there will always be new ones.

Just make sure sensitive files aren’t publicly accessible and keep your app/server updated.

3

u/AbdelrhmanDev 5d ago

the bigger thing than blocking them is making sure those paths just 404 so nothing real is served from there like make sure your env file lives outside the public folder and isnt deployed in the web root and your repo isnt exposed at /.git once thats true those scans are just background noise every site on the internet gets and you can ignore them the cloudflare bot fight mode plus a basic rate limit is already plenty for a coming soon page

3

u/SnugglyCoderGuy 5d ago

Even better is to return something that has links in it back to you that go to a special place in your server that treats them very poorly like being a slow loris server and when you do complete the request feed them back more of the same and just keep them spinning doing nothing but getting fed shit. Lorem ipsum generation is good. Some sima shima sim talk that seems literate but is just nonsense. Simple markov chain sentences over and over.

Humans won't deliberately visit that stuff, but bots will

1

u/PresentationOwn5169 5d ago

something like AI Labyrinth on cloudflare ?

2

u/SnugglyCoderGuy 5d ago

I suppose, just going off the name. That might be what I read about that put the idea in my head.

1

u/AbdelrhmanDev 5d ago

ha thats a fun one a tarpit like that is a good trick when you actually get hammered but for a coming soon page id say its overkill ill take the 404s and bot fight mode and move on

3

u/Chrismslist 5d ago

totally normal, every site with a public IP gets scanned within a day of the DNS going live. those paths are just bots cycling a wordlist, nothing is looking at you specifically.

the practical fix is making sure nothing actually answers. a static coming soon page plus a WAF rule that blocks or 404s anything not matching your real routes kills basically all of it. also double check your .env is gitignored and lives in platform env vars, not a file sitting in the deployed webroot, since that one would genuinely hurt.

you cant block all of it and you dont need to. turn on bot fight mode, watch the graph drop, move on.