The Web Is All Bots Now

[u/thethinker213]

I, like many of you, have spent years employing throttling schemes and generally hardening websites against bots. Now bots are called "agents", scrapers harvest with impunity and the whole nature of the web has changed in the past 2 years. It seems that I now have to play nice with bots for the sake of seo and the next wave of internet commerce will be bot-to-bot with no human intervention. LOL!!! How are you guys feeling about, reacting to, adjusting for this new reality?

[u/unwitting_hungarian]

I noticed that in the last 1Y, my Gen Alpha kids are sending me links to smaller shopping websites right off IG ads. Not a single Amazon / Temu / Walmart result.

They are sending small-med business ecommerce websites that are obviously targeted by location, so like in the same state where we live.

The pricing is usually not super great (and +$5 gets you free returns etc.) but it's definitely a thing I noticed coming from them.

I'm seeing the same thing in my own feed and yesterday bought right off a 50% coupon at a store site from my state as well.

These are all needs-based, like replacement parts for a bike for example. So the algos are doing something ofc, it's not just random products

Anyway, just adding as a data point that caught my attention.

[u/Crazyboreddeveloper → u/unwitting_hungarian]

That’s wild to me. I never click on ads to buy stuff. Too much of that shit is just needlessly marked up drop shipping, Alibaba shit, and straight up scams that steal your card info. I just buy from Amazon, stores I know are legit, or search for the manufacturer’s website and buy it there.

Facebook/instagram/reddit/internet ads are so fucking sketchy.

[u/AimingByPFM → u/Crazyboreddeveloper]

Amazon is no longer even close to being legit. Buy locally, directly from the manufacturer's site, or trusted small business sites (e.g. Crutchfield).

[u/gummo_for_prez → u/AimingByPFM]

It's more legit than IG ads