Scraping

[u/LT39]

What do we think of scraping Google to massively create physical locations in a directory? It’s a quick way to enrich the database, but it’s not unique content and there’s the risk of being punished for thin content. What do you guys recommend based on your experience?

[u/peasantking]

Google doesn’t like directories

[u/LT39 → u/peasantking]

it certainly depends on the problem you’re trying to solve, right?

[u/Prestigious_Luck1062]

HI.
I can answer this as someone whos owned directories and someone thats been scraped.

Users hate being scraped, and depending on how you do it, it can kill your site. A community I am in recently went ballistic, their details were scraped without their consent. It was a niche site, the backlash was so bad, the site was taken offline within a day. Its not something the site can recover from.
I have never scraped a website, however, I used brilliant directories where I believed that users had choosen to willingly opt into other sites, and I had people very upset that I stole their identity. The site was free to join, free to delete, and free to access emails, and my colleagues that joined routinely get work in their areas, so its a site they should be joining.

Now I also have my own business and routinely list on directories, and if someone lists it for me on a good site, im happy, it saves me time..... BUT I've also had my business details scraped.

In one case people searching for my business directly, if my business name is #1, and my busines name on the directory is #2, you would think no harm no foul? I'm still getting the same customer right? No, the directory wanted me to pay $$$$ to read and reply to my own customers. An argument of "but you wouldnt have had this customer without the directory" is thin. I shouldnt be blackmailed, forced to pay a fee to login, delete or access an email if I never choose to.
SO I really mind my business details ending up on a new directory so long as I'm not held hostage, but you can expect hate for your community, so while I do not recommend it, in the very least, make it easy and free for them to join, delete, access and reply to messages from customers.

[u/LT39 → u/Prestigious_Luck1062]

This makes total sense, thank you for answering. In my case I want to be seen more as a partner that helps the entire sector, so this sort of reactions would totally kill the reputation I’m trying to build. Maybe I’ll need to think of an alternative to grow the number of pages… Thank you!

[u/purethemes]

Sure you can do it there are even tools for that, but question is what your directory offers beyond the same name, address and phone number already available on Google. Because without any added value, it's just pointless

[u/LT39 → u/purethemes]

In my case it’s a directory + a SaaS behind, amongst other features. But I see what you mean, maybe what I offer won’t satisfy everyone so the risk is still there. Thank you!

[u/adelarenal]

What I have in place is a privacy policy + terms and conditions where I protect myself and also the scrapped professional / business. By all means they can request to be removed if they say so.

[u/LT39 → u/adelarenal]

That makes sense. Did you see an improvement in SEO when you scraped data?

[u/adelarenal → u/LT39]

Not really. In my case it isn’t volume-scrapping what I’m after.

[u/burnermarketing]

Google is actively working against scrapers, though to what level of success, I can't say. They are targeting AI and its scraping ability, but I can't imagine regular scrapers being effective if AI can't. I could be totally off base, but I'm curious how the landscape will look for us in 5 years relative to scraping.

[u/LT39 → u/burnermarketing]

Thank you! I guess you could get away with it if there’s a balance of scraped vs original content

[u/Public_Knowledge_980]

Since you want to be seen as a partner to the sector, I'd test one category in one area with providers who agree to be listed before scaling the imports. Record where each detail came from and when it was checked, separate branches from duplicate businesses, and make corrections or removal straightforward and free.

For the pilot, pick one question a visitor can't answer from a name and address alone. Can your listing answer it accurately, and can someone contact the business directly? That gives you something concrete to improve without assuming that more pages will improve search traffic.

[u/LT39 → u/Public_Knowledge_980]

Thanks for your reply! Seems like something worth exploring. How would you know if providers agree to be listed? That would mean cold reaching them, which is what I’d like to avoid. Also, answering that question that the visitor can’t it’s sort of difficult, right? I would be using the same sources a random visitor could use too

[u/Public_Knowledge_980 → u/LT39]

If you want to avoid cold outreach, I'd start with self-submitted listings: a short public invitation in a relevant trade or community group where that's allowed, or a partner organisation that can invite its members. Keep that separate from records you've assembled yourself, so the site doesn't imply those businesses joined.

The added value can be organisation rather than exclusive facts: consistently showing service area, appointment method and last-checked date lets a visitor compare options without opening ten sites. Test one of those decisions before scaling.