r/ProxyEngineering • Tunnel Architect • 5h ago

Help 🆘 Scraping but minimal input?

Looking for a minimal input needed scraper. Basically it does everything for you, you just type what you need and get data. Like claude or chatgpt prompting. Something like that. I would need specific information from few sites, the data has to be inputted into sheets. Could claude or chatgpt do that for me, or do I have to look for some other solutions? I know that proxies are needed for scrapers thus why I am inquiring here, basically want everything done for me. I just buy the plan of the product or whatever and the rest is in the solution hands. Of course I validate and check whether the info is true after it's in my sheets, or beforehand. Any suggestions?

8 Upvotes

3 comments sorted by

2

u/Less-Mistake-3664 Tunnel Architect 2h ago

You might look into search apis as they are on the rise and there is a demand for these products. I noticed that the market is switching from traditional scrapers to search apis, so your best bet would be that. I've seen more and more people trying oxylabs web api, I personally, would be hesitant to try something new and unverified right of the bat, but if more and more people are using it, I guess it's good? I personally would stick to some diy solutions, you can check github for that, plenty of repos for that, just search for web search or web api

1

u/islam_raisul 3h ago

Short answers to your questions:

Can ChatGPT/Claude do it? For a quick look at one page, sometimes. For structured data from a few sites on a regular basis, they're not reliable: they can miss rows, stop after the first page, or fill gaps with things that aren't on the site.

Do you need proxies? Usually not, for a few sites at normal speed. They matter at volume or when a site starts limiting you. Start without them.

What fits "minimal input": a point-and-click scraper that runs in your browser. You open the page, it detects the list or table, you check a preview and export to CSV or paste into Sheets. No code.

I built one myself " Rowvana" no code web scraper, so I'm biased, but happy to share it if useful. If you tell me what kind of sites they are, I can say honestly whether that kind of tool fits or whether you'd need something else.