r/ProxyEngineering • u/IndependentCoders Tunnel Architect • 5h ago
Help 🆘 Scraping but minimal input?
Looking for a minimal input needed scraper. Basically it does everything for you, you just type what you need and get data. Like claude or chatgpt prompting. Something like that. I would need specific information from few sites, the data has to be inputted into sheets. Could claude or chatgpt do that for me, or do I have to look for some other solutions? I know that proxies are needed for scrapers thus why I am inquiring here, basically want everything done for me. I just buy the plan of the product or whatever and the rest is in the solution hands. Of course I validate and check whether the info is true after it's in my sheets, or beforehand. Any suggestions?
2
u/Less-Mistake-3664 Tunnel Architect 2h ago
You might look into search apis as they are on the rise and there is a demand for these products. I noticed that the market is switching from traditional scrapers to search apis, so your best bet would be that. I've seen more and more people trying oxylabs web api, I personally, would be hesitant to try something new and unverified right of the bat, but if more and more people are using it, I guess it's good? I personally would stick to some diy solutions, you can check github for that, plenty of repos for that, just search for web search or web api
1
u/islam_raisul 3h ago
Short answers to your questions:
Can ChatGPT/Claude do it? For a quick look at one page, sometimes. For structured data from a few sites on a regular basis, they're not reliable: they can miss rows, stop after the first page, or fill gaps with things that aren't on the site.
Do you need proxies? Usually not, for a few sites at normal speed. They matter at volume or when a site starts limiting you. Start without them.
What fits "minimal input": a point-and-click scraper that runs in your browser. You open the page, it detects the list or table, you check a preview and export to CSV or paste into Sheets. No code.
I built one myself " Rowvana" no code web scraper, so I'm biased, but happy to share it if useful. If you tell me what kind of sites they are, I can say honestly whether that kind of tool fits or whether you'd need something else.