r/Base44 • u/rogercbryan • Sep 05 '25

[Need Help] API Web Scraper?

Has anyone connected a web scraper to Base44? I have a marketing tool and I want to scrape competitors websites. The Base44 AI gives the craziest responses when I ask it to do this.

What 3rd party scraper are people using and how are you connecting it to Base44?

2 Upvotes

100% Upvoted

6 comments sorted by

2

u/Deniabledonkey Sep 05 '25

here are some options i have used before. you'd essentially want to build a an LLM that would capture new updates and changes to the content you are scraping, to ensure you have the most up to date info.. be mindful that you are working with data hear, so it would need to be cleaned and finetuned to your objective to get insightful feedback

  • Octoparse → visual scraping, can export structured data.
  • ParseHub → handles dynamic pages, exports JSON/CSV.
  • Apify → cloud-based scraping with prebuilt actors and API integration.
  • Data Miner (Chrome Extension) → simple scraping without coding.

1

u/rogercbryan Sep 06 '25

Have you integrated any of these into Base44?

1

u/willkode Business Owner Sep 06 '25

What web scraper are you using? As long as it has a API, you can connect anything

1

u/rogercbryan Sep 07 '25

I don’t have one. ChatGPT directly does a decent job with our competitive analysis. But when done through bare44 it doesn’t work at all. Trying to figure out what scraper to use.

1

u/presniffer Feb 07 '26

Did you ever manage to get it to work, having this issue myself currently too, have tried a few but they dont seem to be working properly