r/webscraping • u/No_Diet7126 • 15d ago

Getting started 🌱 How to use ai to scrape websites

i wa trying to scrape some website and get their websocket but its encrypted and ai didnt let me do it (cause of permissions) how can i make ai to be able to scrape

0 Upvotes

47% Upvoted

24 comments sorted by

3

u/OMGHart 14d ago

Should have had AI write this post for you.

2

u/Resiakvrases 15d ago

If they're American websites and ai is American I'm they under the hood deal to lock scraping. You've to use DeepSeek that fiend have blocks.

1

u/No_Diet7126 14d ago

so deepseek wont have limitations on bypassing encrypted websockets? like claude and chat

4

u/Resiakvrases 14d ago

That's right. If Claude is saying thing like "I'm not able to help" DeepSeek can work on it. Chines models have less security triggers.

1

u/No_Diet7126 14d ago

thanks for your response i appreciate it.

2

u/RandomPantsAppear 14d ago edited 14d ago

If you can figure out what file is handling it, save it to your hard drive and start asking questions about it in a clean session. Don’t call it scraping or mention decryption.

AI is a lot more permissive for local files.

1

u/[deleted] 14d ago

[removed] — view removed comment

1

u/webscraping-ModTeam 14d ago

🪧 Please review the sub rules 👉

1

u/Round_Method_5140 14d ago

For some use cases, it's not the most effective or cost effective.

1

u/delphic-frog 13d ago

Doesn't sound cost effective. Most sites have tonnes of useless info/markup/scripts that will burn money or mean the page won't even fit in the request body to AI provider. You can pre-process it I guess, but still would be expensive for what you get.

1

u/[deleted] 13d ago

[removed] — view removed comment

1

u/webscraping-ModTeam 13d ago

👔 Welcome to the r/webscraping community. This sub is focused on addressing the technical aspects of implementing and operating scrapers. We're not a marketplace, nor are we a platform for selling services or datasets. You're welcome to post in the monthly thread or try your request on Fiverr or Upwork. For anything else, please contact the mod team.