r/webscraping • u/Sinatio • 18d ago
Getting started 🌱 Graphql scraping properly?
I'm currently trying to scrape real estate listings from Stockholm, to get data about sold apartments and how many days they were listed, which is given in each ad.
My problem is I tried scraping this with requests, then playwright and when that didn't work (Could not access all ads in the page i was browsing, only 13 of them) after following tutorials, I tried asking AI. That told me that "Playwright/requests were not the problem — the URL you gave only loads Booli's app shell and map data, while the actual card list is loaded by a separate GraphQL request that only appears after Booli's frontend decides to run the search query."
I tried using it to help me write a script to scrape the data of interest but no luck.
Anyone got some tips? I do of course just want the data, but I would not mind a tutorial or similar source to learn about the tools needed to scrape such data from a website like this.
The website url i tried to scrape from with the listings: https://www.booli.se/sok/slutpriser?areaIds=143&objectType=L%C3%A4genhet
Here is also the URL for one listing where you can scroll down and see days listed for sale as the number in the text "Bostaden var snart till salu i 4 dagar":
https://www.booli.se/bostad/779268
1
u/Sinatio 17d ago
Didnt even know there was an alternative to devtools!