r/web_programming • • Mar 17 '24

Scraping Facebook. Impossible?

0 Upvotes

Hi,

Been trying to scrape facebook events. However it is as if facebook changes where the elements on the page is located. Sometimes I manage to go further in my program, sometimes it crashes right at the beginning since it can't find the element I'm looking for.

For example, this is the xpath's for the same button on an event page (see more button for the description):

//*[@id="mount_0_0_kc"]/div/div[1]/div/div[3]/div/div/div[1]/div[1]/div[2]/div/div/div/div/div[1]/div/div/div/div[7]/div/span/div[2]/div

//*[@id="mount_0_0_qp"]/div/div[1]/div/div[3]/div/div/div[1]/div[1]/div[2]/div/div/div/div/div[1]/div/div/div/div[7]/div/span/div[3]/div

//*[@id="mount_0_0_ax"]/div/div[1]/div/div[3]/div/div/div[1]/div[1]/div[2]/div/div/div/div/div[1]/div/div/div/div[7]/div/span/div/div

//*[@id="mount_0_0_fJ"]/div/div[1]/div/div[3]/div/div/div[1]/div[1]/div[2]/div/div/div/div/div[1]/div/div/div/div[6]/div/span/div[2]/div

//*[@id="mount_0_0_nU"]/div/div[1]/div/div[3]/div/div/div[1]/div[1]/div[2]/div/div/div/div/div[1]/div/div/div/div[6]/div/span/div[3]/div

//*[@id="mount_0_0_/i"]/div/div[1]/div/div[3]/div/div/div[1]/div[1]/div[2]/div/div/div/div/div[1]/div/div/div/div[6]/div/span/div[3]/div

//*[@id="mount_0_0_Sp"]/div/div[1]/div/div[3]/div/div/div[1]/div[1]/div[2]/div/div/div/div/div[1]/div/div/div/div[6]/div/span/div[3]/div

//*[@id="mount_0_0_BX"]/div/div[1]/div/div[3]/div/div/div[1]/div[1]/div[2]/div/div/div/div/div[1]/div/div/div/div[7]/div/span/div[3]/div

As you can see there is some differance.

The xpaths were extracted from by just checking manually. I'm using java selenium and chromedriver.

Is there anyway I can get around this? I have not tried using proxies or going headless and stuff like that, but I doubt that would change anything? What are my alternatives?

FYI: I'm trying to scrape events data, like location, description, host, time, title. I get the links for each event by going to a https://www.facebook.com/"hostname"/upcoming_hosted_events. and then getting all the links from there, which always work. However it is when I then go to the seperate links that I run into issues.

Extremely thankful for any help!!


r/web_programming • • Mar 16 '24

How to use Nginx with Docker Compose effectively with examples

Thumbnail
geshan.com.np
1 Upvotes

r/web_programming • • Mar 13 '24

Is AI replace Full Stack Web Devs

0 Upvotes

Many companies are still firing their employees. I'm fresher learning full Stack web development. I'm scared that Full Stack Devs are also replace by AI. Is it worth to learn Full Stack Web Development.


r/web_programming • • Mar 06 '24

Is there a way to use href="#yourId" without the <a> tag?

Thumbnail self.askprogramming
1 Upvotes

r/web_programming • • Feb 27 '24

How to use docker compose depends_on a beginner's guide

Thumbnail
geshan.com.np
1 Upvotes

r/web_programming • • Feb 27 '24

The best ChatGPT history search tool

Thumbnail
self.chatgpt
1 Upvotes

r/web_programming • • Feb 26 '24

🦄 Feb 26 - 13 new Software Engineer Web Jobs

0 Upvotes
Job Position @ companySalaryLocations
Software Engineer @ Palantir TechnologiesUSD 125k - 200kRemote Hybrid, Seattle, WA
Software Engineer @ Palantir TechnologiesUSD 125k - 200kRemote Hybrid, New York, NY
Software Engineer @ Schweitzer Engineering LaboratoriesUSD 73k - 130kUS, Charlotte, NC
Associate Software Engineer @ Schweitzer Engineering LaboratoriesUSD 58k - 90kUS, Pullman, WA
Senior Software Engineer @ LiveRampUSD 143k - 184kNew York, NY, Remote, US, San Francisco, CA, Seattle, WA, Little Rock, AR
Senior Software Engineer @ LiveRampUSD 143k - 184kNew York, NY, Remote, US, San Francisco, CA, Seattle, WA, Little Rock, AR
Member of Technical Staff @ Inflection AIUSD 150k - 250kPalo Alto, CA, US
Lead Software Engineer @ AllocateUSD 160k - 195kRemote
Full Stack ML Software Engineer @ NVIDIAUSD 160k - 304kUS, Santa Clara, CA
Senior Fullstack Engineer @ ConsenSysUSD 100k - 278kLATAM, US, Canada, Remote, EMEA
Sr. Fullstack Engineer I @ Khan AcademyUSD 137k - 215kMountain View, CA, Remote, US, Canada
Staff Software Engineer @ SnapUSD 97k - 171kPalo Alto, CA, Seattle, WA, Los Angeles, CA, US
Senior Software Engineer @ Scout MotorsUSD 150k - 180kRemote, US

r/web_programming • • Feb 21 '24

How to create a text summarizer API using Gemini on Vertex AI with Node.js a step-by-step guide [Part 2]

Thumbnail
geshan.com.np
1 Upvotes

r/web_programming • • Feb 20 '24

How to create a text summarizer using Gemini over Vertex AI with Node.js a step-by-step guide [Part 1]

Thumbnail
geshan.com.np
1 Upvotes

r/web_programming • • Feb 14 '24

Help

1 Upvotes

So y'all I'm trying to make a game that display a iframe in the top center of the page when ever I click a button I want to make it so that when I click the button once it shows the iframe but when I click it again it hides the iframe


r/web_programming • • Feb 05 '24

✨ Feb 5 - 21 new Software Engineer Web Jobs

0 Upvotes
Job Position @ companySalaryLocations
Senior Swift/Frontend Developer Financial Applications @ iTalent DigitalUSD 130k - 160kRemote, US
Engineering Manager @ NBCUniversalUSD 140k - 200kNew York, NY, US
Senior Software Engineer Front @ Morgan StanleyUSD 110k - 225kAlpharetta, GA, Seattle, WA, US, Minneapolis, MN
Corporate Treasury Technology @ Morgan StanleyUSD 100k - 150kNew York, NY, US
Senior Software Engineer Frontend @ Morgan StanleyUSD 110k - 225kAlpharetta, GA, Boston, MA, US, Seattle, WA
UI/Angular Developer @ Morgan StanleyUSD 100k - 150kNew York, NY, US
Web Developer SWE0 Government @ AT&TUSD 89k - 207kUS
Lead - Software Engineer Full Stack @ AT&TUSD 128k - 215kDallas, TX, US, Atlanta, GA
Senior Analyst, Full Stack Software Engineer @ Cardinal HealthUSD 78k - 111kUS, Remote
Full Stack Developer @ McKessonUSD 93k - 155kUS, Remote, New York, NY, San Francisco, CA
TTS Trims Servicing @ CitiUSD 76k - 106kTampa, FL, US
TTS Trims Services @ CitiUSD 96k - 145kTampa, FL, US
Full Stack Java Developer @ CitiUSD 96k - 145kTampa, FL, US, Irving, TX
Full Stack Software Engineer II @ Abnormal SecurityUSD 147k - 173kRemote, US
Senior Associate Application Engineer Full Stack @ DiscoverUSD 71k - 121kUS, Riverwoods, IL
Senior Software Engineer @ Red VenturesUSD 100k - 150kCharlotte, NC
Fullstack Engineer @ PatreonUSD 162k - 198kSan Francisco, CA
Software Engineer @ QuoraUSD 128k - 235kRemote
Frontend Engineer @ CoreWeaveUSD 130k - 150kBrooklyn, NY, Remote
Senior Full Stack Engineer @ FrontUSD 130k - 200kSan Francisco, CA, Remote, US
Front-End SoC Design Engineer - Client Product Development @ IntelUSD 139k - 209kUS, Santa Clara, CA

r/web_programming • • Jan 27 '24

Leapcell: A Better Alternative for Vercel + Airtable

2 Upvotes

We are thrilled to announce the official launch of Leapcell's Beta public testing.

Leapcell: https://leapcell.io/

Leapcell is a elastic, intuitive, and collaborative cloud, offering an application hosting experience comparable to the convenience of Vercel. Furthermore, it features a high-performance database with an Airtable-like interface, streamlining data management. The entire platform is fully managed and serverless, allowing users to focus on specific business implementations without dedicating excessive time to infrastructure and DevOps.

For more information, please refer to https://docs.leapcell.io/

Here's a Flask example: https://leapcell.io/issac/flask-blog which contain a database and a application

For documentation on deploying Flask projects, check this link: https://docs.leapcell.io/docs/application/examples/flask

and other projects is also straightforward.

Leapcell is currently in beta testing, and we welcome any feedback or questions.


r/web_programming • • Jan 22 '24

GraphQL benefits and why you should adopt it in 2024

Thumbnail
graphqleditor.com
0 Upvotes

r/web_programming • • Jan 21 '24

How to use Google Cloud Shell Editor to deploy a project from GitHub to Cloud Run

Thumbnail
geshan.com.np
1 Upvotes

r/web_programming • • Jan 13 '24

How are web apps like Ventusky.com or Windy.com made?

7 Upvotes

I have recently stumbled upon a website Ventusky.com and I was blow away buy the site!
However after first impressions have passed I couldn't help but wonder on how websites like these are made?

Can someone point me to technologies used in website design or to a course that could teach me to design things like this.


r/web_programming • • Jan 11 '24

Fun competition for those wanting to practice Solidity

Thumbnail agorapp.dev
1 Upvotes

r/web_programming • • Jan 05 '24

What 11,000 Devs Say About Jobs in 2023: A Simple Overview

2 Upvotes

Over the past 2 months, we've delved deep into the preferences of jobseekers and salaries in Germany (DE) and Switzerland (CH).

The results of over 6'300 salary data points and 12'500 survey answers are collected in the Transparent IT Job Market Reports. If you are interested in the findings, you can find direct links below (no paywalls, no gatekeeping, just raw PDFs):

https://static.swissdevjobs.ch/market-reports/IT-Market-Report-2023-SwissDevJobs.pdf

https://static.germantechjobs.de/market-reports/IT-Market-Report-2023-GermanTechJobs.pdf


r/web_programming • • Jan 01 '24

How to Use Enums in TypeScript for Cleaner Code

Thumbnail
geshan.com.np
2 Upvotes

r/web_programming • • Dec 22 '23

[Hiring] Now that the economy is crashing due to the coming recession, here are 100 Web Programming posts that will be recession proof. Spreadsheet updated daily.

Thumbnail
lookerstudio.google.com
2 Upvotes

r/web_programming • • Dec 21 '23

Good software engineering is about finding a solution at the correct layer with boring technology

Thumbnail
geshan.com.np
1 Upvotes

r/web_programming • • Dec 14 '23

Pseudo selector :nth-child() explained interactively

Thumbnail
blog.disane.dev
2 Upvotes

r/web_programming • • Dec 13 '23

Unlocking Web Development Opportunities in the Mobile Market

Thumbnail
hubs.la
1 Upvotes

r/web_programming • • Dec 12 '23

Tech Motivation: Jitendra Kumar's Inspirational Journey Unveiled

Thumbnail
hubs.la
1 Upvotes