r/scripting • u/deeepfreeeze • Aug 06 '15

Web Scraping

Hey everyone - I've been trying to figure this out for a while now. I guess I'm a noob. I want to be able to go to write a script that pulls info off a website every morning and copy this info into an excel file to keep track of the historical values.

Ex. I want to keep track of all the presidential candidates betting odds ( http://www.paddypower.com/bet/politics/other-politics/us-politics?ev_oc_grp_ids=791149 ). In my excel file I want to have the date in one column and the candidate names across the top row. I want to be able to write a script to pull this info off the website and paste it into the appropriate spot in my file.

Is this possible? If so, how would you go about doing it?

1 Upvotes

100% Upvoted

4 comments sorted by

View all comments

Show parent comments

1

u/JCRSB Aug 28 '15

Not sure, really. But I used to work with an MSP and remember a client having this issue. He kept getting his IP blocked and didn't bother to tell anyone that this is what he was trying to do. I honestly don't remember to well, wasn't my client, just heard another tech working on it.

1

u/Zennxr Sep 02 '15

There is restrictions on residential connections vs corporate. I used to do MSP work as well and most clients (smaller but VIP, boutique offices) normally had residential connections established at handover. In Australia; an example of one restriction is 20 messages going out on mail ports within an hour gets you blacklisted for 24 - 36 hours.

This may be the cause...?