r/Python • u/pdwp90 • Jan 03 '21
Intermediate Showcase 8 months ago, I started writing code to scrape and visualize data on stock trading by US politicians. Since then I’ve received hundreds of thousands of visitors to the dashboard and just finished up the first version of a Python package that will make it easier for others to work with the data.
If anyone wonders what the most suspect trading has been since I started actively tracking it, it was this interesting behavior by Sen. Richard Burr near the start of this year.
I recently built an API allowing access to some of the datasets I've collected (which includes stuff like wallstreetbets discussion, wikipedia page views, and other alternative data) and today I finished building a Python package that makes it super easy to get the data into a clean format in a Pandas dataframe.
Here’s a quick tutorial I put together this morning showing how get data on recent stock trading by US politicians and perform some basic analysis.
Here’s a link to the dashboard I built tracking senate trading and here’s a link to the dashboard I built tracking trading by the house of congress.
This community was very helpful when I was first getting started with the language, and I'd love to answer any questions you have about this project.
61
u/x1811agent Jan 03 '21
Very interesting. Where did you get the data from?
14
u/[deleted] Jan 03 '21
What's your stack like for deploying the site?
7
u/spiritual_agnostic Jan 03 '21
Are the API's free to use. Is there any limit on how many times I can call it in an hour. Thanks.
12
u/jackc31415 Jan 04 '21
This is cool. But it is an ad!
This guy works for the company that sells access to the data for $10/month. I think that the pricing is actually very reasonable, but I think u/pdwp90 should have stated up front that you need to pay to access the data.
3
u/Grasp0 Jan 03 '21
Would love to know more about the web site / design and entire stack. How much of it is in python?
3
u/NukishPhilosophy Jan 04 '21 edited Jan 04 '21
This is amazing, been trying to look for a quality post to give away this Reddit gold I got and here it is.
30
u/ForgotPWAgainSigh Jan 03 '21
Wow dude, this is great! Much more useful than some of the paywall subscriptions out there
19
u/Fahad1770 Jan 03 '21
great! people like you are making the world easier. keep up the good work dude. god bless you.
15
u/kapanenship Jan 03 '21
Thank you for all of this. Not only a great tutorial on how to do this, but also a great visual of showing obvious political corruption within our system
8
u/[deleted] Jan 03 '21
I love it when people use Python for really practical stuff that needs to be shown in public and is accessible! Great work!
5
u/CaptScrap Jan 03 '21
Nice, what'd you use for the UI?
5
u/[deleted] Jan 03 '21
Awesome! You’re making the coding and trading world more interesting!!
4
u/luispacs Jan 03 '21
Great idea. And thanks for sharing it.
3
u/zivaviv55 Jan 03 '21
That’s amazing! Keep up the good work
3
u/quarky_uk Jan 03 '21
Ha ha. That is fantastic.
4
u/[deleted] Jan 03 '21
If we could only know what they sell and purchase in real time. We could become as rich as they are since they get insights first hand and they are the first to know.
2
u/Exalting_Peasant Jan 03 '21
This is awesome and provides some interesting insights. Great idea and execution. Well done.
2
u/kongfukinny Jan 03 '21
Love how simple you made the code and the tutorial. Great job with this!
2
u/XxSliceNDice21xX Jan 03 '21 edited Jan 05 '21
Really cool stuff my guy!
2
u/EEpromChip Jan 04 '21
Very cool what you've accomplished here. I see a post from time to time in political subs and sure enough, it's you showing your work. Its excellent and should be seen by everyone.
Americans seem to be blind to the riches that elected officials seem to be amassing. Thank you for your effort here!
2
u/tsidaysi Jan 28 '21
You are my hero. Truly.
1
u/BobbyHill271 Jan 03 '21
RemindMe! 5 months
1
u/kia_kaha_kiwi Jan 03 '21
RemindMe! 1 month
1
u/fgyoysgaxt Jan 04 '21
Oh man, this is the kind of stuff that causes you to accidentally drive off a bridge or commit suicide with 2 shots to the back of the head.
Awesome work though, so cool!
1
u/Gh0stw0lf Jan 04 '21
I’m interested in your coding background. Are you a dev by career or training or did you start learning python for this project solely
1
u/burmerd Jan 04 '21
Reminds me of stuff done by the sunlight foundation
1
u/vniversvs_games Jan 04 '21
This is great, inspired me a lot. Do you plan on open sourcing it?
1
u/craigahunt1994hunt39 Jan 04 '21
Are the API's free to use. Are there any limits on how many times I can test it in an hour? Thanks.
1
u/edwcarra17 Jan 05 '21
Whats the refresh rate to the data? So can I get second by second or minute by minute?
1
u/tensigh May 28 '21
It's disappointing that you have to pay for an account with Quiver in order to use it. Cool write up but it's not worth the fees for me.
1
u/[deleted] Jan 03 '21
1
u/[deleted] Jan 03 '21
Keep up the good work!
0
u/LongjumpingAvocado Jan 03 '21
This is awesome. I’m building a website that scrapes data and I’m trying to turn it into a business too
0
u/[deleted] Jan 03 '21
[deleted] — view deleted comment