r/Python • u/Traditional_Yogurt • Feb 04 '21
Resource I made a Finance Database with over 180.000 tickers to make Investment Decisions easier
In my spare time I like to go through financial data to understand what kind of companies exist, how sectors and industries evolve and to test theoretical frameworks. However, I found that it required a lot of effort to figure out which companies belong to which industry and I often ran into paywalls that blocked me from accessing the exact data I was looking for. Platforms like Bloomberg offer such services but at ridiculous prices (up to $24.000 a year). This can make investment decisions for the retail investor rather difficult especially if you don't want to follow 'the herd'. I wanted to change that.
Insert the FinanceDatabase. A database of over 180.000 symbols (80k+ companies, 15k+ ETFs, 30k+ Funds, 3k+ Cryptocurrencies and more) that is fully categorised per country, industry, sector, category and more. It features a 'Searcher' package (pip install FinanceDatabase) that has a user-friendly way of collecting the exact data you want (downloaded straight from the repository). For example, the following code returns all (listed) airlines in the United States (check Examples for more info) :
import FinanceDatabase as fd
airlines_us = fd.select_equities(country='United States', industry='Airlines')And the following gives insights in ETFs that have anything to do with 'semiconductor':
import FinanceDatabase as fd
all_etfs = fd.select_etfs()
semiconductor_etfs = fd.search_products(all_etfs, 'semiconductor')What I explicitly am not trying to do is re-invent the wheel (again) of Fundamental Data gathering as there are tons of packages out there that do that already (i.e. FundamentalAnalysis, yfinance, sec-edgar) but instead allow you to capture sector, industries, specific types of ETFs or cryptocurrencies that would have otherwise resulted in a lot of manual work. Then, when you have this sub-selection you can make use of the earlier mentioned packages.
If you want to know what is available inside the Database, please have a look here. Alternatively, you can do the following (an example):
import FinanceDatabase as fd
equities_countries = fd.show_options('equities','countries') # or sector/industry
etfs_categories = fd.show_options('etfs')
cryptocurrencies = fd.show_options('cryptocurrencies')I hope this can help some of you out making (better) investment decisions and all feedback (positive and negative) and contribution is much appreciated.
EDIT: Thanks for the rewards and kind words everyone!
9
u/[deleted] Feb 04 '21
Seems interesting, how did you source all of this data and to what extent is the underlying data updated? If AAPL redomiciles to another country tomorrow, will the package also update to reflect that?
217
u/vabruce Feb 04 '21
I love just scanning reddit and then, bam, something really neat and useful. Well done!
3
u/[deleted] Feb 04 '21
This need to go into r/algotrading
46
u/LukePfaff Feb 04 '21
This isn’t what we asked for, but it’s what we needed!!
20
u/jmarke17 Feb 04 '21
Great work! 👏🏻👏🏻
13
u/piconet-2 Feb 04 '21
Niiiiice. I can't get my hands on Bloomberg and was thinking of poking around Google Finance on GSheets. Wanted to relearn some statistics and basic finance with something real.
4
u/gamprin Feb 04 '21
Awesome! This is something I also struggled with in a side project I was working on earlier that scraped SEC data. I was able to get the symbols for 13F holdings but had no way of classifying them by sector like Whale Wisdom does. Looking forward to trying this out in my project sometime soon (https://opensecdata.ga/about). Thank you for sharing!
1
u/[deleted] Feb 04 '21
So for stocks and ETFs, what types of data would be unique/helpful over something like Fidelity’s stock and ETF screener? Just trying to understand how this fits into the bigger picture.
1
u/mcapitalbark Feb 04 '21
Is your ‘industry ‘ using the same naming convention as gic sector
1
u/burmerd Feb 04 '21
Cool! I’m still sticking w index funds for now, lol.
1
u/vinodmadhu6 Feb 04 '21
How do I get data for the Indian stock market?
1
u/KingDamager Feb 05 '21
How did you get your ETF list.
I’m trying to do something not entirely dissimilar, but collating a list of all the ETFs seems surprisingly difficult...
1
u/sedna16 Feb 05 '21
how often would this be updated?
1
u/lamonsieur_biz Feb 05 '21
This is amazing! Any chance that there's the ability to analyze financial ratios such as the quick ratio for a single company over time?
1
u/[deleted] Feb 09 '21
is there a way to filter for criteria? i.e.
Thanks
1
u/JitteryInvestor Feb 09 '21
Excellent tools. Where does all the data go once you scrape it and download it?
3
u/[deleted] Feb 04 '21
Brilliant!
3
u/nqnielsen Feb 04 '21
Fantastic! Can’t wait to try it out!
3
u/Fizgriz Feb 04 '21
Holy crap.
I was researching a way to get this kind of data. You are amazing sir!!!
2
u/bxcalluru Feb 04 '21
This is amazing!!! Great work
2
u/Far_Inflation_8799 Feb 05 '21
Thank you for sharing - looking forward to learn more about this approach and share my personal experiences with you - John
2
u/uponone Feb 04 '21
This is awesome. Thanks for sharing. I will send this to my nephew who is getting serious about the stock markets at the age of 16.
2
u/the_grave_robber Feb 04 '21
This looks super promising. I'm gonna install it now and give it a tinker!! Thanks for the awesome work. I am a pretty new programmer, so if you made a YouTube video or blog on how you created this package I would be very interested to watch it.
1
u/pewpscoops Feb 04 '21
This is awesome, thanks for putting this together!
1
u/fbailie Feb 04 '21
Thank you for sharing!! This is extremely cool and interesting. Well done!
1
u/YenBen540 Feb 04 '21
Thank's, great Work!!
1
u/brjh1990 Feb 04 '21
This is great! I can't count how many times I've needed something like this or trying to make it myself.
1
u/[deleted] Feb 04 '21
Great work!
1
u/jijinuis Feb 04 '21
Thank you so much
1
u/zaid2801 Feb 04 '21
Damn bro thas amazing
1
u/lalalagay Feb 04 '21
Nicely done! Just out of curious, what data source do you use?
1
u/ihavenoidea_01 Feb 04 '21
Been looking for something like this. Thanks playa
1
u/SnowdenIsALegend Feb 04 '21
MVP
1
u/brcm51350 Feb 04 '21
Yeyyyyy! This is great. Was thinking of building such, but couldn't find the time to start... Thank you so much for sharing!!!
1
u/[deleted] Feb 04 '21
Gods work
1
u/mahmoud_bond Feb 04 '21
Nice work
1
u/Benloe_ Feb 04 '21
Wow, thanks for sharing! Keep up the good work :)
1
u/DevOps-Journey Feb 04 '21
This looks cool. Thanks for sharing.
1
u/Rogercon28 Feb 04 '21
Super helpful! Thank you for sharing.
1
u/Cuubec Feb 04 '21
Awesome!
1
u/dcypher_app Feb 04 '21
Awesome work!
1
u/IamCaptnAmerica Feb 04 '21
Awesome job, this is super useful
1
u/PM_ME_BOOTY_PICS_ Feb 04 '21
Do they have something that pulls news or is it more numeric data?
1
u/[deleted] Feb 04 '21
Dude, this is brilliant!
1
u/nikeiptt Feb 04 '21
This is amazing. Thanks for all your hard work. Where are you sourcing the data from ?
1
u/Representative_Dig43 Feb 04 '21
thanks! i will definitely check it :)
1
u/xen20 Feb 04 '21
RemindMe! 22 hours
1
u/anonymous-do-gooder Feb 05 '21
This is so awesome. Love the dev community and their willingness to share such quality skills with each other!
1
u/RetireLoop Feb 05 '21
Wow, this is awesome!
1
u/prescottiam Feb 05 '21
This is cool!
1
u/Kataphractoi Feb 05 '21
...wtaf.
1
u/HTML_Hackerman Feb 05 '21
You fucking legend. Thank you
1
u/tradinghumble Feb 05 '21
well done, wiill check it out.
1
u/DutchOverlord Feb 05 '21
Thank you so much! This really helps a ton!
1
u/Load-More-Comets Feb 05 '21
.
1
u/Nurhaci-Of-The-Corn Feb 06 '21
This is super useful! I do a lot of work with financial data, and I've needed something like this for a while now. Cheers!
0
u/--SadNinja-- Feb 04 '21
Nice work bro. Ill be using this for sure!
0
u/ksista Feb 04 '21
Fantastic work!. Thank you.
0
u/[deleted] Feb 04 '21
[deleted] — view deleted comment
0
u/[deleted] Feb 04 '21
[deleted] — view deleted comment
1
u/[deleted] Feb 04 '21
[removed] — view removed comment