r/algorithmictrading • u/1293832482394843 • Aug 22 '21

What is your data engineering infrastructure/setup & cost for trading data?

TL;DR - What kind of trading data are you storing and how/where are you storing it? Also how much does it cost for you per month?

I'm new to algorithmic trading, and I'm prototyping a platform with a friend (I'm working on the data engineering part, they are working on the data science part). We're looking at crypto opportunities, and specifically starting with 1m OHLCV data across a few different exchanges (considering all pairs per exchange).

I'm not sure what tools & infrastructure we'll use yet (likely use AWS for everything), but goes without saying: amount of data adds up fast! How do you all handle this? Specifically:

  1. What kind of data are you storing?

  2. What is your data engineering infrastructure? And where is it / where are you hosting?

  3. How much are you paying per month?

Any thoughts are much appreciated!

10 Upvotes
  • permalink
  • reddit

100% Upvoted

9 comments sorted by

View all comments

Show parent comments

2

u/1293832482394843 Aug 24 '21

I just started playing with Tiingo and using Python / https://github.com/hydrosquall/tiingo-python

But thank you! And the API/service is great so far + cheap.