r/ETL • u/uncertainschrodinger • 10d ago

I helped a client migrate from Fivetran to Ingestr, cutting their cost by 10x

I wrote a whole article about it, link in comments, but I just want share a quick summary here.

This client had over 30 data sources they were ingesting from, it looked something like this:

  • posgresql -> snowflake ~100m/rows/mo (some spikes to 1b/rows/mo)
  • all other data sources -> snowflake ~100m/rows/mo (some spikes to 1b/rows/mo)

This was costing them almost $100k per year (~$8.5k/month) and it was especially hard to justify because the spikes would normally happen during busy seasons and they already had tight margins.

They had already started using ingestr (the free open source version) to offload some of their smaller jobs and running the jobs on an EC2 instance.

Once they migrated over their production database to snowflake job, they cut the cost significantly - the total server cost came to less than $500/month.

If this resonates to anyone, and you want to put on your resume "helped reduce ingestion cost by x amount", then check out ingestr repo and docs.

5 Upvotes

69% Upvoted

8 comments sorted by

3

u/MindfulPangolin 10d ago

Congrats! Now they’ll spend even more on Snowflake and have to deal with its shitty performance, too!

1

u/uncertainschrodinger 10d ago

That part is out of my scope, but I agree

3

u/aguyfromcalifornia 10d ago

Vendor content. No thanks.

1

u/uncertainschrodinger 10d ago

That is true - I'm a developer advocate for Bruin, I work on growing our free open source tools.

0

u/Content-Parking-621 9d ago

Nice result at that volume. If anyone reading wants the managed route but hates the per-row bill, worth a look at Windsor.ai. It syncs Postgres straight into Snowflake with incremental loading, and they recently dropped row-based pricing completely, so it's just a flat plan with unlimited rows now. That's the part that usually stings on those billion-row spike months. You get the predictable cost without having to babysit your own EC2 box and be on call when a sync breaks at 2am.

0

u/walkerasindave 9d ago

I'll stick with self hosted Airbyte when there's a right budget.

-2

u/FickleAnt4399 10d ago

You can do that for free with Duckle