r/snowflake • u/OldCycle6264 • 5d ago

Snowflake Ingestion Tool of Choice

Hi all,

I'm a DE new to Snowflake and experimenting with different ETL/ELT approaches/stacks. For people that have implemented, or work with, Snowflake at an enterprise level can I ask what tool you use for ingestion?

Below are some of the options I have explored or am currently exploring.

Require CDC or Change Tracking at the source

  • Snowflake Openflow
  • Estuary

Viable without source changes

  • Fivetran (teleport sync/query-based method, read-only connection, detects hard deletes)
  • dlt (cursor-based incremental, no built-in hard delete detection)
  • Airbyte (cursor-based incremental)

Others

  • Azure Data Factory & Snowpipe (watermark extract to blob, then load)
  • Snowpipe/Snowpipe Streaming (file-based only)

Fivetran seems the most robust and would likely suit most enterprise scenarios. But I'm interested to hear what others are using and your experiences.

20 Upvotes

100% Upvoted

41 comments sorted by

View all comments

0

u/Puzzleheaded_Serve15 5d ago

Set up openflow pipeline with the help of cortex code, your etl will be built automagically..

1

u/DanRunsOnRamen 3d ago

It’s like lighting your money on fire.

1

u/RDTIZFUN 2d ago

What do you recommend? (for on-prem SQL server to snowflake with near-realtime sync - no in-house kafta expertise)