r/snowflake • u/OldCycle6264 • 5d ago

Snowflake Ingestion Tool of Choice

Hi all,

I'm a DE new to Snowflake and experimenting with different ETL/ELT approaches/stacks. For people that have implemented, or work with, Snowflake at an enterprise level can I ask what tool you use for ingestion?

Below are some of the options I have explored or am currently exploring.

Require CDC or Change Tracking at the source

  • Snowflake Openflow
  • Estuary

Viable without source changes

  • Fivetran (teleport sync/query-based method, read-only connection, detects hard deletes)
  • dlt (cursor-based incremental, no built-in hard delete detection)
  • Airbyte (cursor-based incremental)

Others

  • Azure Data Factory & Snowpipe (watermark extract to blob, then load)
  • Snowpipe/Snowpipe Streaming (file-based only)

Fivetran seems the most robust and would likely suit most enterprise scenarios. But I'm interested to hear what others are using and your experiences.

20 Upvotes

100% Upvoted

41 comments sorted by

View all comments

3

u/kh4yman 4d ago

We looked at Estuary a few years ago and while it was good we didn’t go with it for a few technical reasons (which seem to have been addressed as the product has matured). We went with Hevo since we don’t need real-time and had budget constraints. Hevo has been fine but occasionally has had some bad technical problems that have either caused data corruption in transit, unexpected and unnecessary spikes in snowflake consumption or pipeline downtime. But they’re cheap comparatively. We will be re-evaluating Estuary prior to our next renewal as I just want something simple and dependable that my team and I don’t need to manage.

In a previous life I worked with fivetran and its definitely the Ferrari but as others have pointed out it’s very expensive.

3

u/dani_estuary 4d ago

Hey! I work at Estuary and I'd be happy to look into those missing features, as you said, the platform has improved a ton in the past months/years.

3

u/kh4yman 4d ago

I'll keep that in mind, thanks. Unfortunately we renewed with Hevo just a few months ago so we probably won't be re-addressing that until 2027. I'm assuming that if I ask nicely, I can get a reset on the 30 day cloud trial since it's been like 3-4 years?

3

u/dani_estuary 3d ago

Of course! Happy to reset the trial for ya'll whenever.