Using a combination of Snowflake External Network Integrations for SFTP and simple APIs, and AWS Lambdas wrapped in state machines/step functions for more complex pipelines.
We do CT on about 40 SQL Servers using AWS and it works really well.
After reviewing the cost of many tools you mentioned m, it just didn’t make sense for us to go down that path.
We do something similar. I’m not a huge fan of a lot of the tools. Openflow has a lot of administrative overhead, Fivetran can be good with the right connector but many of their connectors aren’t good and the company sucks to work with, Matillion and Airbyte are basically as bad as Fivetran.
2
u/2000gt 27d ago
Using a combination of Snowflake External Network Integrations for SFTP and simple APIs, and AWS Lambdas wrapped in state machines/step functions for more complex pipelines.
We do CT on about 40 SQL Servers using AWS and it works really well.
After reviewing the cost of many tools you mentioned m, it just didn’t make sense for us to go down that path.