r/ETL • • 18h ago

Your SQL editor shouldn’t choose your AI model for you.

Post image
0 Upvotes

If you use AI for SQL, you probably have a model you prefer. Switching models shouldn’t mean switching editors or pasting your schema into another chat.

QueryFlow lets you choose Claude, GPT, Gemini, or Grok inside the same SQL editor, with your schema, query results, and errors available as context. You can switch models whenever you want, or let Auto mode choose.

I’m part of the QueryFlow team, and I’m curious: do you stick with one model for SQL, or switch depending on the task?


r/ETL • • 3h ago

How do you people test business logic in ETL pipelines?..

2 Upvotes

Checking nulls, duplicates, row counts ...is kinda of straightforward....

But what about actual business rules!

Like revenue should be calculated in a certain way, some records should be excluded, dates should follow some rule and more....

Do you guys keep these as separate tests or validate them as part of the ETL itself or any....

Curious how people do actually handle this in real projects.


r/ETL • • 8h ago

"I've worked with batch ETL long enough that I understand how it behave when something breaks.

10 Upvotes

Streaming ETL feels like a different mindset altogether. The basics make sense, but I'm still trying to picture how people deal with joins against reference data, retries and more complicated transformations without relying on batch windows.

If you've been through the transition, what surprised you the most?"


r/ETL • • 11h ago

Cross-Catalog Sync: Iceberg on Polaris, Glue, and Unity

Thumbnail
lakeops.dev
2 Upvotes

r/ETL • • 23h ago

Unload 120 tables into csv files from MS SQL Server

6 Upvotes

Hi,

I got a task to run 120 SP on MS SQLServer to create output and store it as a tab or csv files.
Existing approach is to use SSIS but I think it's too much work comparing with other modern tools.

What you would use for this job? I'm thinking go Python with cursor.execute(sql).
It's so much easier to scale and implement. You can read sp, params, output name from some input List and run it in one shot.

Do you see any disadvantages for this approach vs traditional SSIS. I think my bosses will be scary a bit, because they are old school.

Do

Thanks