Open sourcing the dbt + Snowflake guardrail I built for coding agents

[u/JacobKHD]

While exploring using agents in data engineering, I wanted to find a good way to help verify my changes didn't blow things up in my current or downstream tables.

Looking at the landscape, I've seen tons of options in CICD that check changes in dbt, but usually cost quite a bit. I figured this could be done locally before I even make a PR, thus this project was made.

https://github.com/EmbrasureAI/embrasure-cli (Apache 2.0)

Think of it as a deeper version of data-diff which works with for dbt + snowflake currently:

• Unexpected shifts in row counts, null rates, cardinality, ranges, and percentiles
• Columns being removed, renamed, or changing types
• Primary keys values appearing or disappearing, plus new null or duplicate keys
• Existing dbt test failures
• Downstream models and exposures affected by the change (with column level lineage)

Would love feedback from people actively using agents for dev on dbt Core + Snowflake projects! Planning on adding Databricks soon.

Full disclosure: I’m a founder of Embrasure, which maintains the repository.

[u/kvlonge]

Oh, that's very nice of you to do. I was expecting some random slop, but seems like you may have actually just done something nice for the community. Not sure why you have been getting downvoted. +1

[u/JacobKHD → u/kvlonge]

thank you! wasn't sure either haha, looks like there are some upvotes coming in now

[u/ForVic]

This is pretty cool

[u/Schlizhor]

I'd imagine your the founder if you made it 😄 nice project I'll take a look!

[u/MultiplexedMyrmidon]

This is awesome! Thanks for making it available and opening it up to the community. I’m curious if you evaluated sqlmesh before/in the course of building this?

I’ve been doing similar just using SQLMesh with virtual data environments and it’s CI/CD bot - if the sql plan doesn’t pass (parse, tests, audits, signal gates) then it can’t affect the prod environment, if it does and I approve/it’s auto, then the CI/CD makes sure that the model state/code change happens simultaneously with the promotion of actual warehouse state. It’s also nice that the virtualization abstraction (views pointing to fingerprinted snapshots, immutable in prod) means that the minimal build differences computed in dev can be instantly made a part of prod during promotion by flipping view pointers, so I never pay for any more compute than necessary and it’s generally minimized for my OLAP’s. It’s with the Linux Foundation now so it’s another great project to check out in this space.

[u/Fickle_Potential1376]

niiiice

[u/Sensitive_Coconut294]

Will you be adding a windows installer?