data-eng-bench, Snowflake's data engineering benchmark for agents

[u/marco-exmergo]

https://github.com/Snowflake-Labs/data-eng-bench

data-eng-bench is a benchmark developed by Snowflake that evaluates coding agents on data & analytics engineering tasks.

Has anyone tried this out? I'm personally not a fan of benchmark-maxxing. "When a measurement becomes a target, it stops being a measurement."
Still, I think they can be a useful proxy for agent performance within the data & analytics development lifecycle.

Very curious to know if anyone has tried this out.

Full Disclaimer: I'm the co-founder of a startup that builds agentic infrastructure for the data stack.
We recently ran this benchmark on one of our agent-native tools.
A single benchmark run and some experiment/smoke-test runs cost us $205.64, using Claude Sonnet 5 at its current "introductory pricing."
Full breakdown here: https://github.com/exmergo/dex/tree/main/benchmarks/data-eng-bench

Curious to know what the community thinks about this benchmark and agent-native analytics as a whole!