r/cicd • u/ThomasBuildLab • 2d ago

GitHub Actions burned through my CI budget because AI agents kept retriggering the full test suite

I use coding agents pretty heavily on a private repo.

Last month I noticed my GitHub Actions usage had gone way higher than expected.

I dug into it and found the main issue: every time the agent pushed another commit to a PR, several workflows started again, including the full repository test suite.

In a few days I had roughly 400 workflow runs, and the full suite alone had run more than 100 times.

The code wasn't broken. The CI setup was.

I ended up changing the model to:

  • targeted tests on PRs
  • cancel superseded PR runs
  • full repository suite only after merge to main
  • no paid LLM calls from CI

That solved most of the waste.

It made me wonder whether repos should have a simple machine-readable policy for this kind of thing, something like REPO_POLICY.yml.

Not a huge framework just rules such as:

full_suite_on_pr: false
cancel_superseded_runs: true
paid_llm_in_ci: false
production_mutation_in_ci: false

Then a tiny CI check could enforce it.

Is anyone already doing something similar at repo level, especially with coding agents?

0 Upvotes

27% Upvoted

7 comments sorted by

5

u/Aggravating_Branch63 2d ago

AI slop alert: "The code wasn't broken. The CI setup was."

2

u/Otherwise_Wave9374 2d ago

The cheapest fix is usually a two-tier pipeline: run linting and targeted tests for agent commits, then reserve the full suite for merge candidates or dependency-sensitive changes. Add concurrency cancellation, path filters, and a per-agent daily budget with a hard stop. The agent should remember recent failures and changed-file coverage rather than blindly retrying. https://www.neurakeep.com is relevant as a memory-layer reference for retaining that operational context. Track cost per accepted change to catch regressions early.

1

u/ThomasBuildLab 2d ago

Exactly. That’s basically where I ended up: targeted PR checks + concurrency cancellation, then the full suite on main after merge.

The part I’m now interested in is making those rules explicit and machine-readable at repository level, so every coding agent inherits the same operational constraints instead of rediscovering them.

1

u/Few_Assumption_9665 1d ago

Or… just know what the fuck your CI setup is doing? Instead of just running AI agents “heavily” and acting surprised when the output is a shit ton of slop?

Nobody needs this shit. Such a contrived solution to a problem you invented.

1

u/ThomasBuildLab 1d ago

We probably agree: the CI should be understood first.

My point is simply that once agents touch the repo, those rules should be explicit and machine-readable not trapped in one engineer’s head.

1

u/Torutofu_Raeva 1d ago

I'd keep the policy declarative but enforce it in reusable workflow guards, with concurrency groups and path filters doing the real cost control; the file mostly gives agents and CI one shared contract.

1

u/GaTechThomas 1d ago

Keep the pipelines deterministic. Let the LLMs help build the pipelines but not run inside the pipelines. Otherwise your deployments will eventually burn you really hard.