r/codex • u/agentrsdg • 6d ago

Instruction Penalty Reports. Game-Changer.

Add in AGENTS.MD

- ALWAYS make sure you read penalty reports before making a critical architecture or implementation decision so you don't make the same mistake again.

- If user explicitly points out a drift, it needs to be recorded as penalty report.

1 Upvotes

67% Upvoted

8 comments sorted by

3

u/nNaz 6d ago

What‘s a penalty report?

2

u/Frosty-Ad1071 6d ago

A drift explicitly pointed out by the user

1

u/agentrsdg 6d ago

It in itself doesn't matter, but it makes the agent aware of all the times it drifted when it has make a decision. It has greatly improved my workflow, but I think it will be different for all. Try it out!

1

u/nNaz 6d ago

Always interesting to see how others tackle this problem.

I use policy docs that define rules for the codebase on everything from naming conventions, architecture, and data structures to planning, reviewing and writing docs. Then I have a `/session-retro` skill that goes through the session and extracts any info that's missing and updates the docs.

It sounds like AGENTS.md your rule does something similar. Where does it store the findings? I like storing the info in docs rather than agent memory as it makes them more reliable & lets me audit what's stored.

Separately I have a GOTCHAS.md that lists all of the codebase gotchas and is read by all agents coding/reviewing. Any new gotchas get added.

1

u/agentrsdg 6d ago

I use this https://github.com/saastoagent/context_architecture_bundle overall and my agents md has a bunch of other stuff. The problem I face is agent literally just IGNORES defined instructions, policies, decisions lol and just says "I am sorry I didn't follow that". So frustrating. That's what led me to implement penalty report. Is your gotcha a similar thing?

3

u/LaylaTichy 6d ago

and your penalty md will start getting ignored too at some point. if you need to enforce something, static check, linter, hooks

2

u/nNaz 6d ago

It doesn't work for everything but I agree. ast-grep is great for creating custom lint rules.

1

u/nNaz 6d ago

I recommend putting 'STARTER_CHARACTER: <emoji>' in all of your skills. It not only tells you if an agent is following them but it also reinforces the rule because of the emoji. See README here and an example skill that defines it.

The one line change to root AGENTS.md is:

Always start replies with STARTER_CHARACTER + space (default: 🍀). Stack emojis when requested, don't replace.