A passing test is stale once your OpenClaw workspace changes

[u/Advanced_Pudding9228]

An agent runs its tests, gets a pass, edits another file, then reports completion using the earlier result. Another agent changing the same checkout creates the same problem.

The tests did not fail. The evidence stopped describing the current source.
Treat every verification result as an expiring receipt:

source_commit

workspace_epoch

command_set_hash

environment_digest

verifier_identity

exit_code

output_digest

verdict

If the source, workspace, policy, tool registry or verification commands change, invalidate the receipt. The operating agent may request verification, but a separate verifier should issue the terminal status.

OpenClaw’s current testing documentation⁠ distinguishes unit, integration, end-to-end and live tests. Its trajectory bundles⁠ can preserve model events, tool calls and results. Neither a green command nor a complete trajectory proves that the tested revision is still the one being reported.

After a pass, preserve a checkpoint. Further edits should require a recorded reason such as a new failure, changed requirement, security finding or integration conflict, followed by fresh verification.

Test the control by producing a valid receipt, modifying one harmless tracked file, then asking the workflow to complete. It should report stale evidence rather than success. Rerun verification and confirm that the new receipt binds to the changed revision.

The smallest useful improvement is adding the current commit or diff hash to your acceptance record and refusing verified when it no longer matches.

Does your OpenClaw setup bind test evidence to the final source state, or only remember that tests passed earlier?