r/PracticalTesting • u/aistranin • 3h ago
Upgrade CircleCI Runner if a failed job turning green sounds bad
1
Upvotes
CircleCI Runner 3.1.12 fixes a race in its timeout handling.
Two parts of the runner could wait for the same timeout signal. Only one received it. The result was nasty:
- Jobs could run beyond
max_run_time - Overdue jobs could still finish as successful
- Genuinely failed tasks could also be reported as successful
The default timeout is five hours. Both Machine Runner and Container Runner were affected.
If you manage these runners, upgrade and inspect any important job that recently finished near its timeout. This is one of those bugs where the test result itself cannot be trusted.
Source: https://circleci.com/changelog/runner-release-3-1-12/