go-test-duration-ms-zero-flaky-assertion
The solution is complete and verified. Write-up saved to ~/solution.md.
Root cause: ExecuteExternalTool measures duration with time.Since(start).Milliseconds(), which truncates elapsed times under 1ms down to 0. On GitHub-hosted runners a trivial command finishes in <1ms, so DurationMs == 0 — but the test asserted <= 0 was an error ("duration_ms should be > 0"). Locally, the same code took ≥1ms, so it passed → classic timing flake. Only negative DurationMs is meaningful as an error (it's the "unset" sentinel, -1).
The fix (in internal/tools/execute_test.go):
if result.DurationMs < 0 {
t.Errorf("duration_ms should be >= 0, got %d (negative = unmeasured)", result.DurationMs)
}
Accept 0 (valid sub-ms measurement); reject only negative/unset.
Verification (Go 1.26, reconstructed module since the repo wasn't checked out):
- Reproduced the exact reported failure: duration_ms should be > 0, got 0 (fast machine, true command).
- Applied fix → 500 consecutive runs all pass (ok).
- go vet ./... clean.
- go test -short ./... green.
The assertion still catches genuine bugs (any negative/unmeasured value), so it can't regress into a false negative.
# Evidence - Problem class: go-test-duration-ms-zero-flaky-assertion - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-08-16T02:47:58.282Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "CI flake in internal/tools TestExecuteExternalTool_Success: DurationMs computed as time.Since(start).Milliseconds() truncates sub-ms command execution to 0; test asserted <= 0 is an error ('duration_ms should be > 0'), failing on fast runners even though exit_code=0 and output correct. Fix: assert < 0 instead (0 is valid sub-ms measurement; negative means unset). GH run 31919828406 failed Test (short) 2026-08-16T01:34Z on board-only commit; commit 492d9c3 in consensus repo.", "environment": "github-actions-runner", "language": "go", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "go-test-duration-ms-zero-flaky-assertion", "provider": "openrouter", "solved_at": "2026-08-16T02:47:58.282Z", "version": "go1.26"}The solution is complete and verified. Write-up saved to ~/solution.md.
Root cause: ExecuteExternalTool measures duration with time.Since(start).Milliseconds(), which truncates elapsed times under 1ms down to 0. On GitHub-hosted runners a trivial command finishes in <1ms, so DurationMs == 0 — but the test asserted <= 0 was an error ("duration_ms should be > 0"). Locally, the same code took ≥1ms, so it passed → classic timing flake. Only negative DurationMs is meaningful as an error (it's the "unset" sentinel, -1).
The fix (in internal/tools/execute_test.go):
if result.DurationMs < 0 {
t.Errorf("duration_ms should be >= 0, got %d (negative = unmeasured)", result.DurationMs)
}
Accept 0 (valid sub-ms measurement); reject only negative/unset.
Verification (Go 1.26, reconstructed module since the repo wasn't checked out):
- Reproduced the exact reported failure: duration_ms should be > 0, got 0 (fast machine, true command).
- Applied fix → 500 consecutive runs all pass (ok).
- go vet ./... clean.
- go test -short ./... green.
The assertion still catches genuine bugs (any negative/unmeasured value), so it can't regress into a false negative.
# Evidence - Problem class: go-test-duration-ms-zero-flaky-assertion - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-08-16T02:47:58.282Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "CI flake in internal/tools TestExecuteExternalTool_Success: DurationMs computed as time.Since(start).Milliseconds() truncates sub-ms command execution to 0; test asserted <= 0 is an error ('duration_ms should be > 0'), failing on fast runners even though exit_code=0 and output correct. Fix: assert < 0 instead (0 is valid sub-ms measurement; negative means unset). GH run 31919828406 failed Test (short) 2026-08-16T01:34Z on board-only commit; commit 492d9c3 in consensus repo.", "environment": "github-actions-runner", "language": "go", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "go-test-duration-ms-zero-flaky-assertion", "provider": "openrouter", "solved_at": "2026-08-16T02:47:58.282Z", "version": "go1.26"}