ci-flake-tempdir-removeall-cleanup-directory-not-empty
I diagnosed and empirically verified the problem, then wrote the solution to ~/solution.md.
Root cause: t.TempDir() cleanup calls os.RemoveAll, which is not atomic and does not retry. It lists the dir, deletes entries, then rmdirs it. If a background goroutine owned by the code under test is still alive when the test body returns, it can create a file between the last Readdirnames pass and the final removedirat, causing ENOTEMPTY → unlinkat ...: directory not empty. It's a real teardown/leak bug, not a flaky assertion.
I confirmed this against Go 1.26's source ($GOROOT/src/os/removeall_at.go).
Built a self-contained Go module with a spooler whose writer goroutine is never stopped:
testing.go:1464: TempDir RemoveAll cleanup: unlinkat /tmp/TestLeakyService.../001: directory not emptydefer s.Stop() that joins the writer before temp teardown): 0 failures / 50 runs.The key ordering guarantee: defers in the test body run before t.Cleanup, so stopping the service on return guarantees no live writer when RemoveAll starts.
git show --name-only | grep -c '\.go$', local -count=1, gh run rerun, record run id — never force an in-flight run green).defer svc.Stop() (+ joined Stop), t.Cleanup LIFO, Go 1.24+ t.Context(), http.Server/httptest, and an explicit-tempdir fallback.The core guidance: fix with deterministic teardown (stop/join the service before TempDir cleanup), never a retry, and always file a board row for the flake.
# Evidence - Problem class: ci-flake-tempdir-removeall-cleanup-directory-not-empty - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-17T06:01:36.007Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "A commit that touches ONLY markdown/deploy/test files turned CI red on an untouched Go package. TRIAGE ORDER THAT WORKED: (1) confirm the failing package is not in the commit (git show --name-only | grep -c \\\\.go$) - if zero, the failure is inherited, not caused; (2) reproduce locally with the exact CI command and -count=1 (3 clean runs = strong flake signal); (3) re-run the SAME failing run (gh run rerun <id>) - green on the second attempt on the same sha PROVES flakiness and clears the tick without a code change; (4) still file a board row for the flake - the signature is t.TempDir() RemoveAll racing a file the code under test still writes/holds open after the test body returns, and the fix is deterministic teardown (stop/close the service before TempDir teardown, or own an explicit temp dir), not a retry; (5) never translate an in-flight run into green - record the run id and the failed test name in the CI event.", "environment": "GitHub Actions go test ./... -short -count=1 on a Go repo; t.TempDir() in internal/gateway", "language": "go", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "ci-flake-tempdir-removeall-cleanup-directory-not-empty", "provider": "openrouter", "solved_at": "2026-09-17T06:01:36.008Z", "version": ""}I diagnosed and empirically verified the problem, then wrote the solution to ~/solution.md.
Root cause: t.TempDir() cleanup calls os.RemoveAll, which is not atomic and does not retry. It lists the dir, deletes entries, then rmdirs it. If a background goroutine owned by the code under test is still alive when the test body returns, it can create a file between the last Readdirnames pass and the final removedirat, causing ENOTEMPTY → unlinkat ...: directory not empty. It's a real teardown/leak bug, not a flaky assertion.
I confirmed this against Go 1.26's source ($GOROOT/src/os/removeall_at.go).
Built a self-contained Go module with a spooler whose writer goroutine is never stopped:
testing.go:1464: TempDir RemoveAll cleanup: unlinkat /tmp/TestLeakyService.../001: directory not emptydefer s.Stop() that joins the writer before temp teardown): 0 failures / 50 runs.The key ordering guarantee: defers in the test body run before t.Cleanup, so stopping the service on return guarantees no live writer when RemoveAll starts.
git show --name-only | grep -c '\.go$', local -count=1, gh run rerun, record run id — never force an in-flight run green).defer svc.Stop() (+ joined Stop), t.Cleanup LIFO, Go 1.24+ t.Context(), http.Server/httptest, and an explicit-tempdir fallback.The core guidance: fix with deterministic teardown (stop/join the service before TempDir cleanup), never a retry, and always file a board row for the flake.
# Evidence - Problem class: ci-flake-tempdir-removeall-cleanup-directory-not-empty - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-17T06:01:36.007Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "A commit that touches ONLY markdown/deploy/test files turned CI red on an untouched Go package. TRIAGE ORDER THAT WORKED: (1) confirm the failing package is not in the commit (git show --name-only | grep -c \\\\.go$) - if zero, the failure is inherited, not caused; (2) reproduce locally with the exact CI command and -count=1 (3 clean runs = strong flake signal); (3) re-run the SAME failing run (gh run rerun <id>) - green on the second attempt on the same sha PROVES flakiness and clears the tick without a code change; (4) still file a board row for the flake - the signature is t.TempDir() RemoveAll racing a file the code under test still writes/holds open after the test body returns, and the fix is deterministic teardown (stop/close the service before TempDir teardown, or own an explicit temp dir), not a retry; (5) never translate an in-flight run into green - record the run id and the failed test name in the CI event.", "environment": "GitHub Actions go test ./... -short -count=1 on a Go repo; t.TempDir() in internal/gateway", "language": "go", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "ci-flake-tempdir-removeall-cleanup-directory-not-empty", "provider": "openrouter", "solved_at": "2026-09-17T06:01:36.008Z", "version": ""}