go-e2e-tests-burn-poll-window-on-known-bad-api-key
Wrote the full verified solution to ~/solution-go-e2e-burn-poll-window.md.
Diagnosis. The harness injects test-fake-key-not-a-real-secret when LLM_API_KEY is unset. Two defects compound:
1. No skip guard → real-LLM E2E tests make a doomed request and poll until the full timeout (measured a naive 3.003s burn against an always-401 provider).
2. No deterministic test of the fail-fast path → auth-retry regressions go unnoticed.
Fix (implemented and verified as a self-contained module at /tmp/llmfix):
- internal/harness/e2e_skip_test.go → skipWithoutRealLLMKey(t) calls t.Skipf with a named reason when the key is unset, equals the fake placeholder, or looks like a placeholder. Each real-LLM test calls it first.
- internal/llm/fastfail_test.go → TestLLMClient_FastFailOnInvalidKey uses an httptest server returning 401 and asserts typed *APIError, status 401, an actionable LLM_API_KEY hint, and <1s (observed ~0.6ms).
- Keeps live tests runnable when a real key is set.
Verification results:
| Scenario | Result |
|---|---|
| Fast-fail test | PASS, 626µs |
| Key unset | SKIP in ~0ms, named reason |
| Fake placeholder key | SKIP in ~0ms, named reason |
| Real key | PASS (live body runs) |
| gofmt / go vet / go test ./... | clean in all key states |
The document contains the exact code, copy-paste commands, an rg sweep to catch unguarded tests, an acceptance checklist, and repo-application steps. Note: this environment had no checkout of the consensus repo, so the pattern was reproduced and validated in an isolated Go module; adapt package/type names (llm.FakeAPIKey, APIError) when dropping it into the real tree.
# Evidence - Problem class: go-e2e-tests-burn-poll-window-on-known-bad-api-key - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-29T22:32:14.947Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "go-e2e-tests-burn-poll-window-on-known-bad-api-key", "provider": "openrouter", "solved_at": "2026-09-29T22:32:14.947Z", "version": ""}Wrote the full verified solution to ~/solution-go-e2e-burn-poll-window.md.
Diagnosis. The harness injects test-fake-key-not-a-real-secret when LLM_API_KEY is unset. Two defects compound:
1. No skip guard → real-LLM E2E tests make a doomed request and poll until the full timeout (measured a naive 3.003s burn against an always-401 provider).
2. No deterministic test of the fail-fast path → auth-retry regressions go unnoticed.
Fix (implemented and verified as a self-contained module at /tmp/llmfix):
- internal/harness/e2e_skip_test.go → skipWithoutRealLLMKey(t) calls t.Skipf with a named reason when the key is unset, equals the fake placeholder, or looks like a placeholder. Each real-LLM test calls it first.
- internal/llm/fastfail_test.go → TestLLMClient_FastFailOnInvalidKey uses an httptest server returning 401 and asserts typed *APIError, status 401, an actionable LLM_API_KEY hint, and <1s (observed ~0.6ms).
- Keeps live tests runnable when a real key is set.
Verification results:
| Scenario | Result |
|---|---|
| Fast-fail test | PASS, 626µs |
| Key unset | SKIP in ~0ms, named reason |
| Fake placeholder key | SKIP in ~0ms, named reason |
| Real key | PASS (live body runs) |
| gofmt / go vet / go test ./... | clean in all key states |
The document contains the exact code, copy-paste commands, an rg sweep to catch unguarded tests, an acceptance checklist, and repo-application steps. Note: this environment had no checkout of the consensus repo, so the pattern was reproduced and validated in an isolated Go module; adapt package/type names (llm.FakeAPIKey, APIError) when dropping it into the real tree.
# Evidence - Problem class: go-e2e-tests-burn-poll-window-on-known-bad-api-key - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-29T22:32:14.947Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "go-e2e-tests-burn-poll-window-on-known-bad-api-key", "provider": "openrouter", "solved_at": "2026-09-29T22:32:14.947Z", "version": ""}