stale-guard-test-asserts-behind-deliberate-code-change
I wrote the full solution to ~/solution.md and reproduced/verified the fix pattern locally.
The repo could not be accessed. dexdat/eduos.dexdat.com.co is private: git clone prompts for credentials and the GitHub API returns 404. /workspace/problem.json contains only the problem description, and a filesystem-wide grep found no source. So I could not patch the real files.
I did the next best thing: I built a minimal reproduction that mirrors the exact scenario and actually ran it.
Two deliberate, already-merged changes moved the ground truth, but guard tests still asserted frozen snapshots:
| Deliberate change | New truth | Stale assert |
|---|---|---|
IMG-CONTRACT-001 adds role image tts |
{ml, api, tts} |
== {"ml","api"} |
| VOICE-GAP-025 raises STT timeout | 30000 |
== 8000 |
Red suite, no production defect. Critically, the fix must not relax the check — I demonstrated that regressed <= contract (subset) silently passes when api is dropped, while == correctly catches it.
ROLE_BUILD_SOURCES from tests/test_ci_image_contract.py and assert exact equality (built == set(ROLE_BUILD_SOURCES)), plus a cardinality check.== 30_000 (or assert against STT_TIMEOUT_MS and pin its value).rg commands to locate stale literals and an anti-pattern check for issubset/<=.Before: 2 failed (assert 30000 == 8000, exact-set mismatch). After: 2 passed. Strength check: subset passes the regression, exact equality fails it — so the guard's strength is preserved.
The markdown is self-contained with locate → patch → verify → commit steps, and is honest about the path caveat and how I verified instead.
# Evidence - Problem class: stale-guard-test-asserts-behind-deliberate-code-change - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-21T00:20:33.137Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Guard tests asserting an exact set/count that a LATER deliberate product change invalidates (e.g. exact role-image set {ml, api} when a third role image tts is added; exact timeoutMs==8000 when stt_timeout deliberately raised to 30s). Symptom: suite red with no production defect; the assert is stale, not the code. Fix pattern: realign the guard to the deliberate change PRESERVING its strength (exact-set stays exact; mirror the authoritative contract file where one exists - here test_ci_image_contract.py ROLE_BUILD_SOURCES), and never relax to a subset/superset check. Update sibling tests in the same commit. Repo: dexdat/eduos.dexdat.com.co services/voice-agent (IMG-CONTRACT-001 + VOICE-GAP-025 follow-on).", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "stale-guard-test-asserts-behind-deliberate-code-change", "provider": "openrouter", "solved_at": "2026-09-21T00:20:33.138Z", "version": ""}I wrote the full solution to ~/solution.md and reproduced/verified the fix pattern locally.
The repo could not be accessed. dexdat/eduos.dexdat.com.co is private: git clone prompts for credentials and the GitHub API returns 404. /workspace/problem.json contains only the problem description, and a filesystem-wide grep found no source. So I could not patch the real files.
I did the next best thing: I built a minimal reproduction that mirrors the exact scenario and actually ran it.
Two deliberate, already-merged changes moved the ground truth, but guard tests still asserted frozen snapshots:
| Deliberate change | New truth | Stale assert |
|---|---|---|
IMG-CONTRACT-001 adds role image tts |
{ml, api, tts} |
== {"ml","api"} |
| VOICE-GAP-025 raises STT timeout | 30000 |
== 8000 |
Red suite, no production defect. Critically, the fix must not relax the check — I demonstrated that regressed <= contract (subset) silently passes when api is dropped, while == correctly catches it.
ROLE_BUILD_SOURCES from tests/test_ci_image_contract.py and assert exact equality (built == set(ROLE_BUILD_SOURCES)), plus a cardinality check.== 30_000 (or assert against STT_TIMEOUT_MS and pin its value).rg commands to locate stale literals and an anti-pattern check for issubset/<=.Before: 2 failed (assert 30000 == 8000, exact-set mismatch). After: 2 passed. Strength check: subset passes the regression, exact equality fails it — so the guard's strength is preserved.
The markdown is self-contained with locate → patch → verify → commit steps, and is honest about the path caveat and how I verified instead.
# Evidence - Problem class: stale-guard-test-asserts-behind-deliberate-code-change - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-21T00:20:33.137Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Guard tests asserting an exact set/count that a LATER deliberate product change invalidates (e.g. exact role-image set {ml, api} when a third role image tts is added; exact timeoutMs==8000 when stt_timeout deliberately raised to 30s). Symptom: suite red with no production defect; the assert is stale, not the code. Fix pattern: realign the guard to the deliberate change PRESERVING its strength (exact-set stays exact; mirror the authoritative contract file where one exists - here test_ci_image_contract.py ROLE_BUILD_SOURCES), and never relax to a subset/superset check. Update sibling tests in the same commit. Repo: dexdat/eduos.dexdat.com.co services/voice-agent (IMG-CONTRACT-001 + VOICE-GAP-025 follow-on).", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "stale-guard-test-asserts-behind-deliberate-code-change", "provider": "openrouter", "solved_at": "2026-09-21T00:20:33.138Z", "version": ""}