pytest-host-user-rules-contamination-triage
No project checkout exists in this environment (only /workspace/problem.json), so I produced the solution as a standalone, directly usable runbook: /workspace/SOLUTION.md.
Root cause (host contamination, not a regression)
- The installer copied shipped YAML into ~/.config/<tool>/rules.d/.
- The engine merges sources with last-writer-wins and treats that dir as USER rules, so same-ID YAML overwrites the stricter hardcoded builtins (auto-script, fork-bomb whitespace, quoted mkfs).
- Verdicts flip allow, turning the 12 tests red. CI has a clean $HOME, so it stays green.
Triage that proves pre-existing
1. Probe 1 – run the identical failing files in a git worktree at pristine pre-change HEAD with the same real HOME; an identical FAILED set = pre-existing.
2. Probe 2 – re-run with user rules neutralized (XDG_CONFIG_HOME=$(mktemp -d) or <TOOL>_USER_RULES_DIR=/nonexistent); green proves the change caused nothing.
Permanent fix (two parts)
- Engine: enforce precedence builtin > user > shipped by loading builtins last and refusing same-ID overrides of builtin origin (code provided).
- Installer: stop writing shipped YAML into rules.d/; load it as SHIPPED from package data, or install it under a separate shipped.d/ (with a migration that only moves byte-identical files).
- Immediate remediation: reversibly move ~/.config/<tool>/rules.d aside rather than deleting it.
Verification/acceptance includes Probe 0 (capture 12 node IDs), Probe 1 (diff empty), Probe 2b, full-suite green after the fix, a regression test asserting builtins are non-overridable, plus a one-shot triage.sh script.
Guardrails — never complete on the bare pytest command; never claim the change broke tests before both probes; if Probe 1 differs from baseline, bisect the change instead of applying the host fix.
# Evidence - Problem class: pytest-host-user-rules-contamination-triage - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-11T01:50:55.278Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Full pytest suite red on a host (12 failed/295 passed/13 skipped) while CI is green. Root cause: the project installs its shipped YAML rules to ~/.config/<tool>/rules.d/, which the engine loads as USER rules with SAME-ID OVERRIDE, so the weaker YAML patterns replace the hardcoded builtins (auto-script, fork-bomb whitespace, quoted mkfs) and flip 12 verdicts to allow. Triage that proves pre-existing rather than regression: (1) run the identical failing test files in a git worktree at pristine pre-change HEAD with the same real HOME - identical FAILED set = pre-existing; (2) re-run with the user-rules dir neutralized (env override or clean HOME) - green suite proves the change caused nothing. Never mark such a task complete on the bare command, and never claim the change broke tests before both probes.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "pytest-host-user-rules-contamination-triage", "provider": "openrouter", "solved_at": "2026-09-11T01:50:55.278Z", "version": ""}