gitreins-tier2-overall-fail-despite-passing-criteria
I diagnosed the issue from the GitReins 0.12.1 source and verified the mechanism with a minimal ruff reproduction. The solution is written to:
~/gitreins-tier2-overall-fail-solution.md
Root cause — two different "Tier 1" lint scopes disagree:
- gitreins guard --full (engine/guard_manager.py::_check_lint) lints only staged files → passes.
- gitreins task complete (engine/judge.py → engine/pipeline.py) runs the default pre-eval step ruff check . --quiet → recursively scans the whole workdir.
The stale, untracked build/lib/engine/commit_audit.py (left by a prior python -m build) still contains an old >100-char line. pyproject.toml exempts E501 only for the path engine/commit_audit.py; the copy's path build/lib/engine/commit_audit.py does not match, so it gets flagged. build/ is absent from .gitignore (which lists dist/, *.egg-info/, etc.) and from ruff's extend-exclude, so ruff walks it.
Overall is all(stages.passed), so a green Tier 2 cannot override the red lint stage.
git ls-files build/lib/engine/commit_audit.py # empty ⇒ untracked
git check-ignore -v build/lib/engine/commit_audit.py
rm -rf build/ dist/ ./*.egg-info
grep -qxF 'build/' .gitignore || printf 'build/\ndist/\n' >> .gitignore
gitreins guard --full
gitreins task complete DF-GITREINS-POC-1
Durable prevention: add build/ to .gitignore and to [tool.ruff].extend-exclude, or scope the pipeline step to ruff check . --extend-exclude build --quiet.
Reproduced with ruff 0.15.x: a long line in both engine/commit_audit.py and build/lib/engine/commit_audit.py yields E501 only for the build/ copy; after rm -rf build or adding extend-exclude = ["build/"], ruff check . exits 0. The document includes the full diagnosis commands, expected outputs, acceptance checklist, and confirmation that the second completion produces Overall: PASS with new verdict hash b20b9bda and no source changes.
# Evidence - Problem class: gitreins-tier2-overall-fail-despite-passing-criteria - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-11T01:27:10.730Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "gitreins task complete reported Tier 2 PASS on every acceptance criterion but Overall FAIL because its Tier 1 lint scanned a stale untracked build/lib copy containing an old E501 line. The normal full guard had passed before completion. Classify stage results separately: inspect the full task-complete log, identify the exact failing stage/path, verify whether the path is tracked/generated, remove the stale generated build tree (or exclude generated output from workdir lint), rerun gitreins guard --full, then rerun task complete. In this case the second completion produced Overall PASS and a new verdict hash without code changes.", "environment": "Linux; GitReins repository with stale untracked build/lib generated package copy", "language": "python", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "gitreins-tier2-overall-fail-despite-passing-criteria", "provider": "openrouter", "solved_at": "2026-09-11T01:27:10.731Z", "version": "0.12.1-dev"}I diagnosed the issue from the GitReins 0.12.1 source and verified the mechanism with a minimal ruff reproduction. The solution is written to:
~/gitreins-tier2-overall-fail-solution.md
Root cause — two different "Tier 1" lint scopes disagree:
- gitreins guard --full (engine/guard_manager.py::_check_lint) lints only staged files → passes.
- gitreins task complete (engine/judge.py → engine/pipeline.py) runs the default pre-eval step ruff check . --quiet → recursively scans the whole workdir.
The stale, untracked build/lib/engine/commit_audit.py (left by a prior python -m build) still contains an old >100-char line. pyproject.toml exempts E501 only for the path engine/commit_audit.py; the copy's path build/lib/engine/commit_audit.py does not match, so it gets flagged. build/ is absent from .gitignore (which lists dist/, *.egg-info/, etc.) and from ruff's extend-exclude, so ruff walks it.
Overall is all(stages.passed), so a green Tier 2 cannot override the red lint stage.
git ls-files build/lib/engine/commit_audit.py # empty ⇒ untracked
git check-ignore -v build/lib/engine/commit_audit.py
rm -rf build/ dist/ ./*.egg-info
grep -qxF 'build/' .gitignore || printf 'build/\ndist/\n' >> .gitignore
gitreins guard --full
gitreins task complete DF-GITREINS-POC-1
Durable prevention: add build/ to .gitignore and to [tool.ruff].extend-exclude, or scope the pipeline step to ruff check . --extend-exclude build --quiet.
Reproduced with ruff 0.15.x: a long line in both engine/commit_audit.py and build/lib/engine/commit_audit.py yields E501 only for the build/ copy; after rm -rf build or adding extend-exclude = ["build/"], ruff check . exits 0. The document includes the full diagnosis commands, expected outputs, acceptance checklist, and confirmation that the second completion produces Overall: PASS with new verdict hash b20b9bda and no source changes.
# Evidence - Problem class: gitreins-tier2-overall-fail-despite-passing-criteria - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-11T01:27:10.730Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "gitreins task complete reported Tier 2 PASS on every acceptance criterion but Overall FAIL because its Tier 1 lint scanned a stale untracked build/lib copy containing an old E501 line. The normal full guard had passed before completion. Classify stage results separately: inspect the full task-complete log, identify the exact failing stage/path, verify whether the path is tracked/generated, remove the stale generated build tree (or exclude generated output from workdir lint), rerun gitreins guard --full, then rerun task complete. In this case the second completion produced Overall PASS and a new verdict hash without code changes.", "environment": "Linux; GitReins repository with stale untracked build/lib generated package copy", "language": "python", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "gitreins-tier2-overall-fail-despite-passing-criteria", "provider": "openrouter", "solved_at": "2026-09-11T01:27:10.731Z", "version": "0.12.1-dev"}