gate-pytest-arm-needs-process-group-budget

Answer

I diagnosed the problem class and built a verified, self-contained solution at /workspace/solution/:

• SOLUTION.md — the full write-up
• tools/run_pytest_arm.sh — the fixed pytest arm wrapper
• tests/test_run_pytest_arm.sh — hermetic regression (passes)
• gate/run_gate.sh — wires the regression into the gate

Root cause

The 1200 s budget signalled only the pytest process, not the process group. On a loaded host, a stalled test's descendants (xdist workers, servers, lock holders) either ignore SIGTERM and survive, or the gate parent dies and takes the budget owner with it — so orphans and locks leak and wedge the next run (REVIEW-AUGER-005 class).

Fix (core)

PYTEST_BUDGET_SECS="${AUGER_PYTEST_BUDGET_SECS:-1200}"
PYTEST_KILL_AFTER_SECS="${AUGER_PYTEST_KILL_AFTER_SECS:-30}"
ARM="pytest"
tmp="$(mktemp -d)"; pgid_file="$tmp/pgid"; trap 'rm -rf "$tmp"' EXIT

# new session/PGID survives the gate parent; timeout owns the budget
setsid --wait bash -c '
  echo "$$" > "$1"; shift
  exec timeout --signal=TERM --kill-after="$1" "$2" python -m pytest "${@:3}"
' _ "$pgid_file" "$PYTEST_KILL_AFTER_SECS" "$PYTEST_BUDGET_SECS" "$@" &
child=$!; set +e; wait "$child"; rc=$?; set -e

if [[ "$rc" -eq 124 || "$rc" -eq 137 ]]; then
  pgid="$(cat "$pgid_file")"
  kill -TERM -- "-$pgid" 2>/dev/null || true; sleep 1
  kill -KILL -- "-$pgid" 2>/dev/null || true
  printf 'GATE-FAIL arm=%s reason=wall-budget-exceeded budget=%ss kill-after=%ss rc=%s\n' \
    "$ARM" "$PYTEST_BUDGET_SECS" "$PYTEST_KILL_AFTER_SECS" "$rc" >&2
  exit 124
fi
exit "$rc"

Verification (actually run)

$ timeout 60 bash tests/test_run_pytest_arm.sh
ok: TERM-ignoring grandchild 302 was killed
wrapper output: GATE-FAIL arm=pytest reason=wall-budget-exceeded budget=1s kill-after=1s rc=137
PASS test_run_pytest_arm
TEST_RC=0

Also verified: - Gate self-test runs first, then the real arm succeeds (GATE_RC=0). - Exit codes pass through untouched (0 on pass, 3 on real failure) — only a budget trip becomes 124. - After SIGKILL of the wrapper, the detached setsid budget owner stayed alive (t=2 yes) and exited on its own (t=6 no), proving the budget survives parent death. - shellcheck -S warning clean.

Full root-cause analysis, the Python gate variant, the regression code, the gate wiring, acceptance-criteria mapping, and rollout notes are in /workspace/solution/SOLUTION.md.