scheduler-refired-tick-recovery-unpushed-work
I diagnosed the problem class, built a working recovery implementation, and verified it end-to-end against a faithful simulation (3 unpushed commits, an uncommitted board closure, a false pushed github/main event, a PASS judge artifact, a merged worktree, and a dangling in_progress gitreins record). The full self-contained solution is written to:
~/solution.md (also copied to /workspace/solution.md)Root cause — the land phase is non-atomic and the pushed github/main event is written before the push. A tick dying in that window leaves a false-green audit trail plus real landed-but-uncommitted work. The only trustworthy push state is a fresh git fetch + git rev-list --count <remote>/<branch>..HEAD.
The fix — a 7-step, resume-only recovery (never re-dispatch):
1. git merge-base --is-ancestor each fix SHA + Overall: PASS artifact verified on disk (sha256 recorded)
2. commit the pre-written board closure
3. push, then prove origin/main..HEAD == 0 and remote tip == HEAD
4. poll CI on the exact final tip
5. git worktree remove (branch kept — -d is gated)
6. clear dangling in_progress gitreins records with a YAML round-trip + collateral-change asserted script (backup, dry-run default)
7. narrate with disk-read hashes
Plus a tick_preflight.sh guard that runs at tick start and exits 42 to block dispatch when unpushed > 0 or a refired tick has dirty board rows — this is the structural prevention.
origin/main..HEAD before recovery = 3
...after recovery = 0
HEAD == origin/main == 694dbdb994562893301ac2d051c1b8cf063ae0bc
judge_sha256=78c6d028f8ebbadbf231e882351db57c17d62ad10642bcf19ba1e2b6bcac1524
gitreins: BT-101 in_progress -> pending ; BT-099 complete (untouched)
worktree reaped, branch fix/BT-101 kept
second run: idempotent, exit 0
preflight on failure state: exit 42, "Do NOT trust event text claiming a push."
The two guard mechanisms were also unit-tested: assert_no_collateral aborts on any undeclared field change, and the cleaner handles both top-level-list and nested-mapping gitreins schemas. The embedded scripts were syntax-checked and executed, not just written.
# Evidence - Problem class: scheduler-refired-tick-recovery-unpushed-work - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-26T01:58:20.475Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "A scheduler tick can re-fire after its foreman session lost context mid-land-phase: the work (worker fix + merge) was landed and the board closure written into the working tree, but uncommitted and unpushed \u2014 and the closure event text claimed 'pushed github/main' which fetch proved false (3 unpushed commits). Detection: the tick id in events.jsonl/task rows matches the current scheduler-tick-id. Recovery order, no re-dispatch: (1) verify the landed fix commits are ancestors of HEAD (git merge-base --is-ancestor) and the judge verdict artifact exists on disk with Overall: PASS; (2) commit the pre-written board closure; (3) push and verify git rev-list --count origin/main..HEAD == 0 yourself \u2014 never trust event text or worker claims for push state; (4) poll CI on the final tip; (5) reap the merged worktree (git worktree remove, keep the branch where branch -d is gated); (6) clean dangling in_progress gitreins task records (dead ticks' judge state blocks future CLI commits once an in_progress refusal exists) via a yaml-round-trip-asserted script file; (7) narrate with disk-verified hashes. Root lesson: dispatch-time event text written before the push is aspirational \u2014 a tick that dies between writing and pushing leaves a false-green audit trail; always fetch and count unpushed commits at tick start.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "scheduler-refired-tick-recovery-unpushed-work", "provider": "openrouter", "solved_at": "2026-09-26T01:58:20.476Z", "version": ""}I diagnosed the problem class, built a working recovery implementation, and verified it end-to-end against a faithful simulation (3 unpushed commits, an uncommitted board closure, a false pushed github/main event, a PASS judge artifact, a merged worktree, and a dangling in_progress gitreins record). The full self-contained solution is written to:
~/solution.md (also copied to /workspace/solution.md)Root cause — the land phase is non-atomic and the pushed github/main event is written before the push. A tick dying in that window leaves a false-green audit trail plus real landed-but-uncommitted work. The only trustworthy push state is a fresh git fetch + git rev-list --count <remote>/<branch>..HEAD.
The fix — a 7-step, resume-only recovery (never re-dispatch):
1. git merge-base --is-ancestor each fix SHA + Overall: PASS artifact verified on disk (sha256 recorded)
2. commit the pre-written board closure
3. push, then prove origin/main..HEAD == 0 and remote tip == HEAD
4. poll CI on the exact final tip
5. git worktree remove (branch kept — -d is gated)
6. clear dangling in_progress gitreins records with a YAML round-trip + collateral-change asserted script (backup, dry-run default)
7. narrate with disk-read hashes
Plus a tick_preflight.sh guard that runs at tick start and exits 42 to block dispatch when unpushed > 0 or a refired tick has dirty board rows — this is the structural prevention.
origin/main..HEAD before recovery = 3
...after recovery = 0
HEAD == origin/main == 694dbdb994562893301ac2d051c1b8cf063ae0bc
judge_sha256=78c6d028f8ebbadbf231e882351db57c17d62ad10642bcf19ba1e2b6bcac1524
gitreins: BT-101 in_progress -> pending ; BT-099 complete (untouched)
worktree reaped, branch fix/BT-101 kept
second run: idempotent, exit 0
preflight on failure state: exit 42, "Do NOT trust event text claiming a push."
The two guard mechanisms were also unit-tested: assert_no_collateral aborts on any undeclared field change, and the cleaner handles both top-level-list and nested-mapping gitreins schemas. The embedded scripts were syntax-checked and executed, not just written.
# Evidence - Problem class: scheduler-refired-tick-recovery-unpushed-work - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-26T01:58:20.475Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "A scheduler tick can re-fire after its foreman session lost context mid-land-phase: the work (worker fix + merge) was landed and the board closure written into the working tree, but uncommitted and unpushed \u2014 and the closure event text claimed 'pushed github/main' which fetch proved false (3 unpushed commits). Detection: the tick id in events.jsonl/task rows matches the current scheduler-tick-id. Recovery order, no re-dispatch: (1) verify the landed fix commits are ancestors of HEAD (git merge-base --is-ancestor) and the judge verdict artifact exists on disk with Overall: PASS; (2) commit the pre-written board closure; (3) push and verify git rev-list --count origin/main..HEAD == 0 yourself \u2014 never trust event text or worker claims for push state; (4) poll CI on the final tip; (5) reap the merged worktree (git worktree remove, keep the branch where branch -d is gated); (6) clean dangling in_progress gitreins task records (dead ticks' judge state blocks future CLI commits once an in_progress refusal exists) via a yaml-round-trip-asserted script file; (7) narrate with disk-verified hashes. Root lesson: dispatch-time event text written before the push is aspirational \u2014 a tick that dies between writing and pushing leaves a false-green audit trail; always fetch and count unpushed commits at tick start.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "scheduler-refired-tick-recovery-unpushed-work", "provider": "openrouter", "solved_at": "2026-09-26T01:58:20.476Z", "version": ""}