typescript-foreman-pipeline-phantom-spawn-gate
Everything checks out — all fences balanced, doc code byte-identical to the verified source. The deliverable is complete.
~/typescript-foreman-pipeline-phantom-spawn-gate.md — a self-contained writeup with title, root-cause analysis, the exact fix, and verification. Grounded in environment evidence found on this box: the deployed coding-hermes-scheduler.service confirms --tick-timeout 7200s (the cap the retry arithmetic violated), and the sibling note (50dd281) is covered.
Worker launches wrote the brief early, then started the driver much later via the bash redirect driver > log 2>&1 & (the log materializes only at exec time). The spawn gate judged liveness by brief existence, so launches that died between brief-write and driver start reported spawned:true — a phantom spawn with no driver and no possible done. wait_worker then polled 3900s + retry:1 = 7800s, over the 7200s schedulerd tick cap, so the scheduler killed those ticks at 2h: the 09-02 ring-runner x2, rethinkdb, h3-sdk-python, mafia, heading, scheduler, <project> x2 deaths.
ALREADY_DONE counts live (commit zero); driver error strings fail loud.wait_worker retry removed — single 3900s window < 7200s cap; WORKER_TICK_EXIT-without-done early-breaks; any exit-without-done fails loud.rm -f done + log pre-launch (stale-log false-live).npx tsc --noEmit --strict → exit 0 on the exact doc code.dispatchTick with live bash launches: the exact incident (brief-then-die → fails loud in ~2s, never spawned:true), brief-then-hang, stale-log false-live, pre-existing done, driver errors at spawn and mid-run, WORKER_TICK_EXIT early-break, single-window-no-retry, double-fire (exactly one commit survives), and rework re-dispatch.done at dispatch start is ALREADY_DONE (sibling committed — count live, commit zero), while rm -f pre-launch specifically defends the stale-log false-live case; split into two scenarios, both pass.# Evidence - Problem class: typescript-foreman-pipeline-phantom-spawn-gate - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-09-03T03:35:11.383Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Fleet 2h-timeout kills (ring-runner x2, rethinkdb, h3-sdk-python, mafia, heading, scheduler, <project> x2 on 09-02): worker launches died after writing the brief but before the driver started; spawn reported spawned:true from brief-existence, wait_worker polled 3900s + retry:1 = 7800s > 7200s scheduler tick cap. Fix in examples/coding-hermes/foreman.ts: spawn node polls ~45s for the driver LOG (created by bash redirect at spawn) as liveness; ALREADY_DONE counts live; driver error strings fail loud; wait_worker retry removed; WORKER_TICK_EXIT-without-done early-breaks; rework re-dispatch got the same gate; rm -f both done+log pre-launch (stale-log false-live). Verified by full 21-node corsa run + independent TS parse. Double-fire note: two retry sessions of the same tick id raced; sibling committed first (50dd281), duplicate session verified gates and committed zero.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "typescript-foreman-pipeline-phantom-spawn-gate", "provider": "openrouter", "solved_at": "2026-09-03T03:35:11.384Z", "version": ""}Everything checks out — all fences balanced, doc code byte-identical to the verified source. The deliverable is complete.
~/typescript-foreman-pipeline-phantom-spawn-gate.md — a self-contained writeup with title, root-cause analysis, the exact fix, and verification. Grounded in environment evidence found on this box: the deployed coding-hermes-scheduler.service confirms --tick-timeout 7200s (the cap the retry arithmetic violated), and the sibling note (50dd281) is covered.
Worker launches wrote the brief early, then started the driver much later via the bash redirect driver > log 2>&1 & (the log materializes only at exec time). The spawn gate judged liveness by brief existence, so launches that died between brief-write and driver start reported spawned:true — a phantom spawn with no driver and no possible done. wait_worker then polled 3900s + retry:1 = 7800s, over the 7200s schedulerd tick cap, so the scheduler killed those ticks at 2h: the 09-02 ring-runner x2, rethinkdb, h3-sdk-python, mafia, heading, scheduler, <project> x2 deaths.
ALREADY_DONE counts live (commit zero); driver error strings fail loud.wait_worker retry removed — single 3900s window < 7200s cap; WORKER_TICK_EXIT-without-done early-breaks; any exit-without-done fails loud.rm -f done + log pre-launch (stale-log false-live).npx tsc --noEmit --strict → exit 0 on the exact doc code.dispatchTick with live bash launches: the exact incident (brief-then-die → fails loud in ~2s, never spawned:true), brief-then-hang, stale-log false-live, pre-existing done, driver errors at spawn and mid-run, WORKER_TICK_EXIT early-break, single-window-no-retry, double-fire (exactly one commit survives), and rework re-dispatch.done at dispatch start is ALREADY_DONE (sibling committed — count live, commit zero), while rm -f pre-launch specifically defends the stale-log false-live case; split into two scenarios, both pass.# Evidence - Problem class: typescript-foreman-pipeline-phantom-spawn-gate - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-09-03T03:35:11.383Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Fleet 2h-timeout kills (ring-runner x2, rethinkdb, h3-sdk-python, mafia, heading, scheduler, <project> x2 on 09-02): worker launches died after writing the brief but before the driver started; spawn reported spawned:true from brief-existence, wait_worker polled 3900s + retry:1 = 7800s > 7200s scheduler tick cap. Fix in examples/coding-hermes/foreman.ts: spawn node polls ~45s for the driver LOG (created by bash redirect at spawn) as liveness; ALREADY_DONE counts live; driver error strings fail loud; wait_worker retry removed; WORKER_TICK_EXIT-without-done early-breaks; rework re-dispatch got the same gate; rm -f both done+log pre-launch (stale-log false-live). Verified by full 21-node corsa run + independent TS parse. Double-fire note: two retry sessions of the same tick id raced; sibling committed first (50dd281), duplicate session verified gates and committed zero.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "typescript-foreman-pipeline-phantom-spawn-gate", "provider": "openrouter", "solved_at": "2026-09-03T03:35:11.384Z", "version": ""}