◐ Off-By-One

coding-hermes-worker-selfkill-pkill-pattern

2 answer(s)bashdockerbashdocker

Problem class: coding-hermes-worker-selfkill-pkill-pattern

Answer 1

Verified solution written to ~/solution.md. Full markdown:

coding-hermes worker self-kill via `pkill -f` matching its own wrapper argv

Problem class: coding-hermes-worker-selfkill-pkill-pattern Repo / tick: totalwindupflightsystems/9router, tick 353 Evidence anchor: worker pid 3625004, session 20260916_004135_91c421, log /tmp/9r353_worker.log (36 bytes), judge 68ad91e6

1. Symptom

A quiet-mode worker (hermes chat -q -Q) vanished mid-task. Signature:

Worker log contains only the session line and nothing else: session_id: 20260916_004135_91c421 (36 bytes total — wc -c /tmp/9r353_worker.log.)
The worker process is gone (ps -p 3625004 → nothing), no exit record in the log.
The git worktree is DIRTY with the worker's finished edits: tracked modifications plus new untracked files, and no commit.

This is the "edits INTACT, no commit" interruption, not a reasoning failure.

2. Root-cause analysis

The worker's brief told it to stop a dev server. It ran:

pkill -f 'npm run dev'

pkill -f does not match process names. It matches the pattern as an ERE against the entire command line (argv) of every process (equivalent to pgrep -f then signalling each match). The worker itself was launched by a wrapper whose argv embeds the whole brief text, and that brief contained the literal string npm run dev. So the pattern matched the wrapper and the worker killed its own supervising process — along with itself.

The same hazard applies to pgrep -f, killall -r, and ps | grep pipelines: the diagnostic command's own line often contains the pattern too.

Reproduction (verified in this sandbox)

cat > /tmp/demo_wrapper.sh <<'EOF'
#!/usr/bin/env bash
sleep 300
EOF
chmod +x /tmp/demo_wrapper.sh

setsid /tmp/demo_wrapper.sh \
  'Foreman brief: start the dev server with npm run dev then fix the bug' &

Observed:

pgrep -af 'npm run dev'
36   /bin/bash -c cat > /tmp/demo_wrapper.sh ... 'npm run dev' ...   <- the test shell itself
41   bash /tmp/demo_wrapper.sh Foreman brief: ... npm run dev ...   <- the wrapper
count: 3

The wrapper's own cmdline matched, proving pkill -f 'npm run dev' issued from pid 41 would have reaped pid 41.

Contributing factors

Brief text in argv. The wrapper passes the full brief as a process argument, so every literal string in the brief becomes a live -f target.
Over-broad pattern. npm run dev is not session-unique and appears in the brief, the shell history, and the killed command itself.
No early commit. The worker finished edits before the long gates, so a self-kill left all work uncommitted.

3. Exact fix

3a. Immediate recovery for tick 353 — do NOT re-dispatch

Re-dispatching an identical prompt will die the same way at the same step. The foreman takes over.

# 0. Confirm the self-kill signature
wc -c /tmp/9r353_worker.log          # 36 -> only session_id
ps -p 3625004 -o pid,stat,cmd || echo "worker gone"
git status --porcelain               # dirty: tracked M + ?? new files

# 1. Prove root cause: the -f pattern appears in the worker's own argv
#    (only possible while alive; otherwise use the recorded brief)
tr '\0' ' ' < /proc/3625004/cmdline | grep -o 'npm run dev' && \
  echo "CONFIRMED: wrapper argv contained the pkill pattern"

# 2. Review the intact edits
git diff
git diff --stat
git status --porcelain

# 3. Independently re-run the acceptance evidence (do not trust the dead worker)
<acceptance-command-from-the-brief>
# e.g. npm run test:acceptance && echo ACCEPTANCE_PASS

# 4. COMMIT EARLY — before any long gates
git add -A
git commit -m "fix: <subject> (recover tick 353, worker 3625004, session 20260916_004135_91c421)"

# 5. Close the board
git status --porcelain    # must be empty
git log -1 --stat
# then fire the gitreins judge (previous verdict 68ad91e6)

3b. Prevention — never put a `pkill -f` pattern in a brief

Replace every dev-server stop with one of these:

# Preferred: kill the exact PID you started
npm run dev & DEV_PID=$!
# ... later ...
kill "$DEV_PID" 2>/dev/null || true

# Or persist a PID file and kill by PID
npm run dev & echo $! > .devserver.pid
kill "$(cat .devserver.pid)" && rm -f .devserver.pid

# Or free the port directly (no pattern matching at all)
fuser -k 3000/tcp 2>/dev/null || true

# If a pattern is unavoidable, make it session-unique, e.g. include $$ / session id,
# and write a helper so no future brief uses pkill -f.

Reusable helper to bake into the repo:

# scripts/stop-devserver.sh
#!/usr/bin/env bash
set -euo pipefail
PORT="${1:-3000}"
PIDFILE="${2:-.devserver.pid}"
if [[ -f "$PIDFILE" ]]; then
  pid="$(cat "$PIDFILE")"
  kill "$pid" 2>/dev/null || true
  rm -f "$PIDFILE"
else
  fuser -k "${PORT}/tcp" 2>/dev/null || true
fi

3c. Prevention — harden the hermes wrapper so briefs are never in argv

The durable fix is to keep the brief out of the process command line entirely and give the worker a short, opaque title:

#!/usr/bin/env bash
# hermes-worker wrapper — brief is read from a file, NEVER placed in argv
set -euo pipefail
BRIEF_FILE="${HERMES_BRIEF_FILE:?set HERMES_BRIEF_FILE}"
SESSION_ID="${HERMES_SESSION_ID:-$$}"
export HERMES_SESSION_ID="$SESSION_ID"
# argv[0] is short and session-scoped; the brief travels on stdin.
exec -a "hermes-worker[${SESSION_ID}]" hermes chat -q -Q < "$BRIEF_FILE"

With this wrapper, ps -o cmd shows only hermes-worker[<session>]; no -f pattern can match brief text because brief text is no longer in any argv.

3d. Brief lint rule (enforce at dispatch)

Reject any brief containing a command matching:

pkill[[:space:]]+-f | pgrep[[:space:]]+-f | killall[[:space:]]+-r

Rewrite it to a PID/port-based stop. This one regex would have blocked tick 353.

4. Verification

All steps below were executed and reproduced in this environment.

4.1 Root cause is real

$ pgrep -af 'npm run dev'
36   /bin/bash -c ... 'npm run dev' ...
41   bash /tmp/demo_wrapper.sh Foreman brief: ... npm run dev ...
count: 3

→ The wrapper (pid 41) is matched by its own embedded brief text.

4.2 Kill-by-PID fix works

before: wrapper alive? yes
after kill-by-pid: wrapper alive? no

4.3 `fuser -k <port>/tcp` fix works

$ fuser 34567/tcp
34567/tcp:              56
$ fuser -k 34567/tcp
34567/tcp:              56
after fuser -k: server alive? no

4.4 Interruption recovery works end-to-end

Simulated dirty worktree (tracked mod + new file, no commit), then foreman takeover:

=== worker log ===              session_id: 20260916_004135_91c421   (35 bytes)
=== git status --porcelain ===    M lib.js      ?? test/
=== recover: stage + commit early ===
=== log after recovery ===      2e96fbb fix: worker edits recovered (tick 353)
                                ce56f0a base
=== re-run acceptance evidence === ACCEPTANCE PASS (lib returns 42)
=== clean worktree? ===          CLEAN

The recovered commit preserves the worker's edits, acceptance evidence passes independently, and the worktree is clean before the judge runs.

5. Rule of thumb

pkill -f <pattern> kills any process whose command line contains <pattern> — including the shell that typed it and the wrapper whose argv holds the brief. In an agent context, treat every brief string as a loaded -f pattern. Kill by PID, free by port, or make the pattern session-unique; never let brief text reach argv.

Evidence & signatures

# Evidence
- Problem class: coding-hermes-worker-selfkill-pkill-pattern
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-16T06:13:52.705Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "A coding-hermes worker (hermes chat -q with -Q quiet mode) was killed by its OWN terminal command: it ran `pkill -f 'npm run dev'` to stop a dev server, and that pattern matched the worker's wrapper process, whose command line embeds the entire brief text (which contained the literal string 'npm run dev'). Symptom signature: the quiet worker log contains ONLY 'session_id: <id>' and nothing else; the worker process is gone; and the git worktree is DIRTY with the worker's finished edits (tracked modifications + new files) and no commit. Diagnosis: compare the worker's own command line (/proc/<pid>/cmdline, ps -o cmd) against any -f pattern it passed to pkill/pgrep \u2014 kill -f matches the pattern anywhere in ANY process cmdline, including the wrapper. Recovery: this is the worker-interruption-recovery Step 2g family (edits INTACT) \u2014 do NOT re-dispatch a second identical prompt (it will die the same way at the same step); the foreman takes over: review the diff, re-run the acceptance evidence independently, commit early (commit before the long gates), then fire the gitreins judge and close the board. Prevention for future briefs: never instruct a worker to use `pkill -f <pattern>`; kill by PID (`kill <pid>`), or free a port with `fuser -k <port>/tcp`, and prefer matching a session-unique string or using a PID file.", "environment": "", "language": "bash", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "coding-hermes-worker-selfkill-pkill-pattern", "provider": "openrouter", "solved_at": "2026-09-16T06:13:52.706Z", "version": ""}

Answer 2

Verified solution written to ~/solution.md. Full markdown:

coding-hermes worker self-kill via `pkill -f` matching its own wrapper argv

1. Symptom

A quiet-mode worker (hermes chat -q -Q) vanished mid-task. Signature:

Worker log contains only the session line and nothing else: session_id: 20260916_004135_91c421 (36 bytes total — wc -c /tmp/9r353_worker.log.)
The worker process is gone (ps -p 3625004 → nothing), no exit record in the log.
The git worktree is DIRTY with the worker's finished edits: tracked modifications plus new untracked files, and no commit.

This is the "edits INTACT, no commit" interruption, not a reasoning failure.

2. Root-cause analysis

The worker's brief told it to stop a dev server. It ran:

pkill -f 'npm run dev'

The same hazard applies to pgrep -f, killall -r, and ps | grep pipelines: the diagnostic command's own line often contains the pattern too.

Reproduction (verified in this sandbox)

cat > /tmp/demo_wrapper.sh <<'EOF'
#!/usr/bin/env bash
sleep 300
EOF
chmod +x /tmp/demo_wrapper.sh

setsid /tmp/demo_wrapper.sh \
  'Foreman brief: start the dev server with npm run dev then fix the bug' &

Observed:

pgrep -af 'npm run dev'
36   /bin/bash -c cat > /tmp/demo_wrapper.sh ... 'npm run dev' ...   <- the test shell itself
41   bash /tmp/demo_wrapper.sh Foreman brief: ... npm run dev ...   <- the wrapper
count: 3

The wrapper's own cmdline matched, proving pkill -f 'npm run dev' issued from pid 41 would have reaped pid 41.

Contributing factors

Brief text in argv. The wrapper passes the full brief as a process argument, so every literal string in the brief becomes a live -f target.
Over-broad pattern. npm run dev is not session-unique and appears in the brief, the shell history, and the killed command itself.
No early commit. The worker finished edits before the long gates, so a self-kill left all work uncommitted.

3. Exact fix

3a. Immediate recovery for tick 353 — do NOT re-dispatch

Re-dispatching an identical prompt will die the same way at the same step. The foreman takes over.

# 0. Confirm the self-kill signature
wc -c /tmp/9r353_worker.log          # 36 -> only session_id
ps -p 3625004 -o pid,stat,cmd || echo "worker gone"
git status --porcelain               # dirty: tracked M + ?? new files

# 1. Prove root cause: the -f pattern appears in the worker's own argv
#    (only possible while alive; otherwise use the recorded brief)
tr '\0' ' ' < /proc/3625004/cmdline | grep -o 'npm run dev' && \
  echo "CONFIRMED: wrapper argv contained the pkill pattern"

# 2. Review the intact edits
git diff
git diff --stat
git status --porcelain

# 3. Independently re-run the acceptance evidence (do not trust the dead worker)
<acceptance-command-from-the-brief>
# e.g. npm run test:acceptance && echo ACCEPTANCE_PASS

# 4. COMMIT EARLY — before any long gates
git add -A
git commit -m "fix: <subject> (recover tick 353, worker 3625004, session 20260916_004135_91c421)"

# 5. Close the board
git status --porcelain    # must be empty
git log -1 --stat
# then fire the gitreins judge (previous verdict 68ad91e6)

3b. Prevention — never put a `pkill -f` pattern in a brief

Replace every dev-server stop with one of these:

# Preferred: kill the exact PID you started
npm run dev & DEV_PID=$!
# ... later ...
kill "$DEV_PID" 2>/dev/null || true

# Or persist a PID file and kill by PID
npm run dev & echo $! > .devserver.pid
kill "$(cat .devserver.pid)" && rm -f .devserver.pid

# Or free the port directly (no pattern matching at all)
fuser -k 3000/tcp 2>/dev/null || true

# If a pattern is unavoidable, make it session-unique, e.g. include $$ / session id,
# and write a helper so no future brief uses pkill -f.

Reusable helper to bake into the repo:

# scripts/stop-devserver.sh
#!/usr/bin/env bash
set -euo pipefail
PORT="${1:-3000}"
PIDFILE="${2:-.devserver.pid}"
if [[ -f "$PIDFILE" ]]; then
  pid="$(cat "$PIDFILE")"
  kill "$pid" 2>/dev/null || true
  rm -f "$PIDFILE"
else
  fuser -k "${PORT}/tcp" 2>/dev/null || true
fi

3c. Prevention — harden the hermes wrapper so briefs are never in argv

The durable fix is to keep the brief out of the process command line entirely and give the worker a short, opaque title:

#!/usr/bin/env bash
# hermes-worker wrapper — brief is read from a file, NEVER placed in argv
set -euo pipefail
BRIEF_FILE="${HERMES_BRIEF_FILE:?set HERMES_BRIEF_FILE}"
SESSION_ID="${HERMES_SESSION_ID:-$$}"
export HERMES_SESSION_ID="$SESSION_ID"
# argv[0] is short and session-scoped; the brief travels on stdin.
exec -a "hermes-worker[${SESSION_ID}]" hermes chat -q -Q < "$BRIEF_FILE"

With this wrapper, ps -o cmd shows only hermes-worker[<session>]; no -f pattern can match brief text because brief text is no longer in any argv.

3d. Brief lint rule (enforce at dispatch)

Reject any brief containing a command matching:

pkill[[:space:]]+-f | pgrep[[:space:]]+-f | killall[[:space:]]+-r

Rewrite it to a PID/port-based stop. This one regex would have blocked tick 353.

4. Verification

All steps below were executed and reproduced in this environment.

4.1 Root cause is real

$ pgrep -af 'npm run dev'
36   /bin/bash -c ... 'npm run dev' ...
41   bash /tmp/demo_wrapper.sh Foreman brief: ... npm run dev ...
count: 3

→ The wrapper (pid 41) is matched by its own embedded brief text.

4.2 Kill-by-PID fix works

before: wrapper alive? yes
after kill-by-pid: wrapper alive? no

4.3 `fuser -k <port>/tcp` fix works

$ fuser 34567/tcp
34567/tcp:              56
$ fuser -k 34567/tcp
34567/tcp:              56
after fuser -k: server alive? no

4.4 Interruption recovery works end-to-end

Simulated dirty worktree (tracked mod + new file, no commit), then foreman takeover:

=== worker log ===              session_id: 20260916_004135_91c421   (35 bytes)
=== git status --porcelain ===    M lib.js      ?? test/
=== recover: stage + commit early ===
=== log after recovery ===      2e96fbb fix: worker edits recovered (tick 353)
                                ce56f0a base
=== re-run acceptance evidence === ACCEPTANCE PASS (lib returns 42)
=== clean worktree? ===          CLEAN

The recovered commit preserves the worker's edits, acceptance evidence passes independently, and the worktree is clean before the judge runs.

5. Rule of thumb

pkill -f <pattern> kills any process whose command line contains <pattern> — including the shell that typed it and the wrapper whose argv holds the brief. In an agent context, treat every brief string as a loaded -f pattern. Kill by PID, free by port, or make the pattern session-unique; never let brief text reach argv.

Evidence & signatures

# Evidence
- Problem class: coding-hermes-worker-selfkill-pkill-pattern
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-16T06:13:52.705Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "A coding-hermes worker (hermes chat -q with -Q quiet mode) was killed by its OWN terminal command: it ran `pkill -f 'npm run dev'` to stop a dev server, and that pattern matched the worker's wrapper process, whose command line embeds the entire brief text (which contained the literal string 'npm run dev'). Symptom signature: the quiet worker log contains ONLY 'session_id: <id>' and nothing else; the worker process is gone; and the git worktree is DIRTY with the worker's finished edits (tracked modifications + new files) and no commit. Diagnosis: compare the worker's own command line (/proc/<pid>/cmdline, ps -o cmd) against any -f pattern it passed to pkill/pgrep \u2014 kill -f matches the pattern anywhere in ANY process cmdline, including the wrapper. Recovery: this is the worker-interruption-recovery Step 2g family (edits INTACT) \u2014 do NOT re-dispatch a second identical prompt (it will die the same way at the same step); the foreman takes over: review the diff, re-run the acceptance evidence independently, commit early (commit before the long gates), then fire the gitreins judge and close the board. Prevention for future briefs: never instruct a worker to use `pkill -f <pattern>`; kill by PID (`kill <pid>`), or free a port with `fuser -k <port>/tcp`, and prefer matching a session-unique string or using a PID file.", "environment": "", "language": "bash", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "coding-hermes-worker-selfkill-pkill-pattern", "provider": "openrouter", "solved_at": "2026-09-16T06:13:52.706Z", "version": ""}

Generated from the verified corpus · MIT licensedBack to the catalog

coding-hermes-worker-selfkill-pkill-pattern

Answer 1

coding-hermes worker self-kill via pkill -f matching its own wrapper argv

1. Symptom

2. Root-cause analysis

Reproduction (verified in this sandbox)

Contributing factors

3. Exact fix

3a. Immediate recovery for tick 353 — do NOT re-dispatch

3b. Prevention — never put a pkill -f pattern in a brief

3c. Prevention — harden the hermes wrapper so briefs are never in argv

3d. Brief lint rule (enforce at dispatch)

4. Verification

4.1 Root cause is real

4.2 Kill-by-PID fix works

4.3 fuser -k <port>/tcp fix works

4.4 Interruption recovery works end-to-end

5. Rule of thumb

Evidence & signatures

Answer 2

coding-hermes worker self-kill via pkill -f matching its own wrapper argv

1. Symptom

2. Root-cause analysis

Reproduction (verified in this sandbox)

Contributing factors

3. Exact fix

3a. Immediate recovery for tick 353 — do NOT re-dispatch

3b. Prevention — never put a pkill -f pattern in a brief

3c. Prevention — harden the hermes wrapper so briefs are never in argv

3d. Brief lint rule (enforce at dispatch)

4. Verification

4.1 Root cause is real

4.2 Kill-by-PID fix works

4.3 fuser -k <port>/tcp fix works

4.4 Interruption recovery works end-to-end

5. Rule of thumb

Evidence & signatures

coding-hermes worker self-kill via `pkill -f` matching its own wrapper argv

3b. Prevention — never put a `pkill -f` pattern in a brief

4.3 `fuser -k <port>/tcp` fix works

coding-hermes worker self-kill via `pkill -f` matching its own wrapper argv

3b. Prevention — never put a `pkill -f` pattern in a brief

4.3 `fuser -k <port>/tcp` fix works