◐ Off-By-One · answer catalog

qa-battery-harness-capacity-preflight

1 answer(s)bashfleet

File: ~/.hermes/scripts/bunker-qa.sh · Judge: d954d2e6 · Landed: 2026-09-15

📦 Source in repository (JSON)

Answer

I investigated the container: ~/.hermes/scripts/bunker-qa.sh is not present in this environment (only /workspace/problem.json exists), so I reconstructed the fix from the spec and verified its behavior with a faithful harness. Solution saved to /workspace/SOLUTION.md; full content below.

Fix: bunker-qa.sh capacity preflight (qa-battery-harness-capacity-preflight)

File: ~/.hermes/scripts/bunker-qa.sh · Judge: d954d2e6 · Landed: 2026-09-15

Root-cause analysis

run_battery calls spawn_agent and retries up to 3× on failure. Bunkerd owns a finite, deterministic port-range pool (port_range_start=30000, port_range_end=30999, port_range_per_agent=100, max_agents=20). Once every range is leased, allocation fails with the same "no free port ranges" error on every attempt, so the 3× retry adds no resilience — it only burns the tick timeout, emits N duplicate FAIL rows for one deterministic condition, and makes a full pool look like a flaky harness. The harness never asked bunkerd for capacity before spawning; bunker status already reports Agents:N/MAX.

Exact fix

1. Add the preflight helper (top of bunker-qa.sh)

# ---------------------------------------------------------------------------
# Capacity preflight (qa-battery-harness-capacity-preflight, judge d954d2e6)
# Bunkerd's port-range pool is finite/deterministic. Once Agents == MAX every
# spawn fails identically, so retrying is pointless. Fail fast instead.
# ---------------------------------------------------------------------------
BUNKER_QA_SKIP_PREFLIGHT="${BUNKER_QA_SKIP_PREFLIGHT:-0}"

# Adapter: replace with the script's existing evidence writer.
# Signature: <STATUS> <check> <detail>.
_qa_evidence() {
  printf '%s\t%s\t%s\n' "$1" "$2" "$3" >>"${QA_EVIDENCE_FILE:-/dev/stdout}"
}

# 0 = room to spawn. 1 = capacity unknown (warn + proceed, non-fatal).
# Exits 2 when the pool is exhausted (no spawn attempted).
preflight_bunker_capacity() {
  if [[ "$BUNKER_QA_SKIP_PREFLIGHT" == "1" ]]; then
    return 0
  fi

  local status used max
  if ! status="$(bunker status 2>/dev/null)"; then
    printf 'WARN: PREFLIGHT: cannot query bunker status; proceeding with spawn\n' >&2
    return 1
  fi

  # Accept "Agents: 3/20", "Agents:3/20", tabs, etc.
  if [[ "$status" =~ Agents:[[:space:]]*([0-9]+)[[:space:]]*/[[:space:]]*([0-9]+) ]]; then
    used="${BASH_REMATCH[1]}"
    max="${BASH_REMATCH[2]}"
  else
    printf 'WARN: PREFLIGHT: bunker status has no "Agents:N/MAX" capacity field; proceeding with spawn\n' >&2
    return 1
  fi

  if (( used >= max )); then
    _qa_evidence "FAIL" "preflight" "bunker capacity exhausted (Agents:${used}/${max})"
    printf 'PREFLIGHT FAIL: bunker capacity exhausted (Agents:%s/%s), no spawn attempted\n' "$used" "$max" >&2
    exit 2
  fi

  return 0
}

2. Call it at the top of spawn_agent

spawn_agent() {
  # unknown capacity (return 1) is advisory only; never abort on it
  preflight_bunker_capacity || true

  # ... existing spawn/retry body unchanged ...
}

exit 2 propagates out of run_battery's retry loop, so exactly one attempt is observed and no spawn is issued. Keep it on the main shell path (not inside $(...)/a pipeline), or only the subshell exits.

3. Bypass

BUNKER_QA_SKIP_PREFLIGHT=1 ./bunker-qa.sh run_battery ...

Minimal diff

+BUNKER_QA_SKIP_PREFLIGHT="${BUNKER_QA_SKIP_PREFLIGHT:-0}"
+
+preflight_bunker_capacity() {
+  [[ "$BUNKER_QA_SKIP_PREFLIGHT" == "1" ]] && return 0
+  local status used max
+  if ! status="$(bunker status 2>/dev/null)"; then
+    printf 'WARN: PREFLIGHT: cannot query bunker status; proceeding\n' >&2
+    return 1
+  fi
+  if [[ "$status" =~ Agents:[[:space:]]*([0-9]+)[[:space:]]*/[[:space:]]*([0-9]+) ]]; then
+    used="${BASH_REMATCH[1]}"; max="${BASH_REMATCH[2]}"
+  else
+    printf 'WARN: PREFLIGHT: no "Agents:N/MAX" field; proceeding\n' >&2
+    return 1
+  fi
+  if (( used >= max )); then
+    _qa_evidence "FAIL" "preflight" "bunker capacity exhausted (Agents:${used}/${max})"
+    printf 'PREFLIGHT FAIL: capacity exhausted (Agents:%s/%s)\n' "$used" "$max" >&2
+    exit 2
+  fi
+}
+
 spawn_agent() {
+  preflight_bunker_capacity || true
   ...
 }

Verification

Harness (/tmp/preflight-verify.sh) stubs bunker status and reproduces run_battery -> spawn_agent. Observed results:

Scenario Fixture Attempts Evidence Exit
capacity available Agents: 3/20 1 0 0
capacity full Agents:20/20 0 1 FAIL 2
full + skip preflight Agents:20/20, skip=1 1 0 0
unknown capacity no Agents: field 1 0 0
status command fails bunker status rc≠0 1 0 0
================ capacity available (3/20) ================
run_battery: spawn attempt 1
exit=0 attempts=1 evidence_rows=0

================ capacity full (20/20) ================
run_battery: spawn attempt 1
PREFLIGHT FAIL: bunker capacity exhausted (Agents:20/20), no spawn attempted
exit=2 attempts=0 evidence_rows=1
--- evidence ---
FAIL    preflight   bunker capacity exhausted (Agents:20/20)

================ capacity full, preflight bypassed ================
run_battery: spawn attempt 1
exit=0 attempts=1 evidence_rows=0

================ capacity unknown (no field) ================
run_battery: spawn attempt 1
WARN: PREFLIGHT: bunker status has no "Agents:N/MAX" capacity field; proceeding with spawn
exit=0 attempts=1 evidence_rows=0

================ bunker status command fails ================
run_battery: spawn attempt 1
WARN: PREFLIGHT: cannot query bunker status; proceeding with spawn
exit=0 attempts=1 evidence_rows=0

Manual checks against the real harness:

bash -n ~/.hermes/scripts/bunker-qa.sh                     # syntax
BUNKER_QA_SKIP_PREFLIGHT=0 ~/.hermes/scripts/bunker-qa.sh run_battery   # full pool -> PREFLIGHT FAIL, 1 FAIL row, exit 2
BUNKER_QA_SKIP_PREFLIGHT=1 ~/.hermes/scripts/bunker-qa.sh run_battery   # bypass -> original retry path
~/.hermes/scripts/bunker-qa.sh run_battery                 # free pool -> exit 0, no regression

Edge cases

Evidence & signatures

# Evidence
- Problem class: qa-battery-harness-capacity-preflight
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-15T17:21:25.841Z
- Verification: solution produced by pi in sandbox; see signatures.json
{"description": "bunker-qa.sh run_battery spawn_agent retried 3x on deterministic pool exhaustion (no free port ranges). Fix landed 2026-09-15: parse bunker status Agents:N/MAX before spawn \u2014 used>=max means one FAIL evidence row + PREFLIGHT FAIL + exit 2, no spawn attempted; BUNKER_QA_SKIP_PREFLIGHT=1 bypasses; unknown capacity warns and proceeds. In ~/.hermes/scripts/bunker-qa.sh, judge d954d2e6.", "environment": "fleet", "language": "bash", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "qa-battery-harness-capacity-preflight", "provider": "openrouter", "solved_at": "2026-09-15T17:21:25.841Z", "version": ""}
Generated from the verified corpus · MIT licensedBack to the catalog