duckbrain-http-apikey-auth-rollout
The solution document is written to ~/SOLUTION-db-gap-031.md (178 lines).
Root cause (two layered causes + one trap):
Cause A: DB-GAP-031's --auth=apikey makes the server build its key table in memory at boot from ~/.duckbrain/auth.json. The foreman's pre-rollout credential was never in the new table, so every POST /api/memories returned 401 — regardless of transport. That's why both Bearer and X-API-Key were rejected: it isn't a header bug, the key itself isn't registered.
Cause B: Because there's no hot reload, minting the new key to disk (~/.duckbrain/auth.json) leaves the running server answering "Invalid API key" until systemctl --user restart duckbrain-http re-reads the store. This "rejection" is expected and is exactly why the restart is mandatory.
Trap: /health is permissive and returns OK even for keys /api/* rejects, so it's a false positive — health is not a valid write-auth probe (this masked the outage mid-tick).
The fix (exact commands):
1. node bin/duckbrain.js token --name=bunker-foreman --namespace=bunker → writes key to ~/.duckbrain/auth.json.
2. systemctl --user restart duckbrain-http (drop-in dbgap031-auth.conf already sets --auth=apikey) → reloads the in-memory table.
3. Send X-API-Key: $KEY (never Bearer), scoped to the bunker namespace.
Verification includes a negative control (bad key → 401), the real write-auth proof (POST /api/memories with the minted key → 2xx), a transport control (Bearer with the same key → 401), and explicitly excludes /health as a pass criterion.
The DuckBrain source and service aren't present in this sandbox (only the task spec), so the document is self-contained and directly runnable on the real host — I reasoned strictly from the described architecture and the exact commands given in the problem.
# Evidence - Problem class: duckbrain-http-apikey-auth-rollout - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-08-25T00:35:21.561Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "DuckBrain :3000 auth rollout (DB-GAP-031) broke foreman tick writes: POST /api/memories returned Unauthorized (Bearer + x-api-key both) mid-tick after the server restarted with --auth=apikey. Fix that worked: (1) mint a namespace-scoped token via `node bin/duckbrain.js token --name=bunker-foreman --namespace=bunker` (saved to ~/.duckbrain/auth.json); (2) the RUNNING server rejects the new key with Invalid API key (in-memory key table) \u2014 restart the user unit `systemctl --user restart duckbrain-http` (drop-in dbgap031-auth.conf); (3) send header X-API-Key (not Bearer). /health accepts the key even when /api/* rejects it, so health is NOT a write-auth probe.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "duckbrain-http-apikey-auth-rollout", "provider": "openrouter", "solved_at": "2026-08-25T00:35:21.561Z", "version": ""}The solution document is written to ~/SOLUTION-db-gap-031.md (178 lines).
Root cause (two layered causes + one trap):
Cause A: DB-GAP-031's --auth=apikey makes the server build its key table in memory at boot from ~/.duckbrain/auth.json. The foreman's pre-rollout credential was never in the new table, so every POST /api/memories returned 401 — regardless of transport. That's why both Bearer and X-API-Key were rejected: it isn't a header bug, the key itself isn't registered.
Cause B: Because there's no hot reload, minting the new key to disk (~/.duckbrain/auth.json) leaves the running server answering "Invalid API key" until systemctl --user restart duckbrain-http re-reads the store. This "rejection" is expected and is exactly why the restart is mandatory.
Trap: /health is permissive and returns OK even for keys /api/* rejects, so it's a false positive — health is not a valid write-auth probe (this masked the outage mid-tick).
The fix (exact commands):
1. node bin/duckbrain.js token --name=bunker-foreman --namespace=bunker → writes key to ~/.duckbrain/auth.json.
2. systemctl --user restart duckbrain-http (drop-in dbgap031-auth.conf already sets --auth=apikey) → reloads the in-memory table.
3. Send X-API-Key: $KEY (never Bearer), scoped to the bunker namespace.
Verification includes a negative control (bad key → 401), the real write-auth proof (POST /api/memories with the minted key → 2xx), a transport control (Bearer with the same key → 401), and explicitly excludes /health as a pass criterion.
The DuckBrain source and service aren't present in this sandbox (only the task spec), so the document is self-contained and directly runnable on the real host — I reasoned strictly from the described architecture and the exact commands given in the problem.
# Evidence - Problem class: duckbrain-http-apikey-auth-rollout - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-08-25T00:35:21.561Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "DuckBrain :3000 auth rollout (DB-GAP-031) broke foreman tick writes: POST /api/memories returned Unauthorized (Bearer + x-api-key both) mid-tick after the server restarted with --auth=apikey. Fix that worked: (1) mint a namespace-scoped token via `node bin/duckbrain.js token --name=bunker-foreman --namespace=bunker` (saved to ~/.duckbrain/auth.json); (2) the RUNNING server rejects the new key with Invalid API key (in-memory key table) \u2014 restart the user unit `systemctl --user restart duckbrain-http` (drop-in dbgap031-auth.conf); (3) send header X-API-Key (not Bearer). /health accepts the key even when /api/* rejects it, so health is NOT a write-auth probe.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "duckbrain-http-apikey-auth-rollout", "provider": "openrouter", "solved_at": "2026-08-25T00:35:21.561Z", "version": ""}