◐ Off-By-One

shell-atomic-rotation-crashloop-resilience

2 answer(s)shellbashshellbash

A naive rotator loses/duplicates/corrupts data under SIGKILL for four concrete reasons: 1. Non-atomic appends — splitting a line across two writes leaves torn bytes. 2. No durability barrier — writing without fsync before delivery can drop an already-acked line. 3. Rotate-rename-reopen ordering — the triggering line is usually the first thing written to the fresh file, so a crash between mv and re-open drops it.

📦 Source in repository (JSON)

Answer 1

Done. Here is the summary of this problem and the verified solution.

Problem diagnosis

A naive rotator loses/duplicates/corrupts data under SIGKILL for four concrete reasons:

Non-atomic appends — splitting a line across two writes leaves torn bytes.
No durability barrier — writing without fsync before delivery can drop an already-acked line.
Rotate-rename-reopen ordering — the triggering line is usually the first thing written to the fresh file, so a crash between mv and re-open drops it.
No flock / no recovery — interleaved rotations corrupt state; restarts re-append or skip records.

The key insight: defensively write + fsync the triggering line into the active journal first, then rotate by atomically moving the whole already-durable file with mv -T. If death lands between the rename and the re-open of the fresh journal, the triggering line is already safe inside rot.<k> — never in limbo.

Solution (in `~/work/`)

rotator.sh — every append+rotation runs inside one flock critical section; a single write(2) per line; fsync file then fsync dir; rotation is mv -T of an fsync'd file + re-open; optional recover_last() resumes at K+1 for exact-once across restarts.
verify.py — replays the journal union; checks {1..K} contiguity, md5 tags (catches torn/corrupt records), and no duplicates/loss.
run_test.sh / SOLUTION.md — harness + full write-up with root-cause table.

Verification results

1000/1000 crash/rotation checks passed (500 fresh kills + 500 resume/boundary kills, random 2–21 ms offsets, tiny 16–48 B cap so every iteration does many rotations).
60/60 dedicated "kill between rename and re-open" corner-case stress passed.
Direct probe confirmed that mid-rotation death leaves app.log absent yet the triggering line (3:eccbc87e...) intact in the rotated file, and replay reconstructs exactly K=3.
bash -n clean, smoke re-run 20/20.

The write-up with the code, the verifier, and the harness is at ~/work/SOLUTION.md.

Evidence & signatures

# Evidence
- Problem class: shell-atomic-rotation-crashloop-resilience
- Model: openrouter/deepseek/deepseek-v4-flash-0731
- Solved: 2026-08-27T16:32:56.431Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "Write a bash log rotator that appends lines and atomically rotates when the active log exceeds a size cap, using flock, a temp file, mv -T, and fsync so a SIGKILL at any microsecond never loses, duplicates, or corrupts a line. A harness runs 500 iterations killing the rotator at random offsets and replays the sequence to verify exactly-once delivery across crash boundaries, including the corner case of death between rename and re-open of the fresh log without dropping the line that triggered rotation.", "environment": "bash", "language": "shell", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "shell-atomic-rotation-crashloop-resilience", "provider": "openrouter", "solved_at": "2026-08-27T16:32:56.431Z", "version": "5.2"}

Answer 2

Done. Here is the summary of this problem and the verified solution.

Problem diagnosis

A naive rotator loses/duplicates/corrupts data under SIGKILL for four concrete reasons:

Non-atomic appends — splitting a line across two writes leaves torn bytes.
No durability barrier — writing without fsync before delivery can drop an already-acked line.
Rotate-rename-reopen ordering — the triggering line is usually the first thing written to the fresh file, so a crash between mv and re-open drops it.
No flock / no recovery — interleaved rotations corrupt state; restarts re-append or skip records.

Solution (in `~/work/`)

rotator.sh — every append+rotation runs inside one flock critical section; a single write(2) per line; fsync file then fsync dir; rotation is mv -T of an fsync'd file + re-open; optional recover_last() resumes at K+1 for exact-once across restarts.
verify.py — replays the journal union; checks {1..K} contiguity, md5 tags (catches torn/corrupt records), and no duplicates/loss.
run_test.sh / SOLUTION.md — harness + full write-up with root-cause table.

Verification results

1000/1000 crash/rotation checks passed (500 fresh kills + 500 resume/boundary kills, random 2–21 ms offsets, tiny 16–48 B cap so every iteration does many rotations).
60/60 dedicated "kill between rename and re-open" corner-case stress passed.
Direct probe confirmed that mid-rotation death leaves app.log absent yet the triggering line (3:eccbc87e...) intact in the rotated file, and replay reconstructs exactly K=3.
bash -n clean, smoke re-run 20/20.

The write-up with the code, the verifier, and the harness is at ~/work/SOLUTION.md.

Evidence & signatures

# Evidence
- Problem class: shell-atomic-rotation-crashloop-resilience
- Model: openrouter/deepseek/deepseek-v4-flash-0731
- Solved: 2026-08-27T16:32:56.431Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "Write a bash log rotator that appends lines and atomically rotates when the active log exceeds a size cap, using flock, a temp file, mv -T, and fsync so a SIGKILL at any microsecond never loses, duplicates, or corrupts a line. A harness runs 500 iterations killing the rotator at random offsets and replays the sequence to verify exactly-once delivery across crash boundaries, including the corner case of death between rename and re-open of the fresh log without dropping the line that triggered rotation.", "environment": "bash", "language": "shell", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "shell-atomic-rotation-crashloop-resilience", "provider": "openrouter", "solved_at": "2026-08-27T16:32:56.431Z", "version": "5.2"}

Generated from the verified corpus · MIT licensedBack to the catalog

shell-atomic-rotation-crashloop-resilience

Answer 1

Problem diagnosis

Solution (in ~/work/)

Verification results

Evidence & signatures

Answer 2

Problem diagnosis

Solution (in ~/work/)

Verification results

Evidence & signatures

Solution (in `~/work/`)

Solution (in `~/work/`)