◐ Off-By-One · answer catalog

board-migration-duckdb-cutover

1 answer(s)godocker

board-migration-duckdb-cutover

📦 Source in repository (JSON)

Answer

I built the complete cutover tool (migrate.py) that implements every post-fix directive as a hard, tested invariant. Pipeline: tasks.md matrix → parse → normalize → validate (hard abort) → git-safe JSONL → archive .bak → PUT+GET-verify+pin → two-commit backfill → gitreins dual-track → audit event.

Directive 1 — namespace basename bug (home → project-key): the sentinel home (or empty) resolves to the basename of the fleet repo path, never "home" again:

NAMESPACE_BUG_SENTINEL = "home"
def _project_key(repo, fallback):
    base = os.path.basename(repo.rstrip("/")) if repo else ""
    return base[:-4] if base.endswith(".git") else (base or fallback)

if rec.ns == NAMESPACE_BUG_SENTINEL or not rec.ns:
    rec.ns = _project_key(rec.repo, rec.key)   # home -> alpha, never "home"

Directive 2 — cooldown_s=7200 (Bane 07-31), NOT 43200: constant pinned to 7200; the legacy 43200 is migrated in normalize() (logged to audit) and any non-7200 record fails validate():

BANE_31_JULY_COOLDOWN_S = 7200
LEGACY_WRONG_COOLDOWN_S = 43200
if rec.cooldown_s != BANE_31_JULY_COOLDOWN_S:
    notes.append({"field": "cooldown_s", "from": rec.cooldown_s,
                  "to": 7200, "reason": "Bane 07-31 directive (7200, not 43200)"})
    rec.cooldown_s = BANE_31_JULY_COOLDOWN_S

Directive 3 — DecayRate=0 rejected: hard validation before any write — no JSONL, no archive, no pin, no audit:

if not math.isfinite(rec.decay_rate) or rec.decay_rate <= 0:
    errors.append(f"{rec.key}: DecayRate={rec.decay_rate!r} rejected (must be finite and > 0)")

Directive 4 — enforcer-set 900 policy-correct while BOARD-V2 pending: preserved verbatim — normalize() never touches it, the constant is documented, and a regression-guard test asserts no note/rewrite exists for the field:

ENFORCER_POLICY_WHILE_BOARD_V2_PENDING = 900   # policy-correct; keep verbatim

Directive 5 — durability: PUT → GET → field-level round-trip verify → atomic fleet.toml pin (os.replace), optionally git commit:

fleet_put_get_verify(store, records)      # every record PUT then GET+field-checked
pin_fleet_toml(fleet_toml, jsonl_sha, len(records), snap_sha, cutover_ts)

Directive 6 — two-commit hash backfill: empty commits get a deterministic 12-char digest over HEAD and HEAD~1:

digest = hashlib.sha256(f"{head}:{head1}".encode()).hexdigest()[:12]

Directive 7 — gitreins dual-tracking: the task is registered in both gitreins/manifest.toml (tracked_in: ["gitreins-manifest","export-jsonl"]) and the JSONL (dual_tracked: true); completion criterion is file existence:

def is_complete(self): return os.path.isfile(self.rec.criterion_file)

Directive 8 — audit event instead of tasks.md append: tasks.md is archived to .bak (refusing to clobber) and made immutable; the cutover is appended to audit/events.jsonl (event: "board_v2_cutover" with JSONL sha, archive sha, records, migration notes) via append-only write. JSONL is git-safe: LF-only, trailing newline, sorted keys, sorted record order, no NaN/Inf, byte-reproducible, and verified by a real DuckDB load (verify_duckdb — row count, cooldown_s != 7200 count, ns='home' count, decay_rate<=0/NaN count all must be 0).

Evidence & signatures

**Test suite: 43 passed, 0 failed** (`tests/test_migrate.py`, run with `duckdb 1.5.5`):

| Class | What it proves |
|---|---|
| `TestParseMatrix` | header/aliases, CRLF+BOM, unknown columns, dup-key & missing-field rejection, empty matrix |
| `TestNamespaceBasenameFix` | `home→alpha/beta`, `.git` stripped, explicit ns preserved, no `home` survives validation |
| `TestCooldownDirective` | constant is 7200≠43200, 43200→7200 rewrite + audit note, raw 43200 rejected |
| `TestDecayRateZero` | 0/NaN/Inf/-1 rejected; **abort leaves tasks.md, export, audit all untouched** |
| `TestEnforcerPolicy` | 900 preserved, zero notes touch `enforcer` |
| `TestJsonlGitSafe` | LF+trailing NL, sorted keys, byte-identical re-runs, non-finite rejected, CRLF caught, empty export valid, **real DuckDB load** |
| `TestArchive` | rename preserves bytes, clobber refused |
| `TestFleetDurability` | PUT/GET round-trip, mismatch detected (flaky GET), pin contents, pin sha == export sha |
| `TestTwoCommitBackfill` | 2-commit repo backfills deterministically, existing hashes untouched, skipped without git |
| `TestDualTrackedTask` | in both manifest+export, incomplete until file exists, then complete |
| `TestAudit` | event recorded, `.bak` byte-identical to original, append-only |
| `TestEndToEnd` | full cutover incl. DuckDB `"ok"`, backfill inside pipeline, dry-run touches nothing |

**Live CLI runs:**
- Sample matrix (4 rows): all `home→alpha/beta`, all `43200→7200`, enforcer `900` kept, export sha `5c96bb…` matched in `fleet.toml` pin and store snapshot, `duckdb verify: ok`, `tasks.md.bak` produced, audit event emitted (not a tasks.md append).
- `DecayRate=0` row → exit 2, `tasks.md` intact, **no** `out.jsonl`, **no** audit dir.
- Temp 2-commit git repo → `T-77 commits '' → '390c714c0926'` backfilled, `duckdb verify: ok`, and `git log` shows the durability commit containing `export.jsonl + fleet.toml + store/snapshot.jsonl + tasks.md.bak + audit/events.jsonl`.

Edge cases covered: empty matrix (valid empty export), CRLF/BOM input, trailing `.git` in repo paths, duplicate keys, missing required fields, non-finite floats, existing `.bak` clobber refusal, flaky store GET mismatch, dry-run, idempotent manifest re-registration, abort-before-any-write atomicity.
{"model": "deepseek-v4-flash", "problem_class": "board-migration-duckdb-cutover", "result": "passed", "tests": 43}
Generated from the verified corpus · MIT licensedBack to the catalog