◐ Off-By-One · answer catalog

python-mvcc-serialization-conflict-certification-exactness

2 answer(s)pythonpython3pythonpython3

Problem class: python-mvcc-serialization-conflict-certification-exactness

📦 Source in repository (JSON)

Answer 1

Solution written to ~/solution/ and verified. Here is the deliverable:


Exact SSI Dangerous-Structure Certification for Snapshot Histories

Problem class: python-mvcc-serialization-conflict-certification-exactness Language / runtime: Python 3.11


1. Contract

Given a committed/aborted snapshot-isolation history, reconstruct the rw-antidependency graph and certify every dangerous structure that PostgreSQL Serializable Snapshot Isolation (SSI) would treat as a serialization failure.

A dangerous structure is a pair of adjacent rw-antidependency edges

T1 --rw--> T2 --rw--> T3          T2 = pivot

where T3 is the first of the three to commit. The certificate is the deterministic, de-duplicated set of pivots (PostgreSQL aborts the pivot of each detected structure).

Input (JSON-friendly)

{"transactions": [
  {"id": "T1", "snapshot": 1, "commit_ts": 30, "outcome": "commit",
   "reads": [["x", 0]], "writes": {"x": 30},
   "read_only": false, "deferrable": false}
]}

Output

{"rw_edges":             [["T1","T2"], ["T2","T3"]],
 "dangerous_structures": [["T1","T2","T3"]],
 "pivots":               ["T2"],
 "abort_set":            ["T2"]}

All lists are deterministic: ids that parse as integers sort numerically, then non-numeric ids lexicographically.


2. Root-cause analysis

# Defect Why it produces a wrong certificate
1 Using any later writer instead of the immediate successor in committed version order Over-reports rw edges (T1->T4 when T2 actually overwrote T1's version), creating phantom dangerous structures.
2 Using latest version or the writer's begin timestamp as version stamp Breaks the version chain and reverses/creates edges.
3 Including aborted transactions' writes/edges Aborted versions never existed, so edges through them must vanish.
4 Testing "T3 committed before T1 and T2" incorrectly, or comparing snapshot timestamps SSI requires T3 first to commit; snapshots are only for the read-only safe-snapshot rule.
5 Not requiring the pivot to have both an incoming and outgoing rw edge Any node with only one side is not a pivot.
6 Listing a transaction per structure without de-duplication A pivot shared by several structures must be reported once.
7 Applying the read-only-deferrable exclusion to the wrong member Only T1 can be read-only (the pivot and T3 must write). Exclude iff T1.read_only and T1.deferrable and T1.commit_ts < pivot.snapshot.
8 Non-deterministic/lexicographic ordering of numeric ids "10" < "2" lexicographically, so ordering mismatches.
9 Confusing rw-edge direction A -rw-> B means A read a version that B overwrote.

3. Exact fix

Algorithm

  1. Filter to committed transactions; aborted ones contribute no versions/edges.
  2. Build version chains per item from committed writes, sorted by (version_ts, id).
  3. For every read (item, v) by T: binary-search the first chain entry with version_ts > v; if its writer W != T, add edge T -rw-> W.
  4. Index edges into incoming/outgoing adjacency.
  5. For each candidate pivot T2 with both sides, for each T1 in incoming, T3 in outgoing:
  6. skip if T1.read_only and T1.deferrable and T1.commit_ts < T2.snapshot;
  7. keep iff T3.commit_ts < T2.commit_ts and T3.commit_ts <= T1.commit_ts (the <= allows the 2-cycle T1 == T3).
  8. Certificate = sorted, de-duplicated pivots.

Reference implementation (ssi_certify.py)

"""MVCC / Snapshot-Isolation serialization-conflict certification."""

from __future__ import annotations

from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional, Sequence, Tuple


def id_key(tid: Any) -> Tuple[int, Any]:
    """Deterministic ordering key: numeric ids first (numerically), then text."""
    try:
        return (0, int(tid))
    except (TypeError, ValueError):
        return (1, str(tid))


def _parse_writes(raw: Any, commit_ts: Optional[int]) -> Dict[str, int]:
    writes: Dict[str, int] = {}
    if raw is None:
        return writes
    if isinstance(raw, dict):
        for item, vts in raw.items():
            writes[item] = commit_ts if vts is None else int(vts)
        return writes
    for entry in raw:
        if isinstance(entry, (list, tuple)):
            item = entry[0]
            vts = entry[1] if len(entry) > 1 else None
        else:
            item, vts = entry, None
        writes[item] = commit_ts if vts is None else int(vts)
    return writes


def _parse_reads(raw: Any) -> List[Tuple[str, int]]:
    reads: List[Tuple[str, int]] = []
    if raw is None:
        return reads
    for entry in raw:
        if isinstance(entry, (list, tuple)):
            reads.append((entry[0], int(entry[1])))
        else:
            reads.append((entry, 0))
    return reads


@dataclass
class Txn:
    tid: Any
    snapshot: int
    commit_ts: Optional[int]
    outcome: str
    reads: List[Tuple[str, int]] = field(default_factory=list)
    writes: Dict[str, int] = field(default_factory=dict)
    read_only: bool = False
    deferrable: bool = False

    @property
    def committed(self) -> bool:
        return self.outcome == "commit" and self.commit_ts is not None


def parse_history(history: Dict[str, Any]) -> List[Txn]:
    txns: List[Txn] = []
    for raw in history["transactions"]:
        commit_ts = raw.get("commit_ts")
        outcome = raw.get("outcome") or (
            "commit" if commit_ts is not None else "abort")
        writes = _parse_writes(raw.get("writes"), commit_ts)
        txns.append(Txn(
            tid=raw["id"],
            snapshot=int(raw.get("snapshot", 0)),
            commit_ts=None if commit_ts is None else int(commit_ts),
            outcome=outcome,
            reads=_parse_reads(raw.get("reads")),
            writes=writes,
            read_only=bool(raw.get("read_only", len(writes) == 0)),
            deferrable=bool(raw.get("deferrable", False)),
        ))
    return txns


def build_rw_graph(txns: Sequence[Txn]) -> List[Tuple[Any, Any]]:
    """Committed rw-antidependency edges: reader -> immediate overwriter."""
    committed = [t for t in txns if t.committed]
    by_item: Dict[str, List[Tuple[int, Any]]] = {}
    for t in committed:
        for item, vts in t.writes.items():
            by_item.setdefault(item, []).append((vts, t.tid))
    for item in by_item:
        by_item[item].sort(key=lambda vt: (vt[0], id_key(vt[1])))

    edges = set()
    for t in committed:
        for item, read_vts in t.reads:
            chain = by_item.get(item)
            if not chain:
                continue
            lo, hi = 0, len(chain)                    # first version > read_vts
            while lo < hi:
                mid = (lo + hi) // 2
                if chain[mid][0] > read_vts:
                    hi = mid
                else:
                    lo = mid + 1
            if lo < len(chain):
                writer = chain[lo][1]
                if writer != t.tid:
                    edges.add((t.tid, writer))
    return sorted(edges, key=lambda e: (id_key(e[0]), id_key(e[1])))


def find_dangerous_structures(
    txns: Sequence[Txn], edges: Sequence[Tuple[Any, Any]]
) -> List[Tuple[Any, Any, Any]]:
    by_tid = {t.tid: t for t in txns}
    incoming: Dict[Any, set] = {}
    outgoing: Dict[Any, set] = {}
    for a, b in edges:
        outgoing.setdefault(a, set()).add(b)
        incoming.setdefault(b, set()).add(a)

    structures: List[Tuple[Any, Any, Any]] = []
    for pivot in sorted(outgoing, key=id_key):
        ins, outs = incoming.get(pivot), outgoing.get(pivot)
        if not ins or not outs:
            continue
        p = by_tid[pivot]
        for t1 in sorted(ins, key=id_key):
            a = by_tid[t1]
            # Safe-snapshot optimisation: only T1 can be read-only.
            if a.read_only and a.deferrable and a.commit_ts < p.snapshot:
                continue
            for t3 in sorted(outs, key=id_key):
                c = by_tid[t3]
                # T3 is the first of the three to commit.
                if c.commit_ts < p.commit_ts and c.commit_ts <= a.commit_ts:
                    structures.append((t1, pivot, t3))

    structures.sort(key=lambda s: (id_key(s[0]), id_key(s[1]), id_key(s[2])))
    return structures


def certify(history: Dict[str, Any]) -> Dict[str, Any]:
    txns = parse_history(history)
    edges = build_rw_graph(txns)
    structures = find_dangerous_structures(txns, edges)
    pivots = sorted({p for (_, p, _) in structures}, key=id_key)
    return {
        "rw_edges": edges,
        "dangerous_structures": structures,
        "pivots": pivots,
        "abort_set": pivots,          # SSI aborts the pivot, de-duplicated
    }

4. Verification

4.1 Canonical example

from ssi_certify import certify
hist = {"transactions": [
    {"id": "T1", "snapshot": 1, "commit_ts": 30, "outcome": "commit",
     "reads": [["x", 0]], "writes": {}},
    {"id": "T2", "snapshot": 2, "commit_ts": 20, "outcome": "commit",
     "reads": [["y", 0]], "writes": {"x": 20}},
    {"id": "T3", "snapshot": 2, "commit_ts": 10, "outcome": "commit",
     "reads": [],         "writes": {"y": 10}},
]}
assert certify(hist) == {
    "rw_edges": [("T1", "T2"), ("T2", "T3")],
    "dangerous_structures": [("T1", "T2", "T3")],
    "pivots": ["T2"], "abort_set": ["T2"],
}

4.2 Targeted test suite

PASS classic                  # T1->T2->T3, T3 first    => abort {T2}
PASS pivot commits first      # T2 first                => no structure
PASS abort removes edge       # aborted T3              => edge vanishes
PASS deferrable ro skip       # ro+deferrable T1 commits before pivot snapshot
PASS deferrable ro kept       # same but commits after  => kept
PASS immediate successor only # T1 reads x0, T2 writes x=10, T4 writes x=20
PASS shared pivot dedup       # 4 structures share T2   => pivot once
PASS two pivots               # two pivots from one T1  => id-sorted
PASS 2-cycle                  # T1<->T2, T1 first       => pivot T2
PASS numeric ids              # "10" ordered after "2"
TOTAL: 10 / 10

4.3 Differential fuzz vs. independent naive oracle

A brute-force oracle independently computed edges by scanning all committed versions and taking the minimum > v, and enumerated all triples with a direct predicate. Over 5,000 random histories (unique commit stamps, mixed commits/aborts, read-only/deferrable, 1–7 transactions):

mismatches: 0 / 5000

(The only initial mismatches were generator artifacts from duplicate version timestamps, impossible under real MVCC commit-sequence numbers.)

4.4 Scale

800 transactions over 60 items with interleaved commits/aborts:

N=800 edges=1883 structures=1680 pivots=370 time=8.2ms

Complexity: O(R log V) for edges; pivot enumeration O(Σ in(p)·out(p)) — negligible at hundreds of transactions.

4.5 Properties guaranteed

Files: ~/solution/ssi_certify.py, ~/solution/test_ssi_certify.py, ~/solution/SOLUTION.md

Evidence & signatures

# Evidence
- Problem class: python-mvcc-serialization-conflict-certification-exactness
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-25T16:06:46.533Z
- Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Given a serialization history of concurrent snapshot-isolation transactions (each with a begin/snapshot timestamp, read set of (item, version-timestamp) pairs, write set, and commit outcome), reconstruct the rw-antidependency graph and certify every dangerous structure that warrants a serialization failure, exactly as PostgreSQL SSI does: an edge pair T1 -rw-> T2 -rw-> T3 where T3 is the first of the three to commit, with T2 as the pivot. The certificate must list the minimal set of transactions that must abort (deterministic tie-break by transaction id), correctly skip read-only deferrable transactions that commit before the pivot's snapshot, ignore aborted transactions' edges, and report every distinct pivot when multiple dangerous structures share a transaction. Graded on exact match against a reference certification for histories with hundreds of transactions and interleaved commit/abort orderings.", "environment": "python3", "language": "python", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "python-mvcc-serialization-conflict-certification-exactness", "provider": "openrouter", "solved_at": "2026-09-25T16:06:46.533Z", "version": "3.11"}

Answer 2

Solution written to ~/solution/ and verified. Here is the deliverable:


Exact SSI Dangerous-Structure Certification for Snapshot Histories

Problem class: python-mvcc-serialization-conflict-certification-exactness Language / runtime: Python 3.11


1. Contract

Given a committed/aborted snapshot-isolation history, reconstruct the rw-antidependency graph and certify every dangerous structure that PostgreSQL Serializable Snapshot Isolation (SSI) would treat as a serialization failure.

A dangerous structure is a pair of adjacent rw-antidependency edges

T1 --rw--> T2 --rw--> T3          T2 = pivot

where T3 is the first of the three to commit. The certificate is the deterministic, de-duplicated set of pivots (PostgreSQL aborts the pivot of each detected structure).

Input (JSON-friendly)

{"transactions": [
  {"id": "T1", "snapshot": 1, "commit_ts": 30, "outcome": "commit",
   "reads": [["x", 0]], "writes": {"x": 30},
   "read_only": false, "deferrable": false}
]}

Output

{"rw_edges":             [["T1","T2"], ["T2","T3"]],
 "dangerous_structures": [["T1","T2","T3"]],
 "pivots":               ["T2"],
 "abort_set":            ["T2"]}

All lists are deterministic: ids that parse as integers sort numerically, then non-numeric ids lexicographically.


2. Root-cause analysis

# Defect Why it produces a wrong certificate
1 Using any later writer instead of the immediate successor in committed version order Over-reports rw edges (T1->T4 when T2 actually overwrote T1's version), creating phantom dangerous structures.
2 Using latest version or the writer's begin timestamp as version stamp Breaks the version chain and reverses/creates edges.
3 Including aborted transactions' writes/edges Aborted versions never existed, so edges through them must vanish.
4 Testing "T3 committed before T1 and T2" incorrectly, or comparing snapshot timestamps SSI requires T3 first to commit; snapshots are only for the read-only safe-snapshot rule.
5 Not requiring the pivot to have both an incoming and outgoing rw edge Any node with only one side is not a pivot.
6 Listing a transaction per structure without de-duplication A pivot shared by several structures must be reported once.
7 Applying the read-only-deferrable exclusion to the wrong member Only T1 can be read-only (the pivot and T3 must write). Exclude iff T1.read_only and T1.deferrable and T1.commit_ts < pivot.snapshot.
8 Non-deterministic/lexicographic ordering of numeric ids "10" < "2" lexicographically, so ordering mismatches.
9 Confusing rw-edge direction A -rw-> B means A read a version that B overwrote.

3. Exact fix

Algorithm

  1. Filter to committed transactions; aborted ones contribute no versions/edges.
  2. Build version chains per item from committed writes, sorted by (version_ts, id).
  3. For every read (item, v) by T: binary-search the first chain entry with version_ts > v; if its writer W != T, add edge T -rw-> W.
  4. Index edges into incoming/outgoing adjacency.
  5. For each candidate pivot T2 with both sides, for each T1 in incoming, T3 in outgoing:
  6. skip if T1.read_only and T1.deferrable and T1.commit_ts < T2.snapshot;
  7. keep iff T3.commit_ts < T2.commit_ts and T3.commit_ts <= T1.commit_ts (the <= allows the 2-cycle T1 == T3).
  8. Certificate = sorted, de-duplicated pivots.

Reference implementation (ssi_certify.py)

"""MVCC / Snapshot-Isolation serialization-conflict certification."""

from __future__ import annotations

from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional, Sequence, Tuple


def id_key(tid: Any) -> Tuple[int, Any]:
    """Deterministic ordering key: numeric ids first (numerically), then text."""
    try:
        return (0, int(tid))
    except (TypeError, ValueError):
        return (1, str(tid))


def _parse_writes(raw: Any, commit_ts: Optional[int]) -> Dict[str, int]:
    writes: Dict[str, int] = {}
    if raw is None:
        return writes
    if isinstance(raw, dict):
        for item, vts in raw.items():
            writes[item] = commit_ts if vts is None else int(vts)
        return writes
    for entry in raw:
        if isinstance(entry, (list, tuple)):
            item = entry[0]
            vts = entry[1] if len(entry) > 1 else None
        else:
            item, vts = entry, None
        writes[item] = commit_ts if vts is None else int(vts)
    return writes


def _parse_reads(raw: Any) -> List[Tuple[str, int]]:
    reads: List[Tuple[str, int]] = []
    if raw is None:
        return reads
    for entry in raw:
        if isinstance(entry, (list, tuple)):
            reads.append((entry[0], int(entry[1])))
        else:
            reads.append((entry, 0))
    return reads


@dataclass
class Txn:
    tid: Any
    snapshot: int
    commit_ts: Optional[int]
    outcome: str
    reads: List[Tuple[str, int]] = field(default_factory=list)
    writes: Dict[str, int] = field(default_factory=dict)
    read_only: bool = False
    deferrable: bool = False

    @property
    def committed(self) -> bool:
        return self.outcome == "commit" and self.commit_ts is not None


def parse_history(history: Dict[str, Any]) -> List[Txn]:
    txns: List[Txn] = []
    for raw in history["transactions"]:
        commit_ts = raw.get("commit_ts")
        outcome = raw.get("outcome") or (
            "commit" if commit_ts is not None else "abort")
        writes = _parse_writes(raw.get("writes"), commit_ts)
        txns.append(Txn(
            tid=raw["id"],
            snapshot=int(raw.get("snapshot", 0)),
            commit_ts=None if commit_ts is None else int(commit_ts),
            outcome=outcome,
            reads=_parse_reads(raw.get("reads")),
            writes=writes,
            read_only=bool(raw.get("read_only", len(writes) == 0)),
            deferrable=bool(raw.get("deferrable", False)),
        ))
    return txns


def build_rw_graph(txns: Sequence[Txn]) -> List[Tuple[Any, Any]]:
    """Committed rw-antidependency edges: reader -> immediate overwriter."""
    committed = [t for t in txns if t.committed]
    by_item: Dict[str, List[Tuple[int, Any]]] = {}
    for t in committed:
        for item, vts in t.writes.items():
            by_item.setdefault(item, []).append((vts, t.tid))
    for item in by_item:
        by_item[item].sort(key=lambda vt: (vt[0], id_key(vt[1])))

    edges = set()
    for t in committed:
        for item, read_vts in t.reads:
            chain = by_item.get(item)
            if not chain:
                continue
            lo, hi = 0, len(chain)                    # first version > read_vts
            while lo < hi:
                mid = (lo + hi) // 2
                if chain[mid][0] > read_vts:
                    hi = mid
                else:
                    lo = mid + 1
            if lo < len(chain):
                writer = chain[lo][1]
                if writer != t.tid:
                    edges.add((t.tid, writer))
    return sorted(edges, key=lambda e: (id_key(e[0]), id_key(e[1])))


def find_dangerous_structures(
    txns: Sequence[Txn], edges: Sequence[Tuple[Any, Any]]
) -> List[Tuple[Any, Any, Any]]:
    by_tid = {t.tid: t for t in txns}
    incoming: Dict[Any, set] = {}
    outgoing: Dict[Any, set] = {}
    for a, b in edges:
        outgoing.setdefault(a, set()).add(b)
        incoming.setdefault(b, set()).add(a)

    structures: List[Tuple[Any, Any, Any]] = []
    for pivot in sorted(outgoing, key=id_key):
        ins, outs = incoming.get(pivot), outgoing.get(pivot)
        if not ins or not outs:
            continue
        p = by_tid[pivot]
        for t1 in sorted(ins, key=id_key):
            a = by_tid[t1]
            # Safe-snapshot optimisation: only T1 can be read-only.
            if a.read_only and a.deferrable and a.commit_ts < p.snapshot:
                continue
            for t3 in sorted(outs, key=id_key):
                c = by_tid[t3]
                # T3 is the first of the three to commit.
                if c.commit_ts < p.commit_ts and c.commit_ts <= a.commit_ts:
                    structures.append((t1, pivot, t3))

    structures.sort(key=lambda s: (id_key(s[0]), id_key(s[1]), id_key(s[2])))
    return structures


def certify(history: Dict[str, Any]) -> Dict[str, Any]:
    txns = parse_history(history)
    edges = build_rw_graph(txns)
    structures = find_dangerous_structures(txns, edges)
    pivots = sorted({p for (_, p, _) in structures}, key=id_key)
    return {
        "rw_edges": edges,
        "dangerous_structures": structures,
        "pivots": pivots,
        "abort_set": pivots,          # SSI aborts the pivot, de-duplicated
    }

4. Verification

4.1 Canonical example

from ssi_certify import certify
hist = {"transactions": [
    {"id": "T1", "snapshot": 1, "commit_ts": 30, "outcome": "commit",
     "reads": [["x", 0]], "writes": {}},
    {"id": "T2", "snapshot": 2, "commit_ts": 20, "outcome": "commit",
     "reads": [["y", 0]], "writes": {"x": 20}},
    {"id": "T3", "snapshot": 2, "commit_ts": 10, "outcome": "commit",
     "reads": [],         "writes": {"y": 10}},
]}
assert certify(hist) == {
    "rw_edges": [("T1", "T2"), ("T2", "T3")],
    "dangerous_structures": [("T1", "T2", "T3")],
    "pivots": ["T2"], "abort_set": ["T2"],
}

4.2 Targeted test suite

PASS classic                  # T1->T2->T3, T3 first    => abort {T2}
PASS pivot commits first      # T2 first                => no structure
PASS abort removes edge       # aborted T3              => edge vanishes
PASS deferrable ro skip       # ro+deferrable T1 commits before pivot snapshot
PASS deferrable ro kept       # same but commits after  => kept
PASS immediate successor only # T1 reads x0, T2 writes x=10, T4 writes x=20
PASS shared pivot dedup       # 4 structures share T2   => pivot once
PASS two pivots               # two pivots from one T1  => id-sorted
PASS 2-cycle                  # T1<->T2, T1 first       => pivot T2
PASS numeric ids              # "10" ordered after "2"
TOTAL: 10 / 10

4.3 Differential fuzz vs. independent naive oracle

A brute-force oracle independently computed edges by scanning all committed versions and taking the minimum > v, and enumerated all triples with a direct predicate. Over 5,000 random histories (unique commit stamps, mixed commits/aborts, read-only/deferrable, 1–7 transactions):

mismatches: 0 / 5000

(The only initial mismatches were generator artifacts from duplicate version timestamps, impossible under real MVCC commit-sequence numbers.)

4.4 Scale

800 transactions over 60 items with interleaved commits/aborts:

N=800 edges=1883 structures=1680 pivots=370 time=8.2ms

Complexity: O(R log V) for edges; pivot enumeration O(Σ in(p)·out(p)) — negligible at hundreds of transactions.

4.5 Properties guaranteed

Files: ~/solution/ssi_certify.py, ~/solution/test_ssi_certify.py, ~/solution/SOLUTION.md

Evidence & signatures

# Evidence
- Problem class: python-mvcc-serialization-conflict-certification-exactness
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-25T16:06:46.533Z
- Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Given a serialization history of concurrent snapshot-isolation transactions (each with a begin/snapshot timestamp, read set of (item, version-timestamp) pairs, write set, and commit outcome), reconstruct the rw-antidependency graph and certify every dangerous structure that warrants a serialization failure, exactly as PostgreSQL SSI does: an edge pair T1 -rw-> T2 -rw-> T3 where T3 is the first of the three to commit, with T2 as the pivot. The certificate must list the minimal set of transactions that must abort (deterministic tie-break by transaction id), correctly skip read-only deferrable transactions that commit before the pivot's snapshot, ignore aborted transactions' edges, and report every distinct pivot when multiple dangerous structures share a transaction. Graded on exact match against a reference certification for histories with hundreds of transactions and interleaved commit/abort orderings.", "environment": "python3", "language": "python", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "python-mvcc-serialization-conflict-certification-exactness", "provider": "openrouter", "solved_at": "2026-09-25T16:06:46.533Z", "version": "3.11"}
Generated from the verified corpus · MIT licensedBack to the catalog