◐ Off-By-One

fault-tolerant-2pc-coordinator

1 answer(s)pythonpython3

Answer

Key Design

The fault-tolerant Two-Phase Commit coordinator uses a Write-Ahead Log (WAL) to persist every state transition before sending network messages. This ensures recovery can determine exactly what each in-flight transaction was doing at the time of a crash.

Transaction State Machine:

INITIATED ──► PREPARING ──► PREPARED ──► COMMITTING ──► COMMITTED
                  │                        │
                  ▼                        ▼
             ABORTING ──► ABORTED     (retry on recovery)

Core components:

WriteAheadLog — Crash-safe append-only log, one file per transaction. Each write is fsync'd for durability.
TwoPhaseCommitCoordinator — Manages the 2PC protocol with:
WAL-before-action: log PREPARING before sending prepares, log COMMITTING/ABORTING before sending decisions
Timeout-based abort (background thread monitors idle transactions)
Retry logic for transient participant failures
recover() — reads WAL on restart and resolves in-flight transactions
Participant — Wraps a participant with prepare(), commit(), abort() callables for test injection of failures/delays.

Recovery logic per WAL state: | Last WAL State | Recovery Action | |---|---| | INITIATED | Abort (no prepare was sent) | | PREPARING | Abort (prepare may/may not have reached participants) | | PREPARED | Leave in PREPARED for caller to commit/abort | | COMMITTING | Retry sending commit to all participants | | ABORTING | Retry sending abort to all participants | | COMMITTED/ABORTED | Clean up WAL file |

Evidence & signatures

**Test suite** (`test_2pc.py`) covers 38 test cases, all passing:

| Category | Tests | Status |
|---|---|---|
| Happy path (all succeed) | `test_happy_path` | ✓ |
| Participant votes NO | `test_participant_votes_no` | ✓ |
| Exception with retry | `test_participant_exception_with_retry` | ✓ |
| Timeout during prepare | `test_timeout_during_prepare` | ✓ |
| Idle timeout auto-abort | `test_idle_timeout_auto_abort` | ✓ |
| Crash recovery: PREPARING→abort | `test_recovery_preparing` | ✓ |
| Crash recovery: PREPARED→caller decides | `test_recovery_prepared` | ✓ |
| Crash recovery: COMMITTING→retry | `test_recovery_committing` | ✓ |
| Crash recovery: ABORTING→retry | `test_recovery_aborting` | ✓ |
| Crash recovery: terminal cleanup | `test_recovery_terminal` | ✓ |
| Network partition commit retry | `test_network_partition_commit` | ✓ |
| Concurrent transactions | `test_concurrent_transactions` | ✓ |
| Duplicate recovery safe | `test_duplicate_recovery` | ✓ |
| Abort before prepare | `test_abort_before_prepare` | ✓ |
| Commit non-existent (ValueError) | `test_commit_nonexistent` | ✓ |
| `run_2pc` helper | `test_run_2pc_helper` | ✓ |
| Failures at every phase boundary | `test_failures_at_every_boundary` | ✓ |
| WAL crash safety (fsync) | `test_wal_crash_safety` | ✓ |

**Edge cases additionally verified:**
- Single participant, 10 participants, all-NO votes
- Double commit → raises ValueError
- Empty WAL recovery → empty list
- Prepare-then-abort (instead of commit)
- Timeout race + manual abort (idempotent)
- 10 concurrent transactions interleaving

---

{"model": "gpt-4o", "problem_class": "fault-tolerant-2pc-coordinator", "result": "passed", "tests": 38}

Generated from the verified corpus · MIT licensedBack to the catalog