Scope: test-only change in packages/flight-search/tests/amadeus-http-client.test.ts.
The referenced heading monorepo was not present in this environment (only /workspace/problem.json), so I reconstructed the exact timing surface in /workspace/repro and verified the arithmetic empirically with vitest 4.1.10 (faked clock, RNG pinned, all 50 draws). The full solution is written to /workspace/SOLUTION.md and reproduced below.
INT-CI-298: a timing assertion that encoded the jitter drawScope: test-only change in packages/flight-search/tests/amadeus-http-client.test.ts.
Product code (packages/flight-search/src/amadeus-http-client.ts) is not changed.
The test configured totalTimeoutMs: 250 and asserted seen.length >= 2 ("the first retry happened"). The backoff is jittered: backoffMs(0) = 200 + floor(random*50) = 200..249 ms. After attempt 0's round-trip r, remaining budget is 250 − r:
draw j |
backoffMs(0) |
sleep | after | next gate |
|---|---|---|---|---|
0..48 |
200+j ≤ 248 |
full | ≈201..249 ms |
passes → 2nd fetch |
49 |
249 |
truncated to 249 |
=250 ms |
now >= deadline → AMAD_BUDGET_EXCEEDED after 1 fetch |
The outcome is a function of Math.random() and host-timed r, not retry semantics. Elapsed was ~250 ms either way, so only the assertion looked wrong. Pinning Math.random narrows the race but doesn't remove it, and stops exercising jitter — use it as the diagnostic only.
Let B = totalTimeoutMs, C = elapsed ceiling:
1. max(backoff0) + r < B → first retry always happens.
2. B < min(backoff2) = 800 → later backoff never fits, truncation path still exercised, terminal error stays the budget code.
3. B ≤ C < 200+400+800 = 1400 → ceiling can catch a truncation regression.
B = 700, C = 1250: 249 + r < 700 < 800, 700 ≤ 1250 < 1400. Fetch count is 2 or 3 for any draw; old C = 1500 sat above the 1400 ms untruncated floor and could not catch the regression.
- totalTimeoutMs: 250,
+ // INT-CI-298: arithmetic invariant, not a tuned timeout.
+ // backoffMs(0)=200..249, backoffMs(1)=400..449, backoffMs(2)=800..849.
+ // 700 > 249 + round-trip => first backoff always fits (retry guaranteed).
+ // 700 < 800 => later backoff always truncated; terminal error is
+ // AMAD_BUDGET_EXCEEDED, never AMAD_SERVER_ERROR.
+ // 1250 < untruncated schedule floor 1400ms => catches loss of truncation.
+ totalTimeoutMs: 700,
@@
- expect(elapsed).toBeLessThanOrEqual(1500);
+ expect(elapsed).toBeLessThanOrEqual(1250);
Run: pnpm --filter flight-search exec vitest run tests/amadeus-http-client.test.ts
Built a faithful reproduction in /workspace/repro (vitest 4.1.10, faked clock, 1 ms modeled round-trip) because the real repo was absent.
4.1 Old constants encode the draw: sweep draws 0..49 at B=250 → draws 0..48 pass (2 fetches), draw 49 fails (1 fetch), elapsed 250 ms — 49 pass / 1 fail.
4.2 New constants are invariant: B=700, C=1250 → 50 pass / 0 fail, fetch count ∈ {2,3}, elapsed 700 ms < 1250.
4.3 Mutation proof:
- A. disable truncation → elapsed: 1403 fails the 1250 ceiling (and 1403 < 1500, so the old ceiling would have missed it). ✅
- B. disable retry → {"fetchCount":1,"code":"AMAD_SERVER_ERROR"} instead of AMAD_BUDGET_EXCEEDED. ✅
Restore check: sha256sum -c → both files OK.
4.4 Self-contention: 5 concurrent copies + main suite → 6/6 green.
An assertion whose outcome moves with a random draw or host speed tests the wrong thing even when it passes. Make all inputs satisfy the assertion (B > max(backoff0)), keep the alternate path reachable (B < min(backoff2)), and put the ceiling below the schedule floor you're defending against (C < 1400). Never retry or skip the flaky test.
# Evidence - Problem class: timing-assertion-encodes-random-jitter-draw-instead-of-invariant - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-20T05:48:34.258Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "A test asserting 'the first retry happened (seen.length >= 2)' passes in isolation (3/3 focused runs, 17/17 each) but fails on the full battery: 'AssertionError: expected 1 to be greater than or equal to 2'. It looks statistical/flaky and invites a retry or a skip. ROOT CAUSE: the scenario's timing budget sat too close to a jittered backoff. backoffMs(0) = clamp(200 + floor(Math.random()*50), 200, 2000) = 200..249ms slept through a 'sleep but never past the remaining budget' helper against totalTimeoutMs: 250. When the draw exceeded the milliseconds of budget left after attempt 0's round-trip, the truncated sleep consumed the budget and the pre-attempt budget gate threw AMAD_BUDGET_EXCEEDED after ONE fetch. The assertion encoded the jitter draw, not the retry semantics; host load only decided which side of the race the clock landed on. CONFIRMATION (pin the randomness, do not retry the suite): monkeypatching Math.random across its whole range reproduces both outcomes exactly at a 250ms budget - jitter 0/10/25 -> 2 fetches PASS, jitter 49 -> 1 fetch FAIL, elapsed ~250ms in every case; a full sweep of draws 0..49 gives OLD budget 49 PASS / 1 FAIL, NEW budget 50 PASS / 0 FAIL. FIX (test-only, in the assertion's own terms): (1) widen the budget to a window where every timing path satisfies every assertion for ANY draw - the first backoff always fits (guaranteeing the retry the test is about) while a later, larger backoff can never fit behind the preceding attempts (so the truncation path is still exercised and the terminal error is still the budget code, not the retry-exhausted one); fetch count becomes 2 or 3 for any draw. (2) Tighten the elapsed ceiling so it sits BELOW the untruncated-schedule floor (~1400ms = 200+400+800) - the old 1500ms ceiling was above that floor and could not catch a truncation regression; 1250ms can. Document both assumptions in the test body. WHY NOT PIN THE JITTER AS THE FIX: pinning (vi.spyOn(Math,'random').mockReturnValue(0)) narrows the race rather than removing it - if the budget barely exceeds the pinned backoff, a late timer still eats the slack - and it stops exercising the jitter path; use pinning as the DIAGNOSTIC only. MUTATION PROOF (required): (A) disable the budget truncation -> the full backoff schedule runs ~1450ms and the elapsed bound fails; (B) disable the retry path -> the error-code assertion fails (AMAD_SERVER_ERROR instead of AMAD_BUDGET_EXCEEDED); restore byte-identical (sha256) and confirm green. LOAD EVIDENCE: use a bounded load generator (capped workers, capped seconds, PDEATHSIG teardown) rather than hand-rolled burn loops, which orphan to systemd --user and have driven hosts to loadavg 220+; where the generator correctly REFUSES to run on a shared host, generate self-contention instead - run 4-5 concurrent copies of the changed file while the full battery runs and count passes (20/20 here). TRANSFERABLE: an assertion whose outcome moves with a random draw or host speed tests the wrong thing even when it passes - convert it into an arithmetic invariant over the configured budgets and make the regression you fear land on a bound you control. A green CI read is not evidence the suite passes: this flake shipped with a Tier-2 verdict reporting '679/679 green' and a green pipeline, then surfaced on the first local full battery.", "environment": "vitest 4.1.10 in a pnpm workspace monorepo (node 22), shared host under fleet load", "language": "typescript", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "timing-assertion-encodes-random-jitter-draw-instead-of-invariant", "provider": "openrouter", "solved_at": "2026-09-20T05:48:34.258Z", "version": ""}The referenced heading monorepo was not present in this environment (only /workspace/problem.json), so I reconstructed the exact timing surface in /workspace/repro and verified the arithmetic empirically with vitest 4.1.10 (faked clock, RNG pinned, all 50 draws). The full solution is written to /workspace/SOLUTION.md and reproduced below.
INT-CI-298: a timing assertion that encoded the jitter drawScope: test-only change in packages/flight-search/tests/amadeus-http-client.test.ts.
Product code (packages/flight-search/src/amadeus-http-client.ts) is not changed.
The test configured totalTimeoutMs: 250 and asserted seen.length >= 2 ("the first retry happened"). The backoff is jittered: backoffMs(0) = 200 + floor(random*50) = 200..249 ms. After attempt 0's round-trip r, remaining budget is 250 − r:
draw j |
backoffMs(0) |
sleep | after | next gate |
|---|---|---|---|---|
0..48 |
200+j ≤ 248 |
full | ≈201..249 ms |
passes → 2nd fetch |
49 |
249 |
truncated to 249 |
=250 ms |
now >= deadline → AMAD_BUDGET_EXCEEDED after 1 fetch |
The outcome is a function of Math.random() and host-timed r, not retry semantics. Elapsed was ~250 ms either way, so only the assertion looked wrong. Pinning Math.random narrows the race but doesn't remove it, and stops exercising jitter — use it as the diagnostic only.
Let B = totalTimeoutMs, C = elapsed ceiling:
1. max(backoff0) + r < B → first retry always happens.
2. B < min(backoff2) = 800 → later backoff never fits, truncation path still exercised, terminal error stays the budget code.
3. B ≤ C < 200+400+800 = 1400 → ceiling can catch a truncation regression.
B = 700, C = 1250: 249 + r < 700 < 800, 700 ≤ 1250 < 1400. Fetch count is 2 or 3 for any draw; old C = 1500 sat above the 1400 ms untruncated floor and could not catch the regression.
- totalTimeoutMs: 250,
+ // INT-CI-298: arithmetic invariant, not a tuned timeout.
+ // backoffMs(0)=200..249, backoffMs(1)=400..449, backoffMs(2)=800..849.
+ // 700 > 249 + round-trip => first backoff always fits (retry guaranteed).
+ // 700 < 800 => later backoff always truncated; terminal error is
+ // AMAD_BUDGET_EXCEEDED, never AMAD_SERVER_ERROR.
+ // 1250 < untruncated schedule floor 1400ms => catches loss of truncation.
+ totalTimeoutMs: 700,
@@
- expect(elapsed).toBeLessThanOrEqual(1500);
+ expect(elapsed).toBeLessThanOrEqual(1250);
Run: pnpm --filter flight-search exec vitest run tests/amadeus-http-client.test.ts
Built a faithful reproduction in /workspace/repro (vitest 4.1.10, faked clock, 1 ms modeled round-trip) because the real repo was absent.
4.1 Old constants encode the draw: sweep draws 0..49 at B=250 → draws 0..48 pass (2 fetches), draw 49 fails (1 fetch), elapsed 250 ms — 49 pass / 1 fail.
4.2 New constants are invariant: B=700, C=1250 → 50 pass / 0 fail, fetch count ∈ {2,3}, elapsed 700 ms < 1250.
4.3 Mutation proof:
- A. disable truncation → elapsed: 1403 fails the 1250 ceiling (and 1403 < 1500, so the old ceiling would have missed it). ✅
- B. disable retry → {"fetchCount":1,"code":"AMAD_SERVER_ERROR"} instead of AMAD_BUDGET_EXCEEDED. ✅
Restore check: sha256sum -c → both files OK.
4.4 Self-contention: 5 concurrent copies + main suite → 6/6 green.
An assertion whose outcome moves with a random draw or host speed tests the wrong thing even when it passes. Make all inputs satisfy the assertion (B > max(backoff0)), keep the alternate path reachable (B < min(backoff2)), and put the ceiling below the schedule floor you're defending against (C < 1400). Never retry or skip the flaky test.
# Evidence - Problem class: timing-assertion-encodes-random-jitter-draw-instead-of-invariant - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-20T05:48:34.258Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "A test asserting 'the first retry happened (seen.length >= 2)' passes in isolation (3/3 focused runs, 17/17 each) but fails on the full battery: 'AssertionError: expected 1 to be greater than or equal to 2'. It looks statistical/flaky and invites a retry or a skip. ROOT CAUSE: the scenario's timing budget sat too close to a jittered backoff. backoffMs(0) = clamp(200 + floor(Math.random()*50), 200, 2000) = 200..249ms slept through a 'sleep but never past the remaining budget' helper against totalTimeoutMs: 250. When the draw exceeded the milliseconds of budget left after attempt 0's round-trip, the truncated sleep consumed the budget and the pre-attempt budget gate threw AMAD_BUDGET_EXCEEDED after ONE fetch. The assertion encoded the jitter draw, not the retry semantics; host load only decided which side of the race the clock landed on. CONFIRMATION (pin the randomness, do not retry the suite): monkeypatching Math.random across its whole range reproduces both outcomes exactly at a 250ms budget - jitter 0/10/25 -> 2 fetches PASS, jitter 49 -> 1 fetch FAIL, elapsed ~250ms in every case; a full sweep of draws 0..49 gives OLD budget 49 PASS / 1 FAIL, NEW budget 50 PASS / 0 FAIL. FIX (test-only, in the assertion's own terms): (1) widen the budget to a window where every timing path satisfies every assertion for ANY draw - the first backoff always fits (guaranteeing the retry the test is about) while a later, larger backoff can never fit behind the preceding attempts (so the truncation path is still exercised and the terminal error is still the budget code, not the retry-exhausted one); fetch count becomes 2 or 3 for any draw. (2) Tighten the elapsed ceiling so it sits BELOW the untruncated-schedule floor (~1400ms = 200+400+800) - the old 1500ms ceiling was above that floor and could not catch a truncation regression; 1250ms can. Document both assumptions in the test body. WHY NOT PIN THE JITTER AS THE FIX: pinning (vi.spyOn(Math,'random').mockReturnValue(0)) narrows the race rather than removing it - if the budget barely exceeds the pinned backoff, a late timer still eats the slack - and it stops exercising the jitter path; use pinning as the DIAGNOSTIC only. MUTATION PROOF (required): (A) disable the budget truncation -> the full backoff schedule runs ~1450ms and the elapsed bound fails; (B) disable the retry path -> the error-code assertion fails (AMAD_SERVER_ERROR instead of AMAD_BUDGET_EXCEEDED); restore byte-identical (sha256) and confirm green. LOAD EVIDENCE: use a bounded load generator (capped workers, capped seconds, PDEATHSIG teardown) rather than hand-rolled burn loops, which orphan to systemd --user and have driven hosts to loadavg 220+; where the generator correctly REFUSES to run on a shared host, generate self-contention instead - run 4-5 concurrent copies of the changed file while the full battery runs and count passes (20/20 here). TRANSFERABLE: an assertion whose outcome moves with a random draw or host speed tests the wrong thing even when it passes - convert it into an arithmetic invariant over the configured budgets and make the regression you fear land on a bound you control. A green CI read is not evidence the suite passes: this flake shipped with a Tier-2 verdict reporting '679/679 green' and a green pipeline, then surfaced on the first local full battery.", "environment": "vitest 4.1.10 in a pnpm workspace monorepo (node 22), shared host under fleet load", "language": "typescript", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "timing-assertion-encodes-random-jitter-draw-instead-of-invariant", "provider": "openrouter", "solved_at": "2026-09-20T05:48:34.258Z", "version": ""}