perl -pi -e 's/gpt-5.1/openai\/gpt-4o-mini/g' \
Bug (MAF-GAP-016): The CLI hardcoded a hallucinated default model gpt-5.1 (does not exist) in 5 source sites + 3 test files across run-game.ts / config.ts / init.ts, and the value violated the project's provider/model convention (openai/gpt-4o-mini). The CLI was sending a bare model name instead of a qualified provider/model string.
Fix: two global, case-sensitive replacements (order matters — the label contains GPT-5.1 with a capital G, so it survives replacement #1 and is fixed by #2):
# 1) bare hallucinated model -> provider/model qualified name (5 src + 3 test sites)
perl -pi -e 's/gpt-5\.1/openai\/gpt-4o-mini/g' \
src/run-game.ts src/config.ts src/init.ts \
test/run-game.test.ts test/config.test.ts test/init.test.ts
# 2) choice label
perl -pi -e 's/GPT-5\.1 \(Recommended\)/GPT-4o mini (Recommended)/g' src/run-game.ts
Before/after per site:
// src/config.ts
- export const DEFAULT_MODEL = "gpt-5.1";
+ export const DEFAULT_MODEL = "openai/gpt-4o-mini";
// src/run-game.ts (choice list + CLI default/help)
- { value: "gpt-5.1", label: "GPT-5.1 (Recommended)" },
+ { value: "openai/gpt-4o-mini", label: "GPT-4o mini (Recommended)" },
- export const HELP_TEXT = "--model <provider/model> (default: gpt-5.1)";
+ export const HELP_TEXT = "--model <provider/model> (default: openai/gpt-4o-mini)";
- export async function selectModel(model: string = "gpt-5.1"): Promise<string> {
+ export async function selectModel(model: string = "openai/gpt-4o-mini"): Promise<string> {
// src/init.ts (generated config template)
- { model: "gpt-5.1", timeoutMs: 30_000 },
+ { model: "openai/gpt-4o-mini", timeoutMs: 30_000 },
// test/*.test.ts — assertions updated to expect "openai/gpt-4o-mini"
Trapping pitfall: never run npx vitest run from the repo root — there is no root vitest config. Always cd apps/cli first.
**Environment caveat (transparency):** this sandbox does not contain the live benchmark checkout — `/workspace` holds only `problem.json`, and there is no network. I therefore reproduced the fix on a faithful reconstruction at `/tmp/maf-gap-016-demo` with the identical affected shape (8 occurrences: 5 source sites in `run-game.ts`×3, `config.ts`×1, `init.ts`×1 + 3 test files×1), applied the exact commands above, and executed the checks below. Real-repo results from the foreman's recorded verification (problem statement) are cited where noted. | Check | Result | |---|---| | `grep -ro "gpt-5.1" src test` before fix | 8 hits (5 src + 3 test) | | `grep -rin "gpt-5.1"` after fix, whole tree (case-insensitive) | **0 hits** (rc=1) | | `grep -rn "GPT-5.1"` after fix | **0 hits** | | `--help` default (HELP_TEXT smoke) | `--model <provider/model> (default: openai/gpt-4o-mini)` | | Runtime assertions (tsx, node smoke): `DEFAULT_MODEL`, `resolveModel()`, choice label `GPT-4o mini (Recommended)`, every choice value matches `/` (provider/model format), init template model | **all passed** | | `npx vitest run` at repo root | no root `vitest.config.*` exists — must run from `apps/cli` (matches foreman's hang report) | | `cd apps/cli && vitest run` | **3 test files / 3 tests passed** (rc=0) | | Real repo suite (foreman, recorded) | 83 tests: 80 pass + 3 skip on dead `TEST_BASE_URL` | Edge cases covered: case-sensitivity ordering (label `GPT-5.1` vs value `gpt-5.1` — label replacement runs second so it isn't clobbered); pre-existing `openai/gpt-4o-mini` choice left untouched by replacement #1; both `selectModel` default arg and `resolveModel` fallback paths return the qualified default; generated config template and help text both qualified.
{"model": "deepseek-v4-flash", "problem_class": "typescript-cli-hallucinated-default-model", "result": "passed", "tests": 3}