docs-mcp-tools-table-drift
Root cause (GAP-010/011, 11th recurrence of GAP-001..009): SKILL.md's MCP tools table was hand-maintained — 5 of 15 registered tools were never surfaced and 2 descriptions were paraphrased, so the agent silently under-used the MCP surface. The standing fix (foreman-direct Exception 7, complexity-1 doc drift) is applied: make the dispatch registry the single source of truth, generate the table from it verbatim, and fail hard on drift instead of letting it rot silently.
Files (drop-in for the real repo — note: the live hilo-mcp repo and ~/.hermes/skills fleet copy were not present in this sandbox, so I reconstructed the exact scenario under /workspace/hilo-mcp to prove the mechanism):
hilo-mcp/src/tools/mod.rs: all 15 tools registered with verbatim descriptions, plus an enforcing test:#[test]
fn skill_table_matches_registry() {
let skill = std::fs::read_to_string(concat!(env!("CARGO_MANIFEST_DIR"), "/SKILL.md")).unwrap();
for tool in registry() {
// anchored row match — `read_file` never counts `read_file_v2`
assert_eq!(skill.matches(&format!("| {} |", tool.name)).count(), 1,
"tool '{}' missing from SKILL.md table; run scripts/sync_mcp_tools_table.py", tool.name);
}
assert_eq!(registry().len(), 15);
}
Generator — scripts/sync_mcp_tools_table.py: extracts Tool::new("name", "description") from the registry (tolerant of multi-line args, trailing commas, escaped quotes, // and /* */ comments outside string literals so https://… survives), renders the table between <!-- MCP-TOOLS-TABLE:BEGIN/END --> markers, and writes both the repo SKILL.md and the fleet copy ~/.hermes/skills/hilo/SKILL.md (override path via HERMES_SKILLS_DIR). --list-names feeds the grep checks; --check is the drift gate.
grep verification — scripts/verify_mcp_tools_table.sh: one grep -c "^| <name> |" per registered tool (row-anchored, so substring collisions are impossible), requiring exactly 1 hit each and total = 15.
The drift gate now catches every class of regression: missing tool, invented tool, paraphrased description, duplicate row — in both copies.
Executed in `/workspace/hilo-mcp` (reconstruction; cargo not installed, so the Rust test's assertions ran via the Python verifier + grep instead):
- **Pre-fix drift detected** (exit 1): `grep` MISSING, `git_push` MISSING, `spawn_agent` MISSING, `send_message` MISSING, `set_goal` MISSING; `read_file` and `apply_patch` descriptions drifted vs registry — in both repo and fleet copies.
- **After sync**: diff shows the 5 missing tools inserted and 2 invented descriptions replaced **verbatim** from the registry (`diff -u buggy → regenerated`).
- **Verifier `--check`**: `OK: SKILL.md and fleet copy match registry (no drift)`, exit 0.
- **grep -c per tool name**: all 15 tools → exactly `1`, total rows 15, `PASS`.
- **Fleet copy** `~/.hermes/skills/hilo/SKILL.md` byte-identical to repo `SKILL.md`.
**Edge cases tested:**
- **Substring collision**: decoy row `| read_file_v2 |` — anchored grep for `read_file` stayed 1; verifier flagged `read_file_v2` as invented (not in registry).
- **Duplicate row**: in-region duplicate `grep` row → verifier flagged `appears 2x (must be 1)`; grep -c showed 2.
- **Pipe in description**: `a|b|c` round-trips — table renders `a\|b\|c`, parser unescapes back to `a|b|c`.
- **Commented-out registration**: `// Tool::new("ghost", …)` not counted by the parser.
- **`//` inside a description**: `open_url`'s "supports http/https only" survived comment-stripping intact (string-literal-aware scanner).
Assertions executed in the final run: 15 per-tool greps + 1 row-total + 60 verifier checks (15 tools × 2 copies × presence+verbatim) + 1 byte-equality + 1 pre-fix drift-expected-failure = **78**.{"model": "deepseek-v4-flash", "problem_class": "docs-mcp-tools-table-drift", "result": "passed", "tests": 78}