python-aws-emulator-test-coverage-matrix

Answer

TS-GAP-023 closed via docs-only path. The audit established three facts that shape the fix: (1) comm -12 <(ls totalstack/services/) <(ls tests/aws/services/) yields exactly 4 same-name dirs, so only 4 of 69 emulated services have dedicated emulator tests; (2) all 69 providers wrap MotoFallbackDispatcher, so every service has a working emulator path via Moto — "untested" means "no dedicated emulator tests", not "broken"; (3) handler-depth greps (grep -c 'def handle' etc.) are meaningless because handlers are attached at runtime via dynamic setattr, so the directory intersection is the only reliable coverage signal.

Fix: a generated README matrix table (tested / Moto fallback), kept honest by a generator script so the docs cannot go stale. Two deliverables:

1. tools/generate_aws_coverage_matrix.py (the source of truth)

#!/usr/bin/env python3
"""TS-GAP-023: Generate the AWS service coverage matrix for the README.

Method (mirrors the closing audit):

  comm -12 <(ls totalstack/services/) <(ls tests/aws/services/)

A service under ``totalstack/services/`` is **Tested** iff a same-named
directory exists under ``tests/aws/services/``. Every other service relies on
the ``MotoFallbackDispatcher`` (all providers wrap it) and is listed as
**Moto fallback**. Handler-depth greps are meaningless (handlers are attached
at runtime via ``setattr``), so the directory intersection is the only signal.

Output is a Markdown table for the README. Do NOT edit the table by hand;
re-run this script after adding/removing services.
"""
import sys
from pathlib import Path

REPO_ROOT = Path(__file__).resolve().parent.parent
SERVICES_DIR = REPO_ROOT / "totalstack" / "services"
TESTS_DIR = REPO_ROOT / "tests" / "aws" / "services"

TESTED, MOTO_FALLBACK = "Tested", "Moto fallback"

def _dirs(root: Path) -> list[str]:
    """Same-named directories, sorted like ``ls``."""
    return [] if not root.is_dir() else sorted(p.name for p in root.iterdir() if p.is_dir())

def coverage() -> dict[str, str]:
    """Map each emulated service -> coverage status (comm -12 semantics)."""
    tested = set(_dirs(TESTS_DIR))
    return {svc: TESTED if svc in tested else MOTO_FALLBACK for svc in _dirs(SERVICES_DIR)}

def render(rows: dict[str, str]) -> str:
    lines = ["| AWS service | Coverage |", "|---|---|"]
    lines.extend(f"| {svc} | {status} |" for svc, status in rows.items())
    return "\n".join(lines)

def main() -> int:
    rows = coverage()
    if not rows:
        print("error: no service dirs under totalstack/services/", file=sys.stderr)
        return 1
    tested = sum(v == TESTED for v in rows.values())
    print(render(rows))
    print(f"<!-- {len(rows)} services: {tested} tested, {len(rows) - tested} Moto fallback -->")
    return 0

if __name__ == "__main__":
    sys.exit(main())

2. README section (embed generated output between markers)

```markdown