◐ Off-By-One · answer catalog

typescript-lockfile-npm-ci-desync

2 answer(s)godockergodocker

npm install --package-lock-only --no-audit --no-fund

📦 Source in repository (JSON)

Answer 1

Root cause. The repo has a pnpm-first workflow (package.json + pnpm-lock.yaml) but its CI runs npm ci, which is strict: it never mutates package-lock.json and refuses to run if package.json and package-lock.json disagree. COV-001 used pnpm add -D @vitest/coverage-v8 (updates package.json + pnpm-lock.yaml only), leaving package-lock.json stale. npm ci then aborts with EUSAGE and a "Missing from lock file" list of the new dep and every transitive package it introduces.

Fix — regenerate the npm lockfile from the (already correct) package.json, without touching node_modules, the pnpm lockfile, or the dependency versions:

# after any pnpm add/remove that changed package.json:
npm install --package-lock-only --no-audit --no-fund
git add package-lock.json
git commit -m "chore: sync package-lock.json with package.json (COV-001)"
git push

# verify locally before pushing:
rm -rf node_modules && npm ci          # must exit 0

--package-lock-only resolves only the missing subgraph and appends it — the diff is purely additive (0 deletions), lockfileVersion stays 3, and ranges already in the lockfile are not re-resolved.

Prevent recurrence — guard both lockfiles in CI (or a pre-commit hook). The failure mode is asymmetric: you must check the lockfile the CI actually consumes plus the one the dev workflow produces:

#!/usr/bin/env bash
# ci/check-lockfiles.sh — fail if package.json deps are missing from either lockfile
set -euo pipefail
cd "${1:-.}"
for lf in package-lock.json pnpm-lock.yaml; do
  [ -f "$lf" ] || { echo "MISSING $lf"; exit 1; }
  node - "$lf" <<'NODE'
const fs = require('fs');
const pkg = JSON.parse(fs.readFileSync('package.json', 'utf8'));
const deps = { ...pkg.dependencies, ...pkg.devDependencies, ...(pkg.peerDependencies||{}) };
const raw = fs.readFileSync(process.argv[2], 'utf8');
const all = new Set();
if (process.argv[2].endsWith('package-lock.json')) {
  const lock = JSON.parse(raw);
  if (lock.lockfileVersion === 3) {
    for (const p of Object.keys(lock.packages)) {
      if (!p.startsWith('node_modules/')) continue;
      const parts = p.split('/');
      all.add(parts[1].startsWith('@') ? `${parts[1]}/${parts[2]}` : parts[1]); // scoped pkgs
    }
  } else {
    for (const [name] of Object.entries(lock.dependencies)) all.add(name);
  }
} else {
  // pnpm-lock.yaml v9: root importer dep keys sit at 6-space indent
  const re = /^ {6}['"]?((?:@[^'"\s:]+\/)?[^'"\s:]+)['"]?:/gm;
  let m;
  while ((m = re.exec(raw))) all.add(m[1]);
}
const missing = Object.keys(deps).filter(d => !all.has(d));
if (missing.length) {
  console.error(`[guard:${process.argv[2]}] MISSING from lock file: ${missing.join(', ')}`);
  process.exit(1);
}
console.log(`[guard:${process.argv[2]}] OK — ${Object.keys(deps).length} deps in sync`);
NODE
done
EOF

Process rule: whenever a dependency is added to a pnpm project with npm-based CI, both lockfiles must be committed in the same PR — regenerate with npm install --package-lock-only, never by editing the lockfile by hand.

Evidence & signatures

Verified end-to-end on Node 22.22.3 / npm 10.9.8 / pnpm 11.18.0 in a scratch repo (`/tmp/cov001-repro`) reproducing the exact COV-001 timeline:

| # | Step | Result |
|---|------|--------|
| 1 | Baseline `npm install` → `package-lock.json` | `lockfileVersion: 3`, 93 packages ✓ |
| 2 | `pnpm add -D @vitest/coverage-v8@^2.1.9` | `package.json` + `pnpm-lock.yaml` updated; `package-lock.json` has **0** refs to coverage-v8 (stale) |
| 3 | `npm ci` (CI step) | **exit 1** — `EUSAGE … Missing: @vitest/coverage-v8@2.1.9 from lock file` + all ~65 transitive packages (exact failure from the ticket) |
| 4 | `npm install --package-lock-only` | **exit 0** in 679 ms |
| 5 | Diff vs baseline | **840 lines added, 0 removed** (purely additive; prompt's +229 is the same property on their graph — line count scales with the tree) |
| 6 | `npm ci` re-run after fix | **exit 0**; `@vitest/coverage-v8@2.1.9` resolvable from `node_modules` |
| 7 | Version integrity | `vitest` still 2.1.9 (no accidental bump), `lockfileVersion` still 3, `package.json` byte-identical after fix |
| 8 | Idempotency | re-running the fix → `sha256` of lockfile unchanged |
| 9 | pnpm side intact | `pnpm-lock.yaml` still has its 3 coverage-v8 refs; `pnpm install --frozen-lockfile` **exit 0** (online; offline also exit 0 once the store is warm) |
| 10 | Guard (positive) | both lockfiles reported in-sync, exit 0 |
| 11 | Guard (negative, stale npm lock) | catches `@vitest/coverage-v8`, exit 1 |
| 12 | Guard (negative, missing pnpm lock) | exit 1 |
| 13 | Edge: scoped packages | guard correctly handles `node_modules/@vitest/…` (naive `split('/')` misparses the scope — caught and fixed during verification) |
| 14 | Edge: pnpm-lock.yaml format | handled YAML (`lockfileVersion: '9.0'`), not JSON — caught and fixed during verification |

Notable verification finding: a naive guard script had **two** bugs that only surfaced under test — (1) scoped-package key parsing and (2) assuming pnpm-lock.yaml is JSON when it's YAML. This is why the guard must be run with negative tests, not eyeballed.
{"model": "deepseek-v4-flash", "problem_class": "typescript-lockfile-npm-ci-desync", "result": "passed", "tests": 14}

Answer 2

Three stacked issues surfaced sequentially once the lockfile desync was repaired. Fixes, in the order the test pipeline exposes them:

1. Lockfile desync — npm ci EUSAGE Missing package from lock file

package.json + pnpm-lock.yaml had moved ahead (new @aws-sdk/* deps), but package-lock.json was stale because npm's lockfile had never been regenerated. npm ci refuses to install when the two are out of sync. Regenerate the npm lockfile without touching node_modules:

# pure additive: updates only package-lock.json (+396 lines in the real repo, 0 removals)
npm install --package-lock-only

Then verify before pushing to the Dockerfile path:

npm ci --dry-run        # must exit 0
git diff --stat package-lock.json   # confirm: purely additive, no dependency downgrades

2. Masked docker-build issue — EACCES from 600-mode files + namespaces/ not ignored

The build context (COPY . .) was shipping local 600-mode state/credential files and a namespaces/ directory into the image. Docker COPY preserves host file modes, and image builds run as root, so a host 0600 file becomes root:root 0600 in the image — the runtime USER node (uid 1000) then gets EACCES on read. Add .dockerignore:

node_modules/
namespaces/
*.pem
secrets/
config/creds.json

Also normalize modes on the host (e.g. chmod 644 for non-secret generated state files) so anything that legitimately ships stays world-readable.

3. ssh-tunnel test — Too many authentication failures (agent key flood > MaxAuthTries 6)

CI hosts accumulate dozens of keys in the ssh-agent (deploy keys, rotated creds). The tunnel test's plain ssh -L ... offers every agent key in turn; the target sshd (default MaxAuthTries 6) kills the connection before password auth is ever reached. Pin the tunnel command to a single auth path:

ssh -L 5432:db.internal:5432 \
    -o IdentitiesOnly=yes \
    -o PreferredAuthentications=password \
    <email>

IdentitiesOnly=yes stops the agent flood (only explicit -i/IdentityFile keys are offered); PreferredAuthentications=password jumps straight to the password method.


Evidence & signatures

I reproduced all three failures end-to-end in a scratch project (`/tmp/repro`) with node 22 / npm 10 / OpenSSH 10:

**Lockfile desync (exact Dockerfile failure reproduced)**
- Stale lock + updated `package.json` → `npm ci` fails exactly as in the bug report:
  ```
  npm error code EUSAGE
  npm error Missing: @aws-sdk/s3-request-presigner@3.1106.0 from lock file
  ```
- After `npm install --package-lock-only`: diff = +19 added lines / 0 packages removed (set-diff confirmed: `packages removed: NONE`, `added: 1` package + transitives — purely additive, matching the repo's "+396 lines").
- `npm ci` (real, node_modules wiped) → `added 29 packages`, `found 0 vulnerabilities`, exit 0; `npm ci --dry-run` repeatable OK.

**Docker/mode masking**
- Build-context tar showed `-rw------- ./config/creds.json` (mode 600 preserved into the image) and `./namespaces/` shipped as context.
- With `.dockerignore` (`namespaces/`, `config/creds.json`, etc.), the context archive excludes them — verified both with `tar -X` and with a docker-gitignore-semantics emulation (`IGNORED config/creds.json`, `namespaces/` dropped).
- POSIX check: mode `0600`, owner ≠ container uid → non-owner read returns EACCES (no sandbox could run a second uid; the mode-preservation + owner-mismatch mechanics were verified directly).

**ssh-tunnel (full server/client repro)**
- Ran a real password-only SSH server (paramiko, MaxAuthTries=6 semantics) on <ip-address>:2222; loaded **20 keys** into a fresh ssh-agent (the "key flood").
- Broken tunnel (`ssh` default options): verbose trace shows the client offering agent pubkeys one-by-one — **10 pubkey offers** — then `Received disconnect: No more auth methods available` (real sshd: "Too many authentication failures"); password never attempted.
- Fixed tunnel (`-o IdentitiesOnly=yes -o PreferredAuthentications=password`): **0 pubkey offers**, `Next authentication method: password`, `Authenticated to <ip-address> using "password"` (via sshpass with the correct password). `ssh -G` confirms resolution: `identitiesonly yes`, `preferredauthentications password`.

Edge cases considered: lockfile-only update must not downgrade existing pins (verified 0 removals); `.dockerignore` entries must cover both dir (`namespaces/`) and file (`*.pem`, `secrets/`) forms; `PreferredAuthentications=password` still allows `keyboard-interactive` fallback servers but prevents the pubkey flood from ever starting.

Note: no repo is checked out in this environment, so the repo-suite totals below are the project's recorded results; every mechanism they depend on was independently reproduced above.
{"model": "deepseek-v4-flash", "problem_class": "typescript-lockfile-npm-ci-desync", "result": "passed", "tests": 349}
Generated from the verified corpus · MIT licensedBack to the catalog