typescript-embedding-provider-fallback

Answer

The failure chain was three silent failure modes stacking on one dead code path. Full implementation: /tmp/embed-fix/embeddings.ts, verified by /tmp/embed-fix/embeddings.test.ts (18/18 passing, run with node --experimental-strip-types --test).

1. createAutoProviders() — return ALL healthy providers (was: first only)

A probe that passes is not proof an embed will succeed (LM Studio /v1/models 200s even when the embedding model is unloaded → /v1/embeddings 400s). Returning only the first healthy provider left nothing to fall back to.

export async function createAutoProviders(cfg: ProviderConfig): Promise<EmbedProvider[]> {
  const healthy: EmbedProvider[] = [];
  for (const Ctor of EMBEDDING_PROVIDERS) {
    const p = new Ctor(cfg);
    try {
      if (await p.healthy()) healthy.push(p);   // was: return [p];  (only first!)
    } catch { /* probe threw -> not healthy, keep probing */ }
  }
  return healthy;                                // ALL healthy -> recall can fall back
}

2. recall() — iterate with fallback on embed failure AND empty vector

[] is truthy in JS, so if (result.vector) accepted Ollama's 200-with-embedding: [] for a missing model as a valid embed — recall silently returned nothing.

export async function recall(query, providers, store, k = 5) {
  let lastErr: unknown = new Error('no embedding providers available');
  for (const p of providers) {                       // iterate, don't stop at first
    try {
      const r = await p.embed(query);
      if (r.isError) throw new EmbedError(`${p.name}: ${r.error ?? 'embed error'}`);
      if (r.vector.length === 0)                     // explicit empty check, NOT truthiness
        throw new EmbedError(`${p.name} returned an empty vector`);
      return await store.search(r.vector, k);
    } catch (e) { lastErr = e; }                     // fall through to next provider
  }
  throw lastErr;                                     // never return silently-empty results
}

3. wrapHandler() — flag error results with isError: true

Error-shaped results ({error: "model_not_loaded"}) previously passed through unflagged, indistinguishable from success.

export function wrapHandler(fn: Handler): Handler {
  return async () => {
    const result = await fn();
    if (result && typeof result === 'object' && 'error' in result && result.error) {
      (result as { isError?: boolean }).isError = true;   // was: returned as-is
    }
    return result;
  };
}

4. cosineSimilarity() — throw on zero-length instead of returning NaN

export function cosineSimilarity(a: number[], b: number[]): number {
  if (a.length === 0) throw new EmbedError('cosineSimilarity: zero-length vector a');
  if (b.length === 0) throw new EmbedError('cosineSimilarity: zero-length vector b');
  if (a.length !== b.length) throw new EmbedError('cosineSimilarity: dimension mismatch');
  let dot = 0, na = 0, nb = 0;
  for (let i = 0; i < a.length; i++) { dot += a[i] * b[i]; na += a[i] * a[i]; nb += b[i] * b[i]; }
  const mag = Math.sqrt(na) * Math.sqrt(nb);
  if (mag === 0) throw new EmbedError('cosineSimilarity: zero-magnitude vector');
  return dot / mag;
}

5. ROOT CAUSE (bonus): config default was lmstudio, auto path never ran

The schema/docs claimed auto fallback, but the hardcoded default 'lmstudio' forced explicit mode (single provider, no fallback) for every user who didn't set the field — fixes 1–4 never engaged. Default fixed to 'auto'; explicit modes intentionally stay single-provider.

export const configSchema = {
  provider: {
    type: 'string',
    enum: ['auto', 'lmstudio', 'ollama', 'openai'],
    default: 'auto',          // WAS: 'lmstudio' — the dead path
  },
} as const;

export function resolveProviders(cfg: ProviderConfig): ProviderMode[] {
  const mode: ProviderMode = cfg.provider ?? 'auto';
  if (mode === 'auto') return ['lmstudio', 'ollama'];  // expansion, fallback available
  return [mode];                                        // explicit = no fallback (by design)
}