typescript-http-pagination-limit-validation

Answer

Root-cause chain (why it was a 108MB DoS): 1. parseInt("-1") → -1, not NaN — the route never rejected it. 2. Route fetched limit + 1 (off-by-one probe for hasMore) → -1 + 1 = 0. 3. queries.ts guarded the LIMIT clause with a truthy check (if (filters.limit)) — 0 is falsy → LIMIT clause dropped entirely → SELECT * FROM memories → full-table dump. 4. total was computed from the fetched rows (Math.min(pageSize, rows.length)) → capped at page size, wrong. 5. hasMore = rows.length > limit with limit = 0 → any non-empty result claimed hasMore: true.

Fix — 1. Strict parseLimit() (route layer):

// validation.ts
export const VALIDATION_ERROR = 'VALIDATION_ERROR';
export const MAX_LIMIT = 1000;
export const DEFAULT_LIMIT = 50;

export function parseLimit(raw: unknown): number {
  if (raw === undefined || raw === null) return DEFAULT_LIMIT;
  const n = typeof raw === 'number' ? raw : Number(raw);
  // rejects NaN, "-1", "1.5", "", non-numeric strings, and negatives
  if (typeof raw === 'string' && raw.trim() === '') {
    throw new ApiError(400, VALIDATION_ERROR, 'limit must be an integer');
  }
  if (!Number.isInteger(n) || n < 0) {
    throw new ApiError(400, VALIDATION_ERROR,
      'limit must be a non-negative integer');
  }
  return Math.min(n, MAX_LIMIT); // cap 1000; 0 is a valid empty page
}

Route wiring — parseInt(-1) is gone; the raw query value goes through validation before the +1 probe:

app.get('/api/memories', async (req, res) => {
  const limit = parseLimit(req.query.limit);   // throws 400 VALIDATION_ERROR
  const page  = await recallTool({ ...filters(req), limit });
  res.json(page);
});

Fix — 2. queries.ts: emit LIMIT 0 for falsy zero (!== undefined, not truthy):

export function buildMemoryQuery(filters: MemoryFilters) {
  const sql = ['SELECT * FROM memories'];
  const params: unknown[] = [];
  if (filters.ownerId !== undefined) { sql.push('WHERE owner_id = ?'); params.push(filters.ownerId); }
  if (filters.limit !== undefined) {          // was: if (filters.limit) — 0 fell through
    sql.push('LIMIT ?');
    params.push(filters.limit);               // 0 → LIMIT 0 → 0 rows scanned
  }
  return { sql: sql.join(' '), params };
}

Fix — 3. recallTool: true COUNT(*) total + correct hasMore:

export async function recallTool(filters: MemoryFilters) {
  const limit = filters.limit ?? DEFAULT_LIMIT;
  const probe  = limit > 0 ? limit + 1 : 0;      // probe only when a next page can exist
  const rows   = await queryMemories({ ...filters, limit: probe });
  const total  = await countMemories(filters);   // true COUNT(*) — never capped by page size

  return {
    memories: limit > 0 ? rows.slice(0, limit) : [],
    total,
    hasMore:  limit > 0 && rows.length > limit,  // limit=0 → always false
  };
}

export async function countMemories(filters: MemoryFilters): Promise<number> {
  const { sql, params } = buildCountQuery(filters); // SELECT COUNT(*) FROM memories <where>
  const row = await db.get(sql, params);
  return Number(row?.count ?? 0);
}

Behavior matrix after fix: