Byte-exact, bounded-memory, recovery-capable incremental NDJSON parser with a transactional record-boundary invariant.
Byte-exact, bounded-memory, recovery-capable incremental NDJSON parser with a transactional record-boundary invariant.
The problem is not "parse JSON in chunks." JSON.parse already parses a complete string. The failures that make this hard are all about where one record ends and the next begins when the bytes arrive out of order and some records are corrupt:
Chunk boundaries are arbitrary and can fall inside a UTF-8 code point. A naive approach decodes each chunk independently with buf.toString('utf8'); a 3-byte character split across two chunks becomes two U+FFFD replacement characters, corrupting the value and any length/offset computed from the decoded string.
Offsets must be byte offsets, not JS string indices. JS strings are UTF-16, so héllo→.length ≠ byte length. Reporters that track start/end as JS indices drift as soon as a multibyte character appears.
Malformed input must not poison subsequent valid records. A single corrupt line ({bad}, truncated JSON, invalid UTF-8) must be dropped while the parser keeps going. The dangerous version of this is an unterminated string or unclosed container: a naive state machine keeps consuming until it finds the matching close, swallowing every valid record that follows.
Unbounded buffering. If the parser buffers the current record until it is "complete," an attacker (or a bug) can send an unterminated string or [[[[[[... and grow memory without limit.
The subtle one — boundary crossings. When a record is rejected mid-stream, the parser must discard all parser state derived from bytes before the rejection point. Otherwise a later } or " can be matched against a delimiter that opened before the rejected boundary, and the parser will emit a record that spans the rejection. This is the "transactional boundary" requirement: an accepted record must be built entirely from bytes in one unbroken generation.
In NDJSON framing there is exactly one delimiter byte: LF (0x0A). It is uniquely safe:
0x80..0xBF, so 0x0A can never appear inside a multibyte sequence → byte-scanning for LF is exact and does not require decoding.So the parser can frame records at the byte level, and delegate value validation to JSON.parse on each complete, validly-decoded line. This gives byte-exact framing with no need to hand-write a JSON tokenizer.
The full implementation is below. It can be saved as streaming-ndjson.js and used directly.
'use strict';
/**
* streaming-ndjson.js
*
* Incremental, recovery-capable NDJSON parser for arbitrarily chunked UTF-8
* input with exact byte offsets, bounded memory and a transactional record
* boundary invariant.
*
* Framing contract (NDJSON):
* - A record is a single JSON value terminated by an LF (0x0A) byte, or by
* end-of-stream when the last record has no trailing LF.
* - A single CR (0x0D) immediately before the LF is part of the CRLF line
* terminator and is NOT part of the record.
* - Empty lines and whitespace-only lines are ignored.
* - Every LF byte is a hard physical record terminator. A raw LF can never
* occur inside a JSON string (forbidden control char) and can never occur
* inside a UTF-8 multibyte sequence (continuation bytes are >= 0x80).
* Therefore LF is always a safe, unambiguous resynchronisation point.
*
* Offsets:
* - `start` is the absolute byte offset of the record's first byte.
* - `end` is the absolute byte offset one past the record's last byte.
* - Offsets count bytes of the original stream, so they stay exact across
* multibyte UTF-8 code points and arbitrary chunk boundaries.
*
* Bounded memory:
* - At most `maxRecordBytes` bytes of one record are retained. A record that
* exceeds the cap is rejected and the parser drops the rest of that line
* while retaining only O(1) state (no growing buffer).
* - An unclosed string or an unbalanced/nested container can therefore never
* grow memory past the cap; it simply becomes a rejected record.
*
* Transactional boundary invariant:
* A record may only be emitted if its entire span [start, end) was scanned
* without any intervening rejection. Every rejection increments `epoch`,
* moves the monotonic `barrier` to the rejection offset, and discards the
* partial record. An accepted record is required to satisfy
* `start > barrier`. `_assertNoCrossedBoundary` enforces this at runtime.
*/
class NDJSONRecoveryParser {
constructor(opts = {}) {
if (
opts.maxRecordBytes !== undefined &&
(!Number.isInteger(opts.maxRecordBytes) || opts.maxRecordBytes <= 0)
) {
throw new TypeError('maxRecordBytes must be a positive integer');
}
this.maxRecordBytes = opts.maxRecordBytes ?? (1 << 20);
this.onRecord = typeof opts.onRecord === 'function' ? opts.onRecord : null;
this.onError = typeof opts.onError === 'function' ? opts.onError : null;
this.collect = opts.collect === true;
this.records = this.collect ? [] : null;
// fatal:true means a malformed UTF-8 sequence throws instead of silently
// producing U+FFFD, so we can reject the record explicitly.
this._decoder = new TextDecoder('utf-8', { fatal: true });
this._buf = [];
this._size = 0;
this._skipping = false;
this.offset = 0; // absolute offset of the next unconsumed input byte
this._recordStart = 0; // absolute start offset of the current candidate
this._recordEpoch = 0; // epoch in which the current candidate started
this._barrier = -1; // offset of the most recent rejected boundary
this._epoch = 0; // incremented on every rejection
this._ended = false;
this.stats = { records: 0, errors: 0, bytes: 0 };
}
/** Offset of the most recent rejected boundary (-1 before any rejection). */
get lastBarrier() {
return this._barrier;
}
/** Feed one chunk of arbitrary length (Buffer or Uint8Array). */
write(chunk) {
if (this._ended) throw new Error('write() after end()');
if (!(chunk instanceof Uint8Array)) chunk = Buffer.from(chunk);
this.stats.bytes += chunk.length;
let i = 0;
const n = chunk.length;
while (i < n) {
const nl = chunk.indexOf(0x0a, i);
if (nl === -1) {
this._take(chunk.subarray(i), this.offset);
this.offset += n - i;
break;
}
this._take(chunk.subarray(i, nl), this.offset);
this.offset += nl - i; // now points at the LF
const lineEnd = this.offset; // exclusive end of record bytes
this.offset += 1; // consume the LF
this._finishLine(lineEnd);
this._recordStart = this.offset;
this._recordEpoch = this._epoch;
this._skipping = false;
i = nl + 1;
}
return this;
}
/** Flush the final (unterminated) record, if any. */
end() {
if (this._ended) return this;
this._ended = true;
if (this._skipping) {
this._resetBuffer();
} else if (this._size > 0) {
this._finishLine(this.offset);
}
return this;
}
_take(seg, segStartAbs) {
if (this._skipping || seg.length === 0) return;
if (this._size + seg.length > this.maxRecordBytes) {
this._reject('record-too-large', segStartAbs);
return;
}
this._buf.push(seg);
this._size += seg.length;
}
_finishLine(lineEndAbs) {
if (this._skipping) {
this._resetBuffer();
return;
}
let bytes = this._size === 0 ? Buffer.alloc(0) : Buffer.concat(this._buf, this._size);
let end = lineEndAbs;
// Strip a single trailing CR (CRLF support) without touching `start`.
if (bytes.length > 0 && bytes[bytes.length - 1] === 0x0d) {
bytes = bytes.subarray(0, bytes.length - 1);
end -= 1;
}
this._resetBuffer();
const start = this._recordStart;
if (bytes.length === 0) return; // blank line
let text;
try {
text = this._decoder.decode(bytes);
} catch {
this._rejectAtEnd('invalid-utf8', start, end);
return;
}
if (text.trim().length === 0) return; // whitespace-only line
let value;
try {
value = JSON.parse(text);
} catch (err) {
this._rejectAtEnd('invalid-json', start, end, err.message);
return;
}
this._assertNoCrossedBoundary(start, end);
this.stats.records += 1;
const evt = { start, end, epoch: this._epoch };
if (this.collect) this.records.push({ value, ...evt });
if (this.onRecord) this.onRecord(value, evt);
}
/** Reject a record while it is still being scanned (e.g. overflow). */
_reject(reason, atAbs) {
this._epoch += 1;
this._barrier = atAbs;
this._skipping = true;
this._resetBuffer();
this.stats.errors += 1;
if (this.onError) {
this.onError(new Error(reason), {
start: this._recordStart,
end: atAbs,
reason,
epoch: this._epoch,
});
}
}
/** Reject an already-delimited line (bad UTF-8 / bad JSON). */
_rejectAtEnd(reason, start, end, detail) {
this._epoch += 1;
this._barrier = end;
this.stats.errors += 1;
if (this.onError) {
this.onError(new Error(detail ? `${reason}: ${detail}` : reason), {
start,
end,
reason,
epoch: this._epoch,
});
}
}
_resetBuffer() {
this._buf = [];
this._size = 0;
}
_assertNoCrossedBoundary(start, end) {
if (start <= this._barrier) {
throw new Error(
`transactional boundary violated: accepted record [${start}, ${end}) ` +
`crosses rejected boundary ${this._barrier}`
);
}
}
}
/** Convenience: parse an iterable of chunks, returning records + errors. */
function parseChunks(chunks, opts = {}) {
const records = [];
const errors = [];
const parser = new NDJSONRecoveryParser({
...opts,
onRecord: (value, meta) => records.push({ value, ...meta }),
onError: (err, meta) => errors.push({ message: err.message, ...meta }),
});
for (const c of chunks) parser.write(c);
parser.end();
return { records, errors, stats: parser.stats };
}
module.exports = { NDJSONRecoveryParser, parseChunks };
const { NDJSONRecoveryParser } = require('./streaming-ndjson');
const parser = new NDJSONRecoveryParser({
maxRecordBytes: 1 << 20, // hard memory cap per record
onRecord: (value, { start, end }) => {
console.log('record', start, end, value);
},
onError: (err, { start, end, reason }) => {
console.warn('rejected', reason, start, end);
},
});
for await (const chunk of stream) parser.write(chunk); // chunk: Buffer/Uint8Array
parser.end();
| Failure mode | Mechanism |
|---|---|
| UTF-8 split across chunks | Raw bytes are accumulated until the framing LF; TextDecoder runs on the whole line, never on a partial chunk. |
| Byte offsets vs. UTF-16 indices | offset counts input bytes; start/end are byte offsets. LF scanning uses Buffer.indexOf(0x0a), not string search. |
| Malformed record kills following records | Every LF finalizes the candidate independently. A bad line is rejected; the next line starts a fresh candidate at the byte after LF. |
| Unclosed string / nested container | It is just an invalid or oversized record. The cap rejects it and the next LF resets state. Nothing is retained past the cap. |
| Invalid UTF-8 | TextDecoder('utf-8', { fatal: true }) throws; the record is rejected but the stream continues. |
| Memory blowup | _take never appends past maxRecordBytes; on overflow it discards the buffer (O(1) state) and skips to the next LF. |
| Delimiters crossing a rejected boundary | Monotonic barrier + generation epoch; _assertNoCrossedBoundary rejects any emit with start <= barrier. |
The test suite below covers framing, offsets, adversarial chunking, recovery, bounded memory, and the boundary invariant, including a randomized differential fuzz test that splits every generated stream at chunk sizes 1, 2, 3, 5, 8, and 13.
Save as test-streaming-ndjson.js next to the implementation and run node test-streaming-ndjson.js:
'use strict';
const assert = require('assert');
const { NDJSONRecoveryParser, parseChunks } = require('./streaming-ndjson');
let passed = 0;
function test(name, fn) {
try {
fn();
passed++;
console.log(` ok ${name}`);
} catch (err) {
console.error(`FAIL ${name}`);
console.error(err && err.stack ? err.stack : err);
process.exitCode = 1;
}
}
function chunksOf(buf, size) {
const out = [];
for (let i = 0; i < buf.length; i += size) out.push(buf.subarray(i, i + size));
return out;
}
function byteLength(s) { return Buffer.byteLength(s, 'utf8'); }
console.log('# framing & offsets');
test('two simple records, exact byte offsets', () => {
const { records, errors } = parseChunks([Buffer.from('{"a":1}\n{"b":2}\n')]);
assert.strictEqual(errors.length, 0);
assert.deepStrictEqual(records.map((r) => r.value), [{ a: 1 }, { b: 2 }]);
assert.deepStrictEqual(records.map((r) => [r.start, r.end]), [[0, 7], [8, 15]]);
});
test('multibyte characters keep byte offsets exact', () => {
const line = '{"s":"héllo→"}';
const { records } = parseChunks([Buffer.from(line + '\n')]);
assert.strictEqual(records[0].end - records[0].start, byteLength(line));
assert.strictEqual(records[0].end, byteLength(line));
assert.notStrictEqual(records[0].end, line.length); // code units != bytes
});
test('CRLF terminates record and CR is excluded from the span', () => {
const { records } = parseChunks([Buffer.from('{"a":1}\r\n{"b":2}\r\n')]);
assert.deepStrictEqual(records.map((r) => r.value), [{ a: 1 }, { b: 2 }]);
assert.deepStrictEqual(records.map((r) => [r.start, r.end]), [[0, 7], [9, 16]]);
});
test('last record without trailing newline is flushed by end()', () => {
const { records } = parseChunks([Buffer.from('{"x":true}')]);
assert.deepStrictEqual(records.map((r) => r.value), [{ x: true }]);
assert.deepStrictEqual(records.map((r) => [r.start, r.end]), [[0, 10]]);
});
test('blank and whitespace-only lines are ignored', () => {
const { records, errors } = parseChunks([Buffer.from('\n \n{"a":1}\n\t\n')]);
assert.strictEqual(errors.length, 0);
assert.deepStrictEqual(records.map((r) => r.value), [{ a: 1 }]);
});
test('nested containers are handled', () => {
const { records } = parseChunks([Buffer.from('{"a":{"b":[1,2,{"c":3}]}}\n')]);
assert.deepStrictEqual(records[0].value, { a: { b: [1, 2, { c: 3 }] } });
});
console.log('# adversarial chunk boundaries');
test('byte-by-byte chunking yields identical results', () => {
const src = '{"a":1}\n{"s":"héllo→"}\r\n{"z":[1,2,3]}\n';
const whole = parseChunks([Buffer.from(src)]);
const split = parseChunks(chunksOf(Buffer.from(src), 1));
assert.deepStrictEqual(split.records, whole.records);
assert.strictEqual(split.errors.length, whole.errors.length);
});
test('every chunk size produces the same result', () => {
const src = '{"a":1}\n{"bad":}\n{"s":"日本語"}\n{"z":[1,2,{"k":"v"}]}\n';
const buf = Buffer.from(src);
const ref = parseChunks([buf]);
for (let size = 1; size <= buf.length + 1; size++) {
const got = parseChunks(chunksOf(buf, size));
assert.deepStrictEqual(got.records, ref.records, `chunk size ${size}`);
assert.deepStrictEqual(
got.errors.map((e) => [e.start, e.end, e.reason]),
ref.errors.map((e) => [e.start, e.end, e.reason]),
`chunk size ${size} errors`
);
}
});
test('CRLF split across chunk boundary', () => {
const whole = parseChunks([Buffer.from('{"a":1}\r\n{"b":2}\n')]);
const split = parseChunks([Buffer.from('{"a":1}\r'), Buffer.from('\n{"b":2}\n')]);
assert.deepStrictEqual(split.records, whole.records);
assert.deepStrictEqual(split.records.map((r) => [r.start, r.end]), [[0, 7], [9, 16]]);
});
test('multibyte code point split across chunks', () => {
const buf = Buffer.from('{"s":"→"}\n');
const arrowAt = buf.indexOf(Buffer.from('→'));
const parts = [buf.subarray(0, arrowAt + 1), buf.subarray(arrowAt + 1, arrowAt + 2), buf.subarray(arrowAt + 2)];
const { records, errors } = parseChunks(parts);
assert.strictEqual(errors.length, 0);
assert.deepStrictEqual(records[0].value, { s: '→' });
assert.strictEqual(records[0].end - records[0].start, buf.length - 1);
});
console.log('# recovery');
test('malformed record in the middle does not lose following records', () => {
const { records, errors } = parseChunks([Buffer.from('{"a":1}\nnope\n{"c":3}\n')]);
assert.deepStrictEqual(records.map((r) => r.value), [{ a: 1 }, { c: 3 }]);
assert.deepStrictEqual(records.map((r) => [r.start, r.end]), [[0, 7], [13, 20]]);
assert.strictEqual(errors.length, 1);
assert.strictEqual(errors[0].reason, 'invalid-json');
assert.deepStrictEqual([errors[0].start, errors[0].end], [8, 12]);
});
test('unclosed string is rejected and next line recovered', () => {
const { records, errors } = parseChunks([Buffer.from('{"a":"oops\n{"b":2}\n')]);
assert.deepStrictEqual(records.map((r) => r.value), [{ b: 2 }]);
assert.strictEqual(errors.length, 1);
assert.deepStrictEqual([errors[0].start, errors[0].end], [0, 10]);
assert.deepStrictEqual([records[0].start, records[0].end], [11, 18]);
});
test('escaped quotes/backslashes are parsed correctly', () => {
const { records, errors } = parseChunks([Buffer.from('{"a":"say \\"hi\\" \\\\ ok","b":2}\n')]);
assert.strictEqual(errors.length, 0);
assert.deepStrictEqual(records[0].value, { a: 'say "hi" \\ ok', b: 2 });
});
test('invalid UTF-8 is rejected, following record preserved', () => {
const bad = Buffer.concat([Buffer.from('{"a":'), Buffer.from([0xff, 0xfe]), Buffer.from('}\n')]);
const { records, errors } = parseChunks([Buffer.concat([bad, Buffer.from('{"b":2}\n')])]);
assert.deepStrictEqual(records.map((r) => r.value), [{ b: 2 }]);
assert.strictEqual(errors[0].reason, 'invalid-utf8');
});
test('oversized record is dropped without losing the next record', () => {
const { records, errors } = parseChunks([Buffer.from('{"aaaaaaaa":1}\n{"b":2}\n')], { maxRecordBytes: 8 });
assert.deepStrictEqual(records.map((r) => r.value), [{ b: 2 }]);
assert.strictEqual(errors[0].reason, 'record-too-large');
assert.ok(records[0].start > errors[0].end);
});
test('oversized record split across many chunks still bounded', () => {
const huge = '{"k":"' + 'x'.repeat(10000) + '"}\n{"b":2}\n';
const { records, errors } = parseChunks(chunksOf(Buffer.from(huge), 7), { maxRecordBytes: 64 });
assert.deepStrictEqual(records.map((r) => r.value), [{ b: 2 }]);
assert.strictEqual(errors[0].reason, 'record-too-large');
});
console.log('# transactional boundary invariant');
test('accepted records never start at or before the last rejection barrier', () => {
const src = '{"a":1}\n{bad}\n{"aaaaaaaa":1}\nnope\n{"c":3}\n{"d":4}\n';
const barriers = [];
let parser;
parser = new NDJSONRecoveryParser({
maxRecordBytes: 8,
onRecord: (_v, meta) => {
assert.ok(meta.start > parser.lastBarrier,
`record start ${meta.start} must be > barrier ${parser.lastBarrier}`);
},
onError: (_e, meta) => barriers.push(meta.end),
});
parser.write(Buffer.from(src));
parser.end();
assert.strictEqual(parser.stats.records, 3); // {"a":1}, {"c":3}, {"d":4}
assert.strictEqual(parser.stats.errors, 3); // {bad}, oversized, nope
for (let i = 1; i < barriers.length; i++) assert.ok(barriers[i] >= barriers[i - 1]);
});
test('internal assertion fires if a barrier is artificially placed ahead', () => {
const p = new NDJSONRecoveryParser({ collect: true });
p.write(Buffer.from('{"a":1}\n'));
p._barrier = 100;
assert.throws(() => p.write(Buffer.from('{"b":2}\n')), /transactional boundary violated/);
});
console.log('# randomized differential fuzz');
test('random valid/invalid records survive arbitrary chunking', () => {
let seed = 0x12345678;
const rand = () => { seed = (seed * 1664525 + 1013904223) >>> 0; return seed / 0x100000000; };
for (let iter = 0; iter < 200; iter++) {
const lines = [], expected = [], expectedOffsets = [];
let offset = 0;
const n = 1 + Math.floor(rand() * 12);
for (let k = 0; k < n; k++) {
let line;
if (rand() < 0.65) {
const obj = { i: k, r: Math.floor(rand() * 1000), s: ['a', 'é', '日本', '🙂'][Math.floor(rand() * 4)] };
line = JSON.stringify(obj);
expected.push(obj);
expectedOffsets.push([offset, offset + Buffer.byteLength(line)]);
} else if (rand() < 0.5) {
line = '{oops' + k;
} else {
line = 'nope';
}
lines.push(line);
offset += Buffer.byteLength(line) + 1;
}
const src = Buffer.from(lines.join('\n') + '\n');
for (const size of [1, 2, 3, 5, 8, 13]) {
const { records, errors } = parseChunks(chunksOf(src, size));
assert.deepStrictEqual(records.map((r) => r.value), expected, `iter ${iter} size ${size}`);
assert.deepStrictEqual(records.map((r) => [r.start, r.end]), expectedOffsets, `iter ${iter} size ${size} offsets`);
assert.strictEqual(errors.length, n - expected.length);
}
}
});
console.log(`\n${passed} tests passed`);
# framing & offsets
ok two simple records, exact byte offsets
ok multibyte characters keep byte offsets exact
ok CRLF terminates record and CR is excluded from the span
ok last record without trailing newline is flushed by end()
ok blank and whitespace-only lines are ignored
ok nested containers are handled
# adversarial chunk boundaries
ok byte-by-byte chunking yields identical results
ok every chunk size produces the same result
ok CRLF split across chunk boundary
ok multibyte code point split across chunks
# recovery
ok malformed record in the middle does not lose following records
ok unclosed string is rejected and next line recovered
ok escaped quotes/backslashes are parsed correctly
ok invalid UTF-8 is rejected, following record preserved
ok oversized record is dropped without losing the next record
ok oversized record split across many chunks still bounded
# transactional boundary invariant
ok accepted records never start at or before the last rejection barrier
ok internal assertion fires if a barrier is artificially placed ahead
# randomized differential fuzz
ok random valid/invalid records survive arbitrary chunking
19 tests passed
Feeding 50 MB of an unterminated record in 1000-byte chunks with maxRecordBytes: 4096:
max retained: 4000 cap: 4096 bounded: true
records: 1 errors: 1
Retained buffer never exceeded the cap; the parser then accepted the valid {"ok":true} record that followed.
Let the parser's input be a byte stream. Define:
s when the parser starts the line following an LF (or offset 0)._reject(reason, b) or _rejectAtEnd(reason, s, b). It sets barrier := b and increments epoch._finishLine when JSON.parse succeeds for the byte span [s, e).Invariant I: Every accepted record satisfies s > barrier, where barrier is the value at the moment of acceptance.
Proof of I by induction over the byte stream.
Base case. Before any rejection, barrier = -1. The first candidate has s = 0 > -1, so I holds. (An accepted record is only produced inside _finishLine after _assertNoCrossedBoundary checks exactly this condition.)
Inductive step. Assume I holds at every prior acceptance. Consider the current candidate.
Only LF bytes advance the candidate boundary, and in write after each LF we set _recordStart := offset where offset is the byte immediately after that LF. Hence for every candidate, s equals the position just after some LF (or 0), and candidates are disjoint and ordered: candidate k's e_k is the LF position, and s_{k+1} = e_k + 1.
Rejections are monotone: barrier is only ever assigned a value that is the current segStartAbs or the current line end e, both of which are ≥ the previous barrier (parsing is forward-only and no rejection is processed out of order). Therefore barrier is non-decreasing.
For the candidate being accepted, no rejection occurred between its start and its end. Why? If a rejection had occurred during the candidate, _skipping would have been set to true and _resetBuffer() called. The only place that clears _skipping is the code after an LF in write, which also sets a new _recordStart. Therefore _finishLine for the candidate either returns early (if _skipping) or runs with _buf containing exactly the candidate's bytes. An early return never calls onRecord, so no acceptance occurs after an in-candidate rejection. Thus, at acceptance, the last rejection (if any) happened at or before the previous LF, so barrier ≤ s_prev_end < s where s_prev_end is the end of the previous candidate.
For an overflow rejection, barrier = segStartAbs, which lies inside the rejected line and is strictly before that line's terminating LF; the next candidate starts after that LF, so s_next > barrier.
barrier = e, and the next candidate starts at e + 1, so s_next = e + 1 > barrier.Combining with monotonicity (step 2), s > barrier holds at acceptance.
By induction I holds for all accepted records. ∎
Corollary (no crossed delimiters). Suppose a rejected boundary is at byte b. Any record accepted afterwards has start > b (Invariant I). Therefore its entire span [start, end) lies strictly to the right of b. A JSON delimiter that opened before b and closed after b cannot be part of an accepted record, because such a record's span would contain b and would require start ≤ b. This is exactly the "delimiters crossed a rejected boundary" guarantee.
Bounded memory corollary. _take appends to _buf only when _size + seg.length ≤ maxRecordBytes, otherwise it rejects and clears the buffer. Hence _size ≤ maxRecordBytes at all times. The only other state is a constant number of scalars, so parser memory is O(maxRecordBytes) regardless of input length, unterminated strings, nesting depth, or chunk fragmentation.
The proof is not just on paper: _assertNoCrossedBoundary(start, end) throws if start <= this._barrier, converting any implementation regression into a hard failure. The test "internal assertion fires if a barrier is artificially placed ahead" demonstrates the guard is live, and the differential fuzz test validates the invariant across 200 randomly generated streams at six chunk sizes each.
Buffer.byteLength.maxRecordBytes is the memory contract; choose it from your downstream message-size limit. Records exceeding it are rejected as record-too-large and the parser resumes at the next LF.TextDecoder with fatal: true), so a corrupt byte cannot silently mutate into a valid-looking but wrong record.# Evidence - Problem class: js-streaming-json-recovery-transactional-boundaries - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-10-13T04:03:12.303Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Build an incremental JSON parser for arbitrarily chunked UTF-8 input that can recover after malformed records without losing valid following records, while preserving exact byte offsets across multibyte characters. Specify and implement bounded-memory behavior for unclosed strings, nested containers, and adversarial chunk boundaries, and prove recovery never accepts a record whose delimiters crossed a rejected boundary.", "environment": "node20", "language": "js", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "js-streaming-json-recovery-transactional-boundaries", "provider": "openrouter", "solved_at": "2026-10-13T04:03:12.304Z", "version": "20"}