bfj
bfj, short for Big-Friendly JSON, provides asynchronous Node.js tools for parsing and serializing large JSON data without monopolizing the event loop. It can read or write whole files, parse streams, select matching values, walk tokens, serialize to a stream, and handle NDJSON. It yields after a configurable number of items and uses bounded serialization buffers. That improves responsiveness and memory behavior, but the README explicitly says it is not designed for speed.
Use bfj when event-loop fairness and incremental Node stream processing matter more than finishing fastest. Pick match, walk, or streamify for bounded workflows; merely swapping JSON.parse for bfj.parse does not remove the memory cost of the completed object.
Use it if
- A large JSON parse or stringify causes unacceptable event-loop stalls in a Node service
- You need token walking or selective extraction without retaining an entire JSON document
- You want backpressure-aware JSON serialization to a file, response, or transform stream
- You need NDJSON parsing and can follow bfj's sequential parse or event-stream conventions
- Raw throughput is the priority: the README answers its own speed question with no and explains that frequent yielding makes total processing slower
- You expect `read`, `parse`, or `stringify` to make the final value small: parse still resolves with the complete JavaScript structure and stringify still resolves with one complete string; use match, walk, streamify, or a different data format for bounded end-state memory
- You run in browsers, edge isolates, or Node before 18: current package metadata requires Node 18 or later and the API is built around Node files, streams, events, and CommonJS
- You require bundled TypeScript declarations or native ESM: 9.1.3 publishes a CommonJS main entry with no types or export map
- You need JSON Schema validation, JSON5 comments, arbitrary JSONPath, or general transformations: match supports only simple child access, indices, and wildcards, with no filters, scripts, or recursive descent
Setup reality
Install with npm install bfj. Version 9 requires Node 18 or later and has three runtime dependencies, but no peer dependencies, native build, credentials, config files, or service process. It is CommonJS and ships no TypeScript declarations, so ESM code relies on CommonJS interop and TypeScript users need a local declaration or community types. Choose the API based on what must remain bounded. `read(path)` and `parse(stream)` avoid one long synchronous JSON.parse turn, but resolve with the entire JavaScript value. `stringify(data)` yields during traversal but eventually creates one full JSON string. For genuinely incremental work, use `match` to emit selected values, `walk` for token events, or `streamify` and `write` for backpressure-aware output. The default `yieldRate` is 1024 items. Lowering it improves event-loop responsiveness at the cost of longer total processing; increasing it does the reverse. Serialization uses a fixed buffer, default length 256, and pauses when downstream backpressure fills it. BFJ serialization is not identical to JSON.stringify: by default it resolves promises, turns buffers into strings, maps into objects, and other iterables into arrays. Disable coercions you do not want. Circular references fail unless `circular: 'ignore'`, which silently omits them and can hide data loss. Syntax failures reject Promise APIs at the first error; streaming match and walk distinguish `dataError` for malformed JSON from `error` raised elsewhere. NDJSON support is uneven: walk and match can continue through the stream, parse returns one root value per sequential call, and read and unpipe do not support NDJSON. Add stream error handling, abort policy, input byte limits, depth expectations, and representative load tests before replacing JSON.parse in a server path.
Patterns
Asynchronously read a large JSON fileread-large-file
const bfj = require('bfj');
try {
const data = await bfj.read('./large.json', {yieldRate: 1024});
console.log(data.records.length);
} catch (error) {
console.error('Invalid or unreadable JSON', error);
}read yields during parsing but still resolves with the complete in-memory JavaScript value.
Parse JSON from a readable streamparse-readable-stream
const fs = require('node:fs');
const bfj = require('bfj');
const input = fs.createReadStream('./large.json');
const data = await bfj.parse(input, {yieldRate: 2048});A larger yieldRate usually finishes sooner but gives each event-loop turn more work.
Serialize data directly to a filewrite-large-file
const bfj = require('bfj');
await bfj.write('./output.json', data, {
space: 2,
bufferLength: 1024,
yieldRate: 1024,
});Pretty printing increases output size. write honors downstream pressure rather than assembling one giant string first.
Pipe serialized JSON with backpressurestream-json-response
const bfj = require('bfj');
response.setHeader('Content-Type', 'application/json');
const output = bfj.streamify(data, {bufferLength: 1024});
output.on('dataError', (error) => response.destroy(error));
output.on('error', (error) => response.destroy(error));
output.pipe(response);dataError reports problems in the value, such as circular references; error covers other stream failures.
Stringify without one long event-loop turnstringify-with-yields
const bfj = require('bfj');
const json = await bfj.stringify(data, {space: 0, yieldRate: 512});The traversal yields, but the resolved json is still one complete string in memory.
Stream selected values with simple JSONPathmatch-json-path
const fs = require('node:fs');
const bfj = require('bfj');
const matches = bfj.match(
fs.createReadStream('./catalog.json'),
'$.products[*]',
{minDepth: 2},
);
matches.on('data', (product) => processProduct(product));
matches.on('dataError', console.error);
matches.on('error', console.error);BFJ supports child properties, numeric indices, and wildcards, but not filters, scripts, or recursive descent.
Select items with a predicatematch-with-predicate
const selected = bfj.match(stream, (key, value, depth) => {
return depth === 3 && key === 'status' && value === 'pending';
}, {minDepth: 3});
for await (const value of selected) {
console.log(value);
}The emitted match is the value for which the predicate returned truthy, not its containing object.
Process JSON tokens without building a full valuewalk-json-tokens
const walker = bfj.walk(stream, {stringChunkSize: 64 * 1024});
walker.on(bfj.events.property, (name) => console.log('property', name));
walker.on(bfj.events.stringChunk, (chunk) => consume(chunk));
walker.on(bfj.events.dataError, console.error);
walker.on(bfj.events.error, console.error);stringChunk events do not replace the final string event; the complete string event still follows its chunks.
Pause and resume a walkpause-token-walk
const walker = bfj.walk(stream);
walker.on(bfj.events.object, async () => {
const resume = walker.pause();
await waitForCapacity();
resume();
});pause returns the resume function. Ensure every path resumes or deliberately terminates the source stream.
Read NDJSON values sequentiallyparse-ndjson
const fs = require('node:fs');
const bfj = require('bfj');
const stream = fs.createReadStream('./events.ndjson');
for (;;) {
const item = await bfj.parse(stream, {ndjson: true});
if (item === undefined) break;
await handle(item);
}Calls must be sequential. read and unpipe do not support NDJSON, and undefined is used as the end marker.
Transform values during parsingrevive-values
const data = await bfj.parse(stream, {
reviver(key, value) {
return key.endsWith('At') && typeof value === 'string'
? new Date(value)
: value;
},
});Like JSON.parse, the reviver runs depth-first and can increase CPU and retained memory.
Disable BFJ-specific value coercionscontrol-serialization-coercion
const output = bfj.streamify(data, {
promises: 'ignore',
buffers: 'ignore',
maps: 'ignore',
iterables: 'ignore',
circular: 'ignore',
});These options can silently omit or alter values. BFJ otherwise resolves promises and converts buffers, maps, and iterables automatically.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| stream-json | npm | You want composable token, filter, stream-array, and transform stages for high-throughput incremental processing |
| JSONStream | npm | You maintain an older Node stream pipeline built around JSONPath-like parse and stringify transforms |
| clarinet | npm | You want a small SAX-style evented parser for Node or browser code |
| jsonparse | npm | You need a low-level incremental parser and are prepared to build selection and stream handling yourself |