package
A micro-library of stream components for building custom JSON and JSONC processing pipelines with a minimal memory footprint — parse, filter, and transform JSON far larger than available memory with a SAX-inspired token API, on Node.js or Web Streams.
npm install stream-jsonGitHub-flavored package documentation.
Across all versions.
Download trend unavailable.
Runtime metadata.
Tagged topics.
stream-json is a micro-library of components for processing JSON files and streams, with a minimal memory footprint. Point it at a document far larger than available memory and it streams straight through — you pick out only the parts you care about and handle them one at a time, instead of loading the whole thing with JSON.parse. Even individual keys, strings, and numbers can be streamed piece by piece, and a SAX-inspired event API is included.
Each component is one stage of a pipeline: the parser turns text into a token stream, filters trim and reshape that stream on the fly, and streamers assemble the surviving tokens back into JavaScript objects. They compose with each other and with your own code through stream-chain, the zero-dependency library this one is built on; TypeScript typings are bundled.
Why it might be for you:
pick, ignore, replace, and filter keep just the subobjects you want out of a massive document and drop the rest — the bytes you skip are never assembled into memory.stream-json is built for data you own or trust — database dumps, exports, logs, and files produced by your own systems. It is not designed for hostile input: do not feed it JSON or JSONC from the open internet or from untrusted users. Untrusted JSON needs validation of its own before it reaches a pipeline.
Pull one array out of a JSON document larger than memory and tally it — one record at a time, in constant memory:
import {parser} from 'stream-json';
import {pick} from 'stream-json/filters/pick.js';
import {streamArray} from 'stream-json/streamers/stream-array.js';
import chain from 'stream-chain';
import fs from 'node:fs';
// data.json: { "meta": {...}, "data": [ ...millions of records... ] }
const pipeline = chain([
fs.createReadStream('data.json'), // a file far bigger than RAM is fine
parser(),
pick({filter: 'data'}), // descend into "data", ignore everything else
streamArray() // emit one array element at a time
]);
const byDepartment
This works because pick selected a single array — streamArray then streams its elements. When pick matches several subobjects, its output is a sequence of separate sub-trees — structurally the same token stream a JSON Streaming source produces — and streamValues() assembles each match into its own object:
import {parser} from 'stream-json';
import {pick} from 'stream-json/filters/pick.js';
import {streamValues} from 'stream-json/streamers/stream-values.js';
import chain from 'stream-chain';
import fs from 'node:fs';
// depts.json: {"departments": [
// {"name": "dev", "head": {"name": "Alice", "id": 1}, "staff": 20},
// {"name": "ops", "head": {"name": "Bob", "id": 2}, "staff": 10}
// ]}
const pipeline = chain([
fs.createReadStream('depts.json'),
parser(),
pick({filter: /^departments\.\d+\.head\b/}), // every department's "head" subobject
streamValues() // assemble each picked sub-tree
Each stage is a building block; stream-chain wires them into one stream and handles the streaming and backpressure. To read straight from a file you can drop createReadStream and use the Node-only parseFile(); to write a stream back to disk, use stringerToFile(). See Recipes for more.
npm install --save stream-jsonESM only; runs on actively-maintained Node, plus Bun and Deno — see Supported runtimes.
Want the details? Follow the links, or browse the wiki — index or search.
stream-json is built on (wire functions, generators, and streams into one chain); also home to streaming JSONL.stream-json-compatible token format: rows as arrays of strings, or as objects when a header row is present.BSD-3-Clause
maxDepth on parsers and verifiers and maxPackedLength on parsers, which cap the memory deep nesting and huge values can take, and lower memory use for RegExp filters (thx whguswo).close() on stringerToFile, a leading BOM accepted again (thx nemphys), a faster pick with once, Unicode-safe output for any byte sink, replacers that follow JSON.stringify (thx mayankdhingra02), and a filter fix (thx mkantautas).The full history is in the wiki: Release history.
maxDepth option on FlexAssembler rules (thx zx), significant performance improvements for string and RegExp filters.__proto__ key no longer replaces an assembled object's prototype; JSONC comments no longer rescan), JSONC comments stream as startComment / commentChunk / endComment plus commentValue (the comment token is renamed), replace takes any plain value as replacement, filters replay keys packed again. Thx Ryan Cruz, Iain, WorldSEnder, and Mike Tunnicliffe.maxDepth option on path filters. Thx ataberk.xyz.stream-json/utils/{pipe,drain} — use stream-chain's generic helpers directly; the file-edge internals now delegate to stream-chain.parseFile, stringerToFile, verifyFile), faithful JSONC comma round-trip (streamCommas / useCommas), JSONL delegated to stream-chain.stream-chain 4.x. See Migrating from 2.x to 3.x.stream-chain 3.x, bundled TypeScript definitions. New: JSONC parser/stringer, FlexAssembler. See Migrating from 1.x to 2.x.Estimated unpacked size across 2 packages.
687.8 kB
Includes dependencies.