package
RocksDB binding for Node.js
npm install @harperfast/rocksdb-jsGitHub-flavored package documentation.
Across all versions.
Download trend unavailable.
Runtime metadata.
Estimated unpacked size across 4 packages.
4.5 MB
Includes dependencies.
3 direct dependencies.
A Node.js binding for the RocksDB library.
'error' event or db.getLastError(), with in-process
recovery (db.resume())const db = RocksDatabase.open('/path/to/db');
for (const key of ['a', 'b', 'c', 'd', 'e']) {
await db.put
cb1kenobi
chris@cb1inc.com
new RocksDatabase(path, options?)Creates a new database instance.
path: string The path to write the database files to. This path does not need to exist, but the
parent directories do.options: object [optional]
arenaBlockSize: number The size in bytes of the blocks a column family's memtable allocates
from. Every column family that has been written to holds at least one block, and with a
writeBufferManagerSize set, those blocks count against that shared budget even when the
column family is idle — so with many column families, large blocks can use up most of a small
budget and cause constant tiny flushes. Defaults to 0, which picks a size automatically: with
a write buffer manager, writeBufferManagerSize / 2048, but at least 64 KiB and at most
writeBufferSize / 8 or 1 MiB, whichever is smaller (a 64 MiB budget gives 64 KiB, 512 MiB
gives 256 KiB, 2 GiB or more gives 1 MiB); without one, RocksDB's own default of
writeBufferSize / 8 capped at 1 MiB (1 MiB at the default writeBufferSize). The automatic
size uses the manager's budget at the time the column family is opened. An explicit value is
used as given, except that RocksDB raises one below 4 KiB to 4 KiB. Like writeBufferSize,
this applies to every column family when a database is first opened in the process and to
column families created later; a column family that is already open keeps its size.
bloomFilterBitsPerKey: number The size of the filter written into each SST file, in bits per
key. A get() of a key that is not in a file checks that file's filter and skips its index and
data blocks, so reads of absent keys, and of keys whose range overlaps several files (keys
written in random order, or updates still in level 0), touch fewer blocks. Filters cover whole
keys only, so range reads are unaffected. Defaults to 10 (about 1% false positives); otherwise
a number from 1 to 100, fractions allowed, or 0 for no filters. Each open SST file holds its
filter in memory beside its index, outside the block cache (about 1.25 bytes per key at 10;
a secondaryPath open holds every file open, so the whole database's filters). 0 also stops
an open from loading the filters files already have, so a reader can trade them for memory. An existing
database opens with the new setting, but its files gain or lose filters only as compaction
rewrites them (compact({ bottommost: true }) rewrites them all). Like writeBufferSize, this
applies to every column family when a database is first opened in the process and to column
families created later; a column family that is already open keeps its setting.
compression: string | { algorithm: string, level?: number } The block/blob compression
algorithm for this column family. Pass an algorithm name — one of 'none', 'snappy',
'zlib', 'bzip2', 'lz4', 'lz4hc', or 'zstd' — or an object with an
algorithm and an optional level (forwarded to RocksDB's compression_opts.level; the
meaning is algorithm-specific). Applies to both SST data blocks and blob files (large values).
Defaults to 'lz4' when the native build supports it, otherwise RocksDB's own default (Snappy
when linked, else no compression). See Compression. Throws if the algorithm is
not compiled into the native build — check supportedCompression for
the available list.
compressionForAllColumnFamilies: boolean When true, applies compression to every column
family opened for the database rather than only the column family specified by name. Requires an explicit
compression. Defaults to false. See Compression.
dbWriteBufferSize: number The total memtable memory budget in bytes shared across all of the
database's column families. When the combined size of all memtables reaches this value, RocksDB
flushes the largest one. 0 (the default) disables this global trigger, so per-column-family
writeBufferSize alone drives flushing. This is distinct from the process-wide
writeBufferManagerSize config option. Database-wide, so it binds when the
path is first opened in this process: a later open of the same path — including from another
worker thread — keeps the first opener's value rather than overriding or rejecting it.
disableWAL: boolean Whether to disable the RocksDB write ahead log. Defaults to false.
enableStats: boolean When true and the database is open, RocksDB will captures stats that
are retrieved by calling db.getStats(). Enabling statistics imposes 5-10% in overhead.
Defaults to false.
infoLogLevel: number The verbosity of RocksDB's informational logging (LOG /
LOG.old.*): 0 (debug), 1 (info), 2 (warn), 3 (error), 4 (fatal), or 5
(header-only). Omit to leave RocksDB's own default (INFO_LEVEL in a release build of the
linked RocksDB library). See db.logOptions.
maxLogFileSize: number The per-file size cap, in bytes, for informational log files (LOG /
LOG.old.*). RocksDB retains up to 5 of these files, so the total informational-log footprint
is bounded at roughly 5 * maxLogFileSize. Defaults to 16 MB (an 80 MB bound), which stops
purely informational logging from growing without bound. A value of 0 is RocksDB's special
"single unbounded log file" mode — it disables size-based rotation entirely, so the log
can grow without limit; only set 0 if you deliberately want that (it forgoes the bounded
footprint this option otherwise provides). See
db.logOptions.
maxOpenFiles: number The maximum number of table files RocksDB keeps open. 0 (the default)
derives a budget from the effective per-process open-file limit (an eighth of the limit —
several databases can share one process — clamped to [1024, 262144]); -1 holds every table
file open (the RocksDB default, which can exhaust the process file-descriptor limit when
compaction falls behind under sustained ingest); a positive int32 is an explicit cap. Reads
only pay a reopen cost when the number of live table files exceeds the budget, so raise the
process fd limit (and with it the derived budget) for very large databases.
maxWriteBufferNumber: number The maximum number of memtables that can be queued per column
family before writes stall. Higher values absorb write bursts while flushes catch up, at the
cost of memory (roughly maxWriteBufferNumber * writeBufferSize per column family). Defaults to
16.
maxWriteBufferSizeToMaintain: number The number of bytes of recent memtable history to keep in
memory for transaction conflict checking. -1 (the default) derives the value from
maxWriteBufferNumber * writeBufferSize (the RocksDB-recommended default for optimistic
transactions) — except when the database has a writeBufferManager
attached, in which case it resolves to 1 so the manager is not filled with history it will
never release. What decides this is the manager the database itself holds, not the current
writeBufferManagerSize: a column family created later on an already-open database is clamped
too, because its history is still charged to that manager. It also applies to any attached
manager, not only a stalling one — writeBufferManagerAllowStall can be changed at runtime
while this value is fixed when a column family is created, and history the budget cannot reclaim
is a problem either way. 1 is the smallest target a transactional database can be given: RocksDB's transaction wrappers rewrite a 0 target to
the derived value, so 0 requests the largest history rather than none, and an explicitly
requested 0 is normalized to 1 for the same reason. A positive target still retains the most
recent flushed memtable per column family until that family's next write. An explicit positive
value is honored as-is; sizing it against the budget and the column-family count is then yours,
and a configuration whose known families already reach the budget is reported on the
'log.warn' channel.
name: string The column family name. Defaults to "default".
noBlockCache: boolean When true, disables the block cache. Block caching is enabled by
default and the cache is shared across all database instances.
parallelismThreads: number The number of background threads to use for flush and compaction.
Defaults to 1.
pessimistic: boolean When true, throws conflict errors when they occur instead of waiting
until commit. Defaults to false.
readOnly: boolean When true, the database is opened in read-only mode. Read operations are
permitted. Write operations will throw an error with code ERR_DATABASE_READONLY. Transactions
are a no-op in read-only mode. flush(),
flushSync(), compact(),
compactSync() and syncWal() are the
exception: they have nothing to do when there are no writes, so they succeed as no-ops rather
than throwing.
A read-only open is a point-in-time snapshot: it never sees later writes. It is safe
against a quiescent database, but unsafe against a live writer: the open holds no reference
on the files it is about to read, so a concurrent compaction, blob GC, or flush in the writing
process can delete one mid-open and the open throws with code ERR_CONCURRENT_COMPACTION — the
database is not corrupt; the reader lost a race, and retrying can succeed. Even a successful
open can fail later reads the same way, because files the snapshot references are opened
lazily while the writer keeps deleting obsolete ones. To follow a database another process is
actively writing, open a secondary instead (secondaryPath below) — that is the supported mode
for a live follower.
secondaryPath: string Opens the database as a secondary instance: a read-only follower of
a live primary that tolerates the primary deleting files and sees new writes on each
catchUpWithPrimary() call. This is the supported way to
read a database another process is actively writing. The value is the secondary instance's own
workspace directory (created if missing) where RocksDB keeps the secondary's private state — it
must be outside path (enforced) and exclusive to one secondary instance. Exclusivity is
enforced: reusing a workspace in-process for a different database is rejected at open (the same
database + workspace share the one follower instance), and a kernel advisory lock on
<secondaryPath>/.secondary.lock (same discipline and caveats as the
backup directory lock) excludes other processes — with the same limits: on
filesystems without advisory locking (flock unsupported — e.g. the FUSE/9p mounts behind
Docker Desktop bind mounts) the lock degrades to a no-op, and on network filesystems with
node-local flock (NFS local_lock, CIFS, 9p) it does not exclude across hosts, so workspace
exclusivity is the caller's job there. Implies
readOnly: true (an explicit readOnly: false throws), so all read-only behavior above
applies. Forces maxOpenFiles: -1: every table and blob file is opened and held for the life
of each version, which is what makes the primary's deletions safe — budget file descriptors
accordingly on large databases (an explicit maxOpenFiles other than -1 is rejected). Column
families created by the primary after the secondary opens are invisible until the secondary
reopens. Note: RocksDB upstream documents secondary instances as unsupported in combination
with integrated BlobDB (which this library enables for values ≥ 2KB); the pinned RocksDB build
handles blob files in secondary mode — covered by native regression tests — but upstream does
not guarantee the combination.
statsLevel: StatsLevel Controls which type of statistics to skip and reduce statistic
overhead. Defaults to StatsLevel.ExceptDetailedTimers.
store: Store A custom store that handles all interaction between the RocksDatabase or
Transaction instances and the native database interface. A store is bound to a single
RocksDatabase instance and cannot be shared between them. See Custom Store for
more information.
transactionLogMaxAgeThreshold: number The threshold for the transaction log file's last
modified time to be older than the retention period before it is rotated to the next sequence
number. Value must be between 0.0 and 1.0. A threshold of 0.0 means ignore age check.
Defaults to 0.75.
transactionLogMaxSize: number The target maximum size of a transaction log file. Transactions
are never split across files: if the complete transaction does not fit, the log rotates before
writing it. A transaction written to an empty file may exceed the target. Defaults to 16 MB.
0 is rejected: it formerly meant unlimited segment size, which a reader's fixed-size mapping
cannot cover. A store configured with 0 must omit the option or set a size before opening;
once it rotates by size, rotated segments older than transactionLogRetention are purged, so
raise the retention to keep more history.
transactionLogRetention: string | number The number of minutes to retain transaction logs
before purging. Defaults to '3d' (3 days).
transactionLogsPath: string The path to store transaction logs. Defaults to
"${db.path}/transaction_logs".
timestampFloorLog: string The name of the transaction log whose batch keys this process
originates. At open, the process-wide monotonic clock is raised above every batch key still
durable in that log, so a backward wall-clock step between runs cannot reissue a transaction
timestamp that is already a key in it. See
Timestamp floor at open. Unset by default, which leaves the clock
alone. Its first use in a process must be on a writable primary open; read-only and secondary
handles can reuse an already-established seed for the same log but cannot establish one.
If that first seed warned because the log was absent, the log may be created fresh, but a
nonempty directory that reappears before first use is refused until every handle closes and the
path reopens with this option. The name must be a single path component that does not start with
.: startup discovery skips such directories, so the log could never be scanned again.
verificationTable: boolean When true, this handle participates in the process-global
Verification Table: writes through it invalidate the verification slot
for each written key, and reads through it may seed slots. Enable this only for column families
whose records are cached (e.g. the primary column family of a table). Defaults to false; a
handle that leaves it unset joins a column family another handle opened with true, and an
explicit false throws while any handle on that column family participates (see
Verification Table). Requires verificationTableEntries to be configured
before the first database is opened.
writeBufferSize: number The per-column-family memtable size in bytes at which the memtable is
sealed and flushed to an SST file. Smaller values produce more frequent, faster flushes; larger
values batch more writes per SST file at the cost of memory. Defaults to 16777216 (16 MB).
db.close()Closes a database. This function can be called multiple times and will only close an opened
database. A database instance can be reopened once it is closed. Closing the last handle flushes the
memtables and waits for that flush, then cancels background compaction instead of waiting for it:
a running compaction is abandoned and redone later, so close does not wait for the compaction
backlog (the flush itself still waits out a write stall). A flush failure leaves the native database quarantined so shutdown()
can retry without losing unflushed data; an explicit destroy() can instead delete it. A failure
from RocksDB's own final close is reported after native teardown completes; the optional
compactOnClose pass still runs to completion before background work is cancelled, but it is
best-effort and its errors do not fail the close, since a skipped compaction loses no data. All
native close errors emit database:closeFailed. The quarantine applies to both writable and
read-only opens because both modes share the physical path lifecycle.
const db = RocksDatabase.open('foo');
db.close();db.columns: string[]Returns the list of column families in the RocksDB database.
const db = RocksDatabase.open('path/to/db');
console.log(db.columns); // ['default']
db.use('users');
console.log(db.columns); // ['default', 'users']db.compression: { algorithm: string, level?: number }Returns the compression currently in effect for this database's column family, read live from
RocksDB. algorithm is a friendly name (e.g. 'lz4', 'zstd', 'none'); level is present only
when a non-default compression level is set. The database must be open. See
Compression.
const db = RocksDatabase.open('path/to/db', { compression: { algorithm: 'zstd', level: 3 } });
console.log(db.compression); // { algorithm: 'zstd', level: 3 }db.setCompression(compression)Dynamically changes the compression algorithm (and optional level) for this database's column family on an already-open database — no close, no reopen. See Compression for full semantics and caveats.
const db = RocksDatabase.open('path/to/db', { compression: 'none' });
db.setCompression({ algorithm: 'zstd', level: 19 });db.config(options)Sets global database settings.
options: object
blockCacheSize: number The amount of memory in bytes to use to cache uncompressed blocks.
Defaults to 32MB. Set to 0 (zero) disables block cache for future opened databases. Existing
block cache for any opened databases is resized immediately. Negative values throw an error.commitThreads: number The maximum number of dedicated threads each database opened after the
call uses to run async transaction.commit() calls. Defaults to min(4, cores); accepts
integers from 1 through 64. A thread starts only when a commit finds every started thread
busy, so a database that is never committed to concurrently owns one. Async commits never use
the libuv threadpool, so slow commits cannot delay fs, dns, crypto or async get() calls.
With more than one thread, RocksDB validates and applies concurrent commits on several cores
and can combine them into one write: with four worker threads committing 64-key transactions to
one database, this committed about twice as many transactions per second as one thread.
Concurrent commits finish, and their promises resolve, in any order, as concurrent
commitSync() calls always have, and their transaction-log entries are appended in the order
the commits reach the log; a log reader never sees an entry before its transaction's data is
committed. Log position is not the order in which writes to the same key were applied: a
commit that conflicts and is retried keeps its original log position but lands after commits
logged behind it. Its transaction timestamp does not advance across a retry either, so
replaying entries for one key in timestamp order does not recover true apply order — a
consumer that needs one must have the producer embed its own last-writer-wins marker.
Concurrent commit threads make this more frequent; a single commit thread and commitSync()
callers could already produce it. With 1, a database's async commits run one at a time in
the order commit() was called. Databases already open keep their limit.compactOnClose: boolean When true, compacts the database on close. Defaults to false.shutdownOnExit: boolean When true, the main thread runs shutdown()
from a process.on('exit') listener, so process.exit() still flushes and closes every
database. Defaults to true. Set to false to leave memtables unflushed at a Node
process.exit(), e.g. to exercise crash recovery.transactionLogAbortMarkers: boolean When true, a transaction that wrote its
transaction-log batch and then ends without committing (an abort(), a commit error it gives
up on, or a worker torn down with it pending) gets that batch marked aborted in place, and every
reader skips it; see Abort markers. Defaults to false, which behaves as
before. Read when a transaction starts and when it is reset for a retry.transactionLogWriteIntents: boolean When true with transactionLogAbortMarkers on, a
conflicting attempt is refused before it writes its batch when an older transaction holds a
write intent on one of its keys; see Abort markers. This shortens the worst
waits of transactions that overlap heavily, but adds retries and costs throughput when
transactions are large or contention is low. Defaults to false. Needs verificationTable.
Read with transactionLogAbortMarkers.occValidation: 'parallel' | 'serial' How writable optimistic databases opened after the call
validate commits for conflicts. Defaults to 'parallel' (RocksDB's default). Databases already
open keep their policy. Pessimistic, read-only and secondary opens are unaffected.
'parallel' locks the commit's keys in a per-database pool of occLockBuckets mutexes,
checks for conflicts, and holds the locks through the write. Commits on different threads
that touch different buckets can then be batched into one RocksDB write. Non-transactional
writes (db.putSync() and friends) take no bucket, so one that lands between a commit's
conflict check and its write is overwritten by that commit without an ERR_BUSY.'serial' checks for conflicts inside RocksDB's write group instead, and allocates no bucket
pool, which saves its memory. Because the check and the write happen in one step, a
non-transactional write cannot land between them and be silently overwritten, and a column
family cannot be dropped between them. The cost: RocksDB never batches a serial-validated
commit with another writer, including the database's other commit threads, so it gives back
what commitThreads gains. With four worker threads
committing 64-key transactions to one database it committed about 4,500 transactions per
second against about 9,900 for 'parallel', the same as a single commit thread; it came out
ahead (about 7%) only at 1,000 keys per transaction with the default bucket count, where
raising occLockBuckets gains more. Concurrent commitSync() calls lose throughput the same
way. It suits a database committed to one transaction at a time (commitThreads: 1).occLockBuckets: number The number of commit lock buckets each writable optimistic database
allocates when it is opened with occValidation: 'parallel'. Defaults to 65536 (2^16), not
RocksDB's 2^20. Accepts integers from 16 through 16777216 (2^24). Applies to databases
opened after the call; databases already open keep their count. Memory is the count times the
platform mutex size (40 bytes on Linux x64, 64 bytes on macOS), so the default is about 2.5 MiB
per database on Linux, against about 40 MiB at RocksDB's default. Multiple keys can share a
bucket, so this is not a transaction write limit. A small pool costs throughput only when
commits to the same database overlap (its commitThreads, commitSync() callers, or libuv
commits under ROCKSDB_JS_COMMIT_THREAD=0) and their keys share a bucket: colliding commits
run one at a time and cannot share a RocksDB write. Collisions grow with transaction size. With
four worker threads and the default four commit threads, 65536 buckets matched 2^20 and 2^22 at
64 keys per transaction (and 4096 gave about a third fewer commits per second), but at 1,000 keys
65536 committed 336 transactions per second against 466 at 2^20 and 685 at 2^22, so databases
with large transactions should raise the count.lifecycleWaitSeconds: number How long a synchronous open, destroy, or shutdown waits for a
conflicting lifecycle operation already in progress on the same path (e.g. another open or
close) before throwing a retryable timeout error. It does not bound the separate, intentionally
unbounded wait that destroy()/shutdown() make for in-flight backups, checkpoints, or other
async work still using the database — see db.destroy(). Defaults to 30
seconds and must be a positive integer.verificationTableEntries: number The number of slots in the process-global
Verification Table. Each slot is 8 bytes, so the default of 131072
(128K) slots is 1 MB. Set to 0 to disable the verification table. This must be configured
before the first database is opened; once the table is materialized, attempts to change this
value throw.writeBufferManagerAllowStall: boolean When true, writes are stalled once the manager's
buffer_size is exceeded, providing a hard cap on memtable memory. When false, memtables are
allowed to grow past the limit and flushes are simply scheduled more aggressively. Off by
default to favor write throughput over hard memory bounding. Defaults to false. A stall here
is invisible to RocksDB's own stall counters and to
db.isWriteStalled() — see
getWriteBufferManagerStats(),
which also describes the watchdog that logs a sustained stall.writeBufferManagerFlushPolicy: 'oldest' | 'largest' | 'largestAcrossDBs' What the write
buffer manager flushes when it is over budget. 'oldest' flushes the oldest memtable of the
database that is being written; 'largest' its largest memtable (databases here are opened with
atomic flush, so this currently behaves like 'oldest'); 'largestAcrossDBs' has a background
thread flush whichever database holds the most memtable memory, whether or not it is being
written. When not set, the policy is 'largestAcrossDBs' while
writeBufferManagerAllowStall is true and 'oldest' otherwise, and it follows later changes
to writeBufferManagerAllowStall. With stalls allowed, 'oldest' can stop writes permanently:
a write to one database waits for memory held by other databases that nothing is writing to,
so nothing flushes them. Without stalls, 'largestAcrossDBs' has measured slower. Can be
changed while databases are open. Once set, the value stays in effect for the life of the
process; it is reported as flushPolicy by
getWriteBufferManagerStats().writeBufferManagerCostToCache: boolean When true, memtable memory is "charged" against the
shared block cache so the block cache and write buffers draw from a single pool. During write
bursts the cache shrinks to make room for memtables; once memtables flush, the cache can grow
back into the reclaimed space. Defaults to false.writeBufferManagerSize: number Total memtable memory limit (bytes) shared across every
database opened in this process. When set, RocksDB uses a single WriteBufferManager so write
buffers are bounded process-wide rather than per database. Defaults to 0, which means no
manager. Setting 0 later stops new opens from attaching one, but does not detach or resize
the manager a database already holds — write_buffer_manager is fixed for the life of an open
database, so its memtables stay charged against that budget.RocksDatabase.config({
blockCacheSize: 100 * 1024 * 1024, // 100MB
commitThreads: 4, // per database; applies to subsequent opens
compactOnClose: true,
occLockBuckets: 1 << 20, // per optimistic database with parallel validation
occValidation: 'parallel', // or 'serial'; applies to subsequent optimistic opens
writeBufferManagerAllowStall: false,
writeBufferManagerCostToCache: false,
writeBufferManagerSize: 64 * 1024 * 1024, // 64MB
});getWriteBufferManagerStats(): WriteBufferManagerStatsReads the live state of the WriteBufferManager.
Process-wide, not per-database. The manager is a singleton shared by every database opened in
this process, worker_threads included, so these values describe the whole process regardless of
where the call was made.
enabled: boolean Whether a manager has been created. bufferSize, memoryUsage,
mutableMemoryUsage, stallActive, stallActiveMs, watchdogRunning and columnFamilies are
0/false when not; allowStall and costToCache still reflect the configured setting (a
manager is only created once writeBufferManagerSize is also set), and inventoryAvailable is
true (there is nothing to fail to collect).bufferSize: number The budget in bytes (writeBufferManagerSize, read live).memoryUsage: number Total memtable memory in bytes charged against the manager.mutableMemoryUsage: number The share of memoryUsage held by active (mutable) memtables; the
rest is memtables awaiting flush plus retained write history.allowStall: boolean, costToCache: boolean The manager's configuration.flushPolicy: 'oldest' | 'largest' | 'largestAcrossDBs' The flush policy in effect (see
writeBufferManagerFlushPolicy); before a manager exists, the policy one would be created with.stallActive: boolean Whether the manager is currently stalling writes.stallActiveMs: number How long the current stall has been active; 0 when not stalled.
Sampled once a second by the watchdog, so 0 for a stall's first second and whenever
watchdogRunning is false.watchdogRunning: boolean Whether the stall watchdog thread is running.columnFamilies: number Live column families across every database attached to this manager. A
dropped column family keeps charging the manager until its last handle closes, so it is counted
until then.inventoryAvailable: boolean false when the inventory could not be collected because the
database registry or a column-family inventory was locked. The two inventory fields are then
empty and everything else is still live; the call never blocks on database work that may itself
be stalled.maxWriteBufferSizeToMaintain: Record<string, number> Effective per-column-family retained-history
target (as a decimal string) to how many of those column families carry it. Effective, not
requested: RocksDB rewrites a requested 0 for a transaction database.bufferSize, memoryUsage, mutableMemoryUsage, stallActive and stallActiveMs are also in
db.getStats() and db.getStat() under the same
writeBufferManager. prefix, for scraping; enabled, allowStall, costToCache, flushPolicy,
watchdogRunning, and the column-family inventory are only here, because collecting them either
walks the database registry or isn't scrape-shaped.
stallActive is the signal that distinguishes a stalled process from an idle one.
db.isWriteStalled(), the 'writeStall' event
and rocksdb.stall.micros all track RocksDB's WriteController, which a WriteBufferManager
stall never goes through.
While a manager exists with writeBufferManagerAllowStall, one low-frequency thread samples the
manager once a second. When a stall has been continuously active past
ROCKSDB_JS_WBM_STALL_WARN_MS (default 5000; 0 disables the watchdog) it writes one line
to stderr and emits it as a process-wide 'log.warn' event — once per stall episode, not per
blocked writer and not per sample:
[rocksdb-js] WriteBufferManager write stall active for 5.0s - no write can complete until memtable
memory drops below the budget. budget=661.2MB usage=662.2MB (100.2%) mutable=12.4MB (1.9%)
allowStall=true costToCache=true columnFamilies=28 maxWriteBufferSizeToMaintain={268435456:28}It needs its own thread because every other candidate is blocked by the very condition it reports:
a writer is parked inside RocksDB, RocksDB's stall callbacks are WriteController-only, and a JS
timer cannot fire on a thread parked in a synchronous write. The line goes to stderr as well as
the event so it is not lost when nothing has registered a 'log.warn' listener; route one and
ignore stderr if you would rather have it in your own log.
RocksDatabase.on('log.warn', (message) => logger.warn(message));
const wbm = getWriteBufferManagerStats();
if (wbm.stallActive) {
logger.warn(`writes stalled for ${wbm.stallActiveMs}ms: ${wbm.memoryUsage}/${wbm.bufferSize}`);
}db.isOpen(): booleanReturns true if the database is open, otherwise false.
console.log(db.isOpen()); // true or falsedb.logOptions: { maxLogFileSize: number, infoLogLevel: number }Returns the informational-log settings currently in effect for this database, read live from
RocksDB. These are database-wide settings (not per-column-family). maxLogFileSize is the
per-file size cap for informational log files (LOG / LOG.old.*); infoLogLevel is the logging
verbosity. The database must be open.
const db = RocksDatabase.open('path/to/db', { maxLogFileSize: 4 * 1024 * 1024 });
console.log(db.logOptions); // { maxLogFileSize: 4194304, infoLogLevel: 1 }db.name: stringReturns the database column family's name.
const db = new RocksDatabase('path/to/db');
console.log(db.name); // 'default'
const db2 = new RocksDatabase('path/to/db', { name: 'users' });
console.log(db.name); // 'users'db.open(): RocksDatabaseOpens the database at the given path. This must be called before performing any data operations.
import { RocksDatabase } from '@harperfast/rocksdb-js';
const db = new RocksDatabase('path/to/db');
db.open();There's also a static open() method for convenience that performs the same thing:
const db = RocksDatabase.open('path/to/db');db.secondaryPath: string | undefinedThe secondary instance's workspace directory when the database was opened as a
secondary (a read-only follower of a live primary), or
undefined for a regular or plain read-only open.
const follower = RocksDatabase.open('/path/to/database', {
secondaryPath: '/path/to/follower-workspace',
});
console.log(follower.secondaryPath); // '/path/to/follower-workspace'
console.log(follower.readOnly); // truedb.status: 'opened' | 'closed'Returns a string 'opened' or 'closed' indicating if the database is opened or closed.
console.log(db.status);db.use(name, options?): RocksDatabaseReturns a RocksDatabase bound to the name column family of this same database, opening — and
creating, if it does not exist — the column family on first use. This is a factory (like
useLog), not a stateful switch: the returned view is an
independent instance whose own reads and writes target its column family, while sharing the same
underlying database. Because a single RocksDB database backs every column family, a transaction,
backup, or checkpoint still spans all of them.
name: string The column family name.options?: object Options for the column family (same shape as the constructor's, minus name),
overriding the options inherited from this database. Options only take effect when the view is
(re)opened.use() is get-or-create, backed by a weak cache: db.use('events') === db.use('events') while the
view is still referenced and open, and calling with this database's own column-family name
returns this. A view that has been closed (or garbage-collected — the cache does not pin it) is
transparently recreated on the next use(). Views are independent handles: closing this database
does not close them (and vice versa); the underlying database stays open until every handle is closed
or collected.
The view's store is derived by Store#createColumnFamilyStore(name, options), which builds an
independent store of the same class with its own codec state — views never share a mutable
encoder/decoder. Consequently a database configured with a pre-constructed encoder/decoder instance
cannot derive views (its name/structures would be shared and corrupt both column families); use an
encoder factory ({ Encoder }) or a named encoding instead. A custom Store whose constructor
can't be recreated from (path, options) (e.g. it takes injected dependencies) should override
createColumnFamilyStore to build its views.
const db = RocksDatabase.open('path/to/db');
const events = db.use('events');
await events.put('e1', payload);
await db.put('k', 'v'); // default column family, unaffected
console.log(events.get('e1')); // payload
console.log(db.get('e1')); // undefined — different column family
events.close();
db.close();db.catchUpWithPrimary(): Promise<void>Advances a secondary instance to the primary's current state by tailing and replaying the primary's MANIFEST and WAL. A secondary does not see the primary's writes — flushed or not — until it catches up. Reads through the handle remain safe while the catch-up runs; catch-ups on the same database are serialized internally.
Throws with code ERR_NOT_SECONDARY on a database that was not opened with secondaryPath.
Column families the primary created after the secondary opened stay invisible until the secondary
reopens; ones the primary dropped remain readable until then. Catch-up advances the database
view only: transaction-log reads through a secondary serve the stores discovered at open, and a
store the primary creates afterward becomes visible only on reopen. Whether entries appended to an
already-open store are visible depends on where the writer is: a cross-process primary's appends
are not (the reader's view of the file extent is fixed at open), while a writer in the same
process shares the store object, so its appends are. Either way a log entry can describe data the
database view does not have yet — the log write completes before the RocksDB commit for every
writer, so log-leads-database is the normal direction and a consumer has to tolerate it. Serialize
catch-up calls per database. Each async call holds a libuv worker for the whole replay, and
concurrent calls queue on an internal per-database mutex while holding theirs, so a handful of
overlapping catch-ups on a backlogged follower can exhaust the default four-thread pool and stall
unrelated fs/dns/crypto work in the process. Await one before starting the next rather than
firing one per timer tick.
const primary = RocksDatabase.open('/path/to/database');
const follower = RocksDatabase.open('/path/to/database', {
secondaryPath: '/path/to/follower-workspace',
});
primary.putSync('foo', 'bar');
console.log(follower.getSync('foo')); // undefined — not caught up yet
await follower.catchUpWithPrimary();
console.log(follower.getSync('foo')); // 'bar'db.catchUpWithPrimarySync(): voidSynchronous version of catchUpWithPrimary(). The replay runs on the JS thread, so prefer the
async form unless the caller is already blocking.
follower.catchUpWithPrimarySync();db.clear(options?): Promise<number>Asychronously removes all data in the current database.
options: object
batchSize?: number The number of records to remove at once. Defaults to 10000.Returns the number of entries that were removed.
Note: This does not remove data from other column families within the same database path.
for (let i = 0; i < 10; i++) {
db.putSync(`key${i}`, `value${i}`);
}
const entriesRemoved = await db.clear();
console.log(entriesRemoved); // 10db.clearSync(options?): numberSynchronous version of db.clear().
options: object
batchSize?: number The number of records to remove at once. Defaults to 10000.for (let i = 0; i < 10; i++) {
db.putSync(`key${i}`, `value${i}`);
}
const entriesRemoved = db.clearSync();
console.log(entriesRemoved); // 10db.compact(options?): Promise<void>Compacts a range of keys in the database. In RocksDB, deleted keys are not immediately removed from the database. Instead, they are marked as deleted and a tombstone is written. This function triggers a manual compaction which removes the tombstones and reclaims space. Only one compaction per database path can be performed at a time.
options: object
start?: Key The start key of the range to compact.end?: Key The end key of the range to compact.bottommost?: boolean Also compact the bottommost level, rewriting every file in range.
RocksDB skips that level by default when no compaction filter is installed, and it holds most
of the data — so an ordinary compaction leaves it untouched. Because a changed
compression algorithm governs only newly written files, this is the way to
re-encode data that already exists. It rewrites the whole range regardless of whether RocksDB
considers it worthwhile, so it costs as much as the data is large. Defaults to false.await db.compact();
await db.compact({ start: 'a', end: 'z' });
// Re-encode everything already on disk under the column family's current codec
await db.compact({ bottommost: true });On a read-only database this is a no-op: arguments are still validated, but the returned promise resolves without compacting rather than rejecting.
db.compactSync(options?): voidSynchronous version of compact(). On a read-only database it
validates its arguments and then returns without compacting, rather than throwing.
db.compactSync();
db.compactSync({ start: 'a', end: 'z' });
db.compactSync({ bottommost: true });db.destroy(): voidCompletely removes a database based on the db instance's path including all data, column families,
and files on disk. Destruction owns the physical path for the process: it closes every writable and
read-only handle for that path, waits for registered backups and checkpoints to stop using the
native database, and prevents another handle from reopening the path until removal finishes. Those
waits are synchronous and can outlive lifecycleWaitSeconds once destruction has claimed the path,
because releasing the native database beneath an active copy would be unsafe.
A previously opened instance does not need to remain open, which allows an explicit destroy()
retry after failed physical cleanup. A never-opened or read-only instance cannot destroy the
database. shutdown() reports a pending cleanup tombstone but never retries deletion; only an
explicit destroy() can remove the path. Destruction flushes the memtables before it closes the
native database, without waiting out a write stall, so a database whose files outlive the call (the
deletion fails, or the process dies first) keeps writes made with disableWAL. If that flush fails,
destruction still deletes the database and reports the flush error through database:closeFailed;
only when the deletion then fails too are the unflushed disableWAL writes lost.
db.destroy();
console.log(fs.existsSync(db.path)); // falsedb.drop(): Promise<void>Drops the column family the database was opened with (name). For the default column family this
clears all entries instead.
const db = RocksDatabase.open('path/to/db', { name: 'users' });
await db.drop();
db.close();A drop retires the column family logically before it returns: the name is gone from
db.columns, a later open() with the same name creates a fresh, empty column family, and any
transaction that then stages a write to a handle of the dropped family, or commits one it staged
earlier, is refused whole with ERR_COLUMN_FAMILY_DROPPED (Column family "users" was dropped).
That terminal refusal releases the transaction's verification-table intents and bars further
writes or commit attempts; retained reads continue until the caller aborts the transaction, and
they still serve that transaction's own staged writes — values no commit will ever produce. A
caller that catches the refusal instead of letting it propagate must not read a value back through
the transaction and carry it forward.
If a staging call fails while the generation is being retired, retirement takes precedence and the
transaction is refused whole even when the immediate RocksDB failure was a pessimistic lock timeout.
Handles other threads still hold keep reading the dropped data until they close; a
non-transactional putSync/removeSync through such a handle is discarded.
The physical RocksDB drop is deferred behind commits already admitted when the drop lands: a
commit claims every column family its batch names before it writes its transaction-log batch and
releases them after RocksDB has applied it, and the physical drop runs from whichever
side releases last (or, for a commit a mid-flight close() tore out of its pipeline, from the next
drop, open of that name, or close on the database). With no such commit (the common case) drop()/dropSync() perform the
physical drop before returning, exactly as before. This is what keeps a drop racing another
thread's commit from latching RocksDB's fatal Invalid column family specified in write batch
error on the whole database.
Consequences to know about:
open() of a name whose previous generation is still held by an admitted commit waits for the
full admission-to-reclamation interval (bounded by ROCKSDB_JS_CF_RECLAIM_WAIT_MS, default
30000) before creating the fresh column family; if the previous generation's physical drop
failed, the open retries it once and throws with that error if it fails again.open() of that name, or close, and is
reported through the global log.warn event and the columnFamily.pendingReclaims stat. When
the failing drop was the one drop() itself ran, the call rejects with that error as well.What is not guaranteed: a process that exits while a physical drop is still pending, or after one failed, leaves the column family on disk under its name, and the next open of the database opens it as a live column family. A backup or checkpoint taken inside that window copies it. Callers that need a drop to survive a crash record their own durable tombstone before acknowledging it.
db.dropSync(): voidSynchronous version of db.drop(), with the same deferral contract.
const db = RocksDatabase.open('path/to/db');
db.dropSync();
db.close();db.flush(options?): Promise<void>Flushes all in-memory data to disk asynchronously.
options: object
allowWriteStall?: boolean Whether the flush may proceed even though it will stall writes for
its duration. Defaults to false, which — despite how that reads — means the flush waits
until it can run without causing a stall, with no timeout: on a database stuck in a stall
condition (immutable-memtable backlog, L0 stop trigger, an exhausted WriteBufferManager
budget), the returned promise never settles, and since flush() runs on the libuv threadpool
(default size 4), a handful of concurrently stalled flushes can exhaust the pool and stall
every unrelated fs/dns/crypto call and cold-cache get() in the process, not only this
database's. Pass true up front, before issuing a flush you expect might stall, when it's a
durability gate you'd rather have stall writers than wait indefinitely — there's no way to
cancel an in-flight flush(), so this isn't a rescue for one already hung, and forcing the
memtable switch can itself prolong an existing L0 stop-trigger condition rather than clear it.
That cost is database-wide, covering every column family on the (process-global,
worker_threads-shared) database handle, not just the caller's — and it relocates the hang
rather than removing it: a stalled write blocks each of the database's commit threads that
reaches it, so every async Transaction.commit() behind them queues up too, including ones
from callers that never touched flush. This is a different knob from
writeBufferManagerAllowStall (see new RocksDatabase()
options), with nearly opposite polarity: that one governs whether the WriteBufferManager may
stall writers at all, this one governs whether one manual flush is willing to cause a stall
rather than wait one out.await db.flush();
// Chosen up front, as a durability gate willing to pay the stall cost
await db.flush({ allowWriteStall: true });On a read-only database this is a no-op: the options bag is still validated, but the returned promise resolves without flushing rather than rejecting.
db.flushSync(options?): voidFlushes all in-memory data to disk synchronously. Note that this can be an expensive operation, so
it is recommended to use flush() if you want to keep the event loop free.
options: object Same as flush(). The default
allowWriteStall: false wait is taken on the calling thread, which here is the JS thread — so
the hazard described there is strictly larger on this entry point, not smaller: instead of
parking one libuv pool worker it freezes the event loop outright, and it holds the in-flight
operation claim that close() waits on, so the database cannot be closed out from under it
either. Prefer flush() if there is any chance the database is in a stall condition.db.flushSync();
db.flushSync({ allowWriteStall: true });On a read-only database it validates its options and then returns without flushing, rather than throwing.
db.syncWal(): Promise<void>Syncs the write-ahead log to stable storage (DB::SyncWAL), so every WAL-backed write committed
before the call survives a power loss afterwards. Unlike flush() it
writes no SST file and never stalls writers. Writes that skipped the WAL (disableWAL) and
transaction-log files, which keep their own durability, are not covered. RocksDB serializes
concurrent syncs of one WAL, so keep one in flight per database rather than queueing them on the
libuv threadpool.
await db.syncWal();On a read-only database the returned promise resolves without syncing.
db.get(key: Key, options?: GetOptions): MaybePromise<any>Retreives the value for a given key. If the key does not exist, it will resolve undefined.
const result = await db.get('foo');
assert.equal(result, 'foo');If the value is in the memtable or block cache, get() will immediately return the value
synchronously instead of returning a promise.
const result = db.get('foo');
const value = result instanceof Promise ? await result : result;
assert.equal(result, 'foo');Note that all errors are returned as rejected promises.
See GetOptions for the available options.
When the expectedVersion option is set and the Verification Table records
a matching version for the key, get() returns the FRESH_VERSION_FLAG sentinel
(constants.FRESH_VERSION_FLAG) instead of reading the value — signalling that any value the
caller has already cached for this key is still fresh and no read was performed. Be sure to check
for this sentinel before treating the result as a value:
import { constants } from '@harperfast/rocksdb-js';
const result = db.get(key, { expectedVersion: cachedEntry.version });
if (result === constants.FRESH_VERSION_FLAG) {
// the cached value is still valid; no read occurred
return cachedEntry.value;
}db.getSync(key: Key, options?: GetOptions): anySynchronous version of get(). Like get(), this can return the FRESH_VERSION_FLAG sentinel when
the expectedVersion option is used.
db.getEstimatedKeyCount(): numberRetrieves the estimated number of keys in the database. This is an alias for
db.getDBIntProperty('rocksdb.estimate-num-keys'); use estimateCount() for range support and a
confidence indicator.
const estimated = db.getEstimatedKeyCount();
console.log(estimated);db.estimateCount(options?: CountEstimateOptions): CountEstimateEstimates the number of keys in the database, or within a key range, returning
{ count, confidence }. Unlike getKeysCount(), this never iterates: the estimate is derived
from RocksDB statistics (memtable stats plus approximate SST sizes converted through the entry
density of the SSTs overlapping the range), so its cost scales with the number of SSTs overlapping
the range rather than the number of keys. Reading cold table properties can do I/O through the
table cache, so narrow ranges are preferable. A start-only range is computed as the
whole-database estimate minus the complement, so it does the work of the range below start.
Accuracy improves with range size. Resolution is bounded by SST data-block granularity, so a range
narrower than a block is unreliable in either direction: it may over-report or report 0 for present
keys, and its low confidence is the signal. Recently deleted or overwritten entries may be counted
until compaction.
Estimates always reflect committed state; writes pending in a transaction are not included. Set
reverse: true to use getRange()'s reverse convention (start is the upper bound and end is
the lower bound). An inverted range (start ≥ end) returns
{ count: 0, confidence: 1 }.
confidence is a heuristic 0–1 indicator of how trustworthy count is — exactly 1 only when the
count is exact. It is derived from the estimate's resolution (data-block/memtable-sampling
granularity relative to the count), the tombstone fraction of the overlapping SSTs, and — for
start-only ranges — the error compounded by complement subtraction. Treat it as an ordering
signal (e.g. when to trust an estimate for query planning vs fall back to a heuristic), not a
statistical bound.
const { count, confidence } = db.estimateCount({ start: 'a', end: 'z' });db.createCountEstimator(options?: CountEstimatorOptions): CountEstimatorCreates an estimator that progressively refines a range count estimate while the range is being
iterated — useful for reporting a total alongside a page of results without scanning the full
range. Before any traversal, estimate() returns the pure statistical estimate (same as
estimateCount(range)). As the caller reports progress with advance(lastKey, count) (e.g. once
per page), estimate() returns the exact traversed count plus a statistical estimate of the
remainder, calibrated by the observed ratio of actual-to-estimated entries over the portion
already traversed — so the count converges toward the exact total. confidence is the
exactness-weighted blend of the traversed portion and the remainder's confidence, so it approaches 1
as the exact portion grows, although a checkpoint may decrease when calibration makes a large
correction.
Each checkpoint reads committed state, so a traversal performed against a transaction snapshot may
be calibrated against data committed after that snapshot.
When traversal completes, call finish() and estimate() returns the exact count with
confidence 1. Reverse ranges follow getRange: set start to the upper bound and end to the
lower bound, then set reverse: true. The caller owns the progress contract: cursors must move
monotonically through the range and each entry must be reported exactly once.
const range = { start: 'a', end: 'z' };
const estimator = db.createCountEstimator(range);
let lastKey;
let pageSize = 0;
for (const { key } of db.getRange({ ...range, limit: 25 })) {
lastKey = key;
pageSize++;
}
estimator.advance(lastKey, pageSize);
const { count, confidence } = estimator.estimate();db.getKeys(options?: IteratorOptions): ExtendedIterableRetrieves all keys within a range.
for (const key of db.getKeys()) {
console.log(key);
}db.getKeysCount(options?: RangeOptions): numberRetrieves the exact number of keys in a database or a range.
const count = db.getKeysCount(); // estimated number of keys
const range = db.getKeysCount({ start: 'a', end: 'z' }); // exact number of keys in the rangedb.getMonotonicTimestamp(): numberReturns the current timestamp as a monotonically increasing timestamp in milliseconds represented as a decimal number. This process-wide clock supplies each transaction's initial timestamp, which is also the key of the transaction-log batch the transaction is written under.
const ts = db.getMonotonicTimestamp();
console.log(ts); // 1764307857213.739It is a wall-clock (Unix epoch) value made strictly increasing: on a tie or a backward step of the
host clock it advances by one floating-point ulp per call until the wall clock catches up. That
keeps transaction timestamps ordered and durable, but it does not measure elapsed time — after a
backward step, differences between two calls understate real time. Use
steadyClockNow() for elapsed durations and deadlines.
The clock is monotonic within a process, not across restarts: a new process reads the wall clock
again, so a backward step between runs can reissue a timestamp that is already a batch key in this
node's transaction log. Opening with
timestampFloorLog raises the clock
above every batch key still durable in the named log before the database handle is returned, so no
transaction can be constructed below it. Keys in segments retention has purged are not durable, and
neither is a bound advertised to peers; carry those across a restart yourself with
raiseTimestampFloor().
Name only a log this process originates. A log written under a timestamp adopted from another
node — a replication receiver calling txn.setTimestamp() — is
keyed by that node's clock; seeding from it would ratchet this process's clock to the fastest of
those nodes at every restart, and there is no way for the database to tell the two kinds of log
apart on its own.
The named floor is fail closed: an unreadable segment, incomplete scan, malformed framing, or a key more than ten years ahead of the wall clock refuses the opted-in open rather than reissuing a durable batch key. The floor is process-wide, so it applies to every database open in the process. The scan runs while the process serializes database opens and closes, so increasing its budget can delay unrelated opens and closes too.
Naming an absent log warns so a fresh local log can be configured before its first write. The first
use may create that fresh log, but if a nonempty directory appears after the warning it is refused:
those keys were not part of the seed, and a transaction timestamp is already chosen before
txn.useLog() could scan it. Close every handle for the path and reopen with timestampFloorLog to
scan the restored log before using it.
db.getOldestSnapshotTimestamp(): numberReturns a number representing a unix timestamp of the oldest unreleased snapshot.
Snapshots are only created during transactions. When the database is opened in optimistic mode (the default), the snapshot will be created on the first read. When the database is opened in pessimistic mode, the snapshot will be created on the first read or write.
console.log(db.getOldestSnapshotTimestamp()); // returns `0`, no snapshots
const promise = db.transaction(async (txn) => {
// perform a write to create a snapshot
await txn.get('foo');
await setTimeout(100);
});
console.log(db.getOldestSnapshotTimestamp()); // returns `1752102248558`
await promise;
// transaction completes, snapshot released
console.log(db.getOldestSnapshotTimestamp()); // returns `0`, no snapshotsdb.getDBProperty(propertyName: string): string | undefinedGets a RocksDB database property as a string.
propertyName: string The name of the property to retrieve (e.g., ) 'rocksdb.levelstats'.Returns undefined if the property is not found.
const db = RocksDatabase.open('/path/to/database');
const levelStats = db.getDBProperty('rocksdb.levelstats');
const stats = db.getDBProperty('rocksdb.stats');db.getDBIntProperty(propertyName: string): number | undefinedGets a RocksDB database property as an integer.
propertyName: string The name of the property to retrieve (e.g., ) 'rocksdb.num-blob-files'.Returns undefined if the property is not found.
const db = RocksDatabase.open('/path/to/database');
const blobFiles = db.getDBIntProperty('rocksdb.num-blob-files');
const numKeys = db.getDBIntProperty('rocksdb.estimate-num-keys');db.isWriteStalled(): booleanWhether RocksDB is currently applying write backpressure to this database —
delaying (rate-limiting) or fully stopping writes. Reads the live
rocksdb.is-write-stopped and rocksdb.actual-delayed-write-rate properties.
This is the authoritative, live pull counterpart to the
'writeStall' event: the event pushes per-column-family
entries into a stall (rate-limited), while this reports the current state on
demand and can never be stale. It is database-wide — the write controller is
shared across every column family, so it answers "are writes stalled anywhere"
rather than for one column family.
if (db.isWriteStalled()) {
console.warn('writes are currently throttled or blocked');
}db.getRange(options?: IteratorOptions): ExtendedIterableRetrieves a range of keys and their values. Supports both synchronous and asynchronous iteration.
// sync
for (const { key, value } of db.getRange()) {
console.log({ key, value });
}
// async
for await (const { key, value } of