This library is meant to replace all global JavaScript functions with isomorphic module imports. Additionally, it implements several performance-oriented utility modules. Most noteworthy are the binary encoding/decoding modules [lib0/encoding] / [lib0/decoding], the randomized testing framework [lib0/testing], the fast Pseudo Random Number Generator [lib0/PRNG], the small socket.io alternative [lib0/websocket], and the logging module [lib0/logging] that allows colorized logging in all environments. Lib0 has only one dependency, which is also from the author of lib0. If lib0 is transpiled with rollup or webpack, very little code is produced because of the way that it is written. All exports are pure and are removed by transpilers that support dead code elimination. Here is an example of how dead code elemination and mangling optimizes code from lib0:
// How the code is optimized by transpilers:// lib0/json.jsexport const stringify = JSON.stringifyexport const parse = JSON.parse// index.jsimport * as json from 'lib0/json'export const f = (arg1, arg2) => json.stringify(arg1) + json.stringify(arg2)// compiled with rollup and uglifyjs:const s=JSON.stringify,f=(a,b)=>s(a)+s(b)export {f}
Performance resources
Each function in this library is tested thoroughly and is not deoptimized by v8 (except some logging and comparison functions that can't be implemented without deoptimizations). This library implements its own test suite that is designed for randomized testing and inspecting performance issues.
The code style might be a bit different from what you are used to. Stay open. Most of the design choices have been thought through. The purpose of this code style is to create code that is optimized by the compiler and that results in small code bundles when used with common module bundlers. Keep that in mind when reading the library.
No polymorphism!
Modules should only export pure functions and constants. This way the module bundler can eliminate dead code. The statement const x = someCondition ? A : B cannot be eleminated, because it is tied to a condition.
Use Classes for structuring data. Classes are well supported by jsdoc and are immediately optimized by the compiler. I.e. prefer class Coord { constructor (x, y) { this.x = x; this.y = y} } instead of { x: x, y: y }, because the compiler needs to be assured that the order of properties does not change. { y: y, x: x } has a different hidden class than { x: x, y: y }, which will lead to code deoptimizations if their use is alternated.
The user of your module should never create data objects with the new keyword. Prefer exporting factory functions like const createCoordinate = (x, y) => new Coord(x, y).
The use of class methods is discouraged, because method names can't be mangled or removed by dead code elimination.
The only acceptable use of methods is when two objects implement functionality differently.
E.g. class Duck { eat () { swallow() } } and class Cow { eat () { chew() } } have the
same signature, but implement it differently.
Prefer const variable declarations. Use let only in loops. const always leads to easier code.
Keep the potential execution stack small and compartmentalized. Nobody wants to debug spaghetti code.
Give proper names to your functions and ask yourself if you would know what the function does if you saw it in the execution stack.
Avoid recursion. There is a stack limit in most browsers and not every recursive function is optimized by the compiler.
lib0 contains isomorphic modules that work in nodejs, the browser, and other environments. It exports modules as the commonjs and the new esm module format.
If possible,
ESM module
import module from 'lib0/[module]' // automatically resolves to lib0/[module].js
CommonJS
require('lib0/[module]') // automatically resolves to lib0/dist/[module].cjs
Manual
Automatically resolving to commonjs and esm modules is implemented using conditional exports which is available in node>=v12. If support for older versions is required, then it is recommended to define the location of the module manually:
import module from 'lib0/[module].js'// require('lib0/dist/[module].cjs')
Modules
[lib0/array] Utility module to work with Arrays.
import * as array from 'lib0/array'
array.last(arr: ArrayLike<L>): L
Return the last element of an array. The element must exist
[lib0/broadcastchannel] Helpers for cross-tab communication using broadcastchannel with LocalStorage fallback.
import * as broadcastchannel from 'lib0/broadcastchannel'
// In browser window A:broadcastchannel.subscribe('my events', data => console.log(data))broadcastchannel.publish('my events', 'Hello world!') // => A: 'Hello world!' fires synchronously in same tab// In browser window B:broadcastchannel.publish('my events', 'hello from tab B') // => A: 'hello from tab B'
Copy the content of an Uint8Array view to a new ArrayBuffer.
buffer.encodeAny(data: any): Uint8Array
Encode anything as a UInt8Array. It's a pun on typescripts's any type.
See encoding.writeAny for more information.
buffer.decodeAny(buf: Uint8Array): any
Decode an any-encoded value.
[lib0/cache] An implementation of a map which has keys that expire.
import * as cache from 'lib0/cache'
new cache.Cache(timeout: number) cache.removeStale(cache: module:cache.Cache<K, V>): number cache.set(cache: module:cache.Cache<K, V>, key: K, value: V) cache.get(cache: module:cache.Cache<K, V>, key: K): V | undefined cache.refreshTimeout(cache: module:cache.Cache<K, V>, key: K) cache.getAsync(cache: module:cache.Cache<K, V>, key: K): V | Promise<V> | undefined
Works well in conjunktion with setIfUndefined which has an async init function.
Using getAsync & setIfUndefined ensures that the init function is only called once.
[lib0/decoding] Efficient schema-less binary decoding with support for variable length encoding.
import * as decoding from 'lib0/decoding'
Use [lib0/decoding] with [lib0/encoding]. Every encoding function has a corresponding decoding function.
Encodes numbers in little-endian order (least to most significant byte order)
and is compatible with Golang's binary encoding (https://golang.org/pkg/encoding/binary/)
which is also used in Protocol Buffers.
[lib0/encoding] Efficient schema-less binary encoding with support for variable length encoding.
import * as encoding from 'lib0/encoding'
Use [lib0/encoding] with [lib0/decoding]. Every encoding function has a corresponding decoding function.
Encodes numbers in little-endian order (least to most significant byte order)
and is compatible with Golang's binary encoding (https://golang.org/pkg/encoding/binary/)
which is also used in Protocol Buffers.
Calculate the metric prefix for a number. Assumes E.g. prefix(1000) = { n: 1, prefix: 'k' }
[lib0/mutex] Mutual exclude for JavaScript.
import * as mutex from 'lib0/mutex'
mutex.createMutex(): mutex
Creates a mutual exclude function with the following property:
const mutex = createMutex()mutex(() => { // This function is immediately executed mutex(() => { // This function is not executed, as the mutex is already active. })})
[lib0/number]
import * as number from 'lib0/number'
number.MAX_SAFE_INTEGER number.MIN_SAFE_INTEGER number.LOWEST_INT32 number.HIGHEST_INT32: number number.isInteger number.isNaN number.parseInt
[lib0/object] Utility functions for working with EcmaScript objects.
observable.Observable#on(name: N, f: function) observable.Observable#once(name: N, f: function) observable.Observable#off(name: N, f: function) observable.Observable#emit(name: N, args: Array<any>)
Emit a named event. All registered event listeners that listen to the
specified name will receive the event.
observable.Observable#destroy() websocket.WebsocketClient#on(name: N, f: function) websocket.WebsocketClient#once(name: N, f: function) websocket.WebsocketClient#off(name: N, f: function) websocket.WebsocketClient#emit(name: N, args: Array<any>)
Emit a named event. All registered event listeners that listen to the
specified name will receive the event.
Given a seed a PRNG generates a sequence of numbers that cannot be reasonably predicted.
Two PRNGs must generate the same random sequence of numbers if given the same seed.
Create a Xoroshiro128plus Pseudo-Random-Number-Generator.
This is the fastest full-period generator passing BigCrush without systematic failures.
But there are more PRNGs available in ./PRNG/.
prng.bool(gen: module:prng~PRNG): Boolean
Generates a single random bool.
prng.int53(gen: module:prng~PRNG, min: Number, max: Number): Number
Generates a random integer with 53 bit resolution.
prng.uint53(gen: module:prng~PRNG, min: Number, max: Number): Number
Generates a random integer with 53 bit resolution.
prng.int32(gen: module:prng~PRNG, min: Number, max: Number): Number
Generates a random integer with 32 bit resolution.
prng.uint32(gen: module:prng~PRNG, min: Number, max: Number): Number
Generates a random integer with 53 bit resolution.
prng.int31(gen: module:prng~PRNG, min: Number, max: Number): Number prng.real53(gen: module:prng~PRNG): Number
Generates a random real on [0, 1) with 53 bit resolution.
prng.char(gen: module:prng~PRNG): string
Generates a random character from char code 32 - 126. I.e. Characters, Numbers, special characters, and Space:
Checks if an object is a promise using ducktyping.
Promises are often polyfilled, so it makes sense to add some additional guarantees if the user of this
library has some insane environment where global Promise objects are overwritten.
[lib0/random] Isomorphic module for true random numbers / buffers / uuids.
import * as random from 'lib0/random'
Attention: falls back to Math.random if the browser does not support crypto.
random.rand random.uint32 random.uint53 random.oneOf(arr: Array<T>): T random.uuidv4
[lib0/set] Utility module to work with sets.
import * as set from 'lib0/set'
set.create set.toArray(set: Set<T>): Array<T> set.first(set: Set<T>): T set.from(entries: Iterable<T>): Set<T>
[lib0/sort] Efficient sort implementations.
import * as sort from 'lib0/sort'
Note: These sort implementations were created to compare different sorting algorithms in JavaScript.
Don't use them if you don't know what you are doing. Native Array.sort is almost always a better choice.
This algorithm beats Array.prototype.sort in Chrome only with arrays with 10 million entries.
In most cases [].sort will do just fine. Make sure to performance test your use-case before you
integrate this algorithm.
Note that Chrome's sort is now a stable algorithm (Timsort). Quicksort is not stable.
[lib0/statistics] Utility helpers for generating statistics.
import * as statistics from 'lib0/statistics'
statistics.median(arr: Array<number>): number statistics.average(arr: Array<number>): number
[lib0/storage] Isomorphic variable storage.
import * as storage from 'lib0/storage'
Uses LocalStorage in the browser and falls back to in-memory storage.
storage.varStorage
This is basically localStorage in browser, or a polyfill in nodejs
[lib0/symbol] Utility module to work with EcmaScript Symbols.
import * as symbol from 'lib0/symbol'
symbol.create(): Symbol
Return fresh symbol.
symbol.isSymbol(s: any): boolean
[lib0/testing] Testing framework with support for generating tests.
import * as testing from 'lib0/testing'
// test.js template for creating a test executableimport { runTests } from 'lib0/testing'import * as log from 'lib0/logging'import * as mod1 from './mod1.test.js'import * as mod2 from './mod2.test.js'import { isBrowser, isNode } from 'lib0/environment.js'if (isBrowser) { // optional: if this is ran in the browser, attach a virtual console to the dom log.createVConsole(document.body)}runTests({ mod1, mod2,}).then(success => { if (isNode) { process.exit(success ? 0 : 1) }})
// mod1.test.js/** * runTests automatically tests all exported functions that start with "test". * The name of the function should be in camelCase and is used for the logging output. * * @param {t.TestCase} tc *\/export const testMyFirstTest = tc => { t.compare({ a: 4 }, { a: 4 }, 'objects are equal')
[lib0/time] Utility module to work with time.
import * as time from 'lib0/time'
time.getDate(): Date
Return current time.
time.getUnixTime(): number
Return current unix time.
time.humanizeDuration(d: number): string
Transform time (in ms) to a human readable format. E.g. 1100 => 1.1s. 60s => 1min. .001 => 10μs.
Create an Uint8Array view of the next len bytes and advance the position by len.
Important: The Uint8Array still points to the underlying ArrayBuffer. Make sure to discard the result as soon as possible to prevent any memory leaks.
Use buffer.copyUint8Array to copy the result into a new Uint8Array.
Important: The Uint8Array still points to the underlying ArrayBuffer. Make sure to discard the result as soon as possible to prevent any memory leaks.
Use buffer.copyUint8Array to copy the result into a new Uint8Array.
Create an DataView of the next len bytes. Use it to write data after
calling this function.
// write float32 using DataViewconst dv = writeOnDataView(encoder, 4)dv.setFloat32(0, 1.1)// read float32 using DataViewconst dv = readFromDataView(encoder, 4)dv.getFloat32(0) // => 1.100000023841858 (leaving it to the reader to find out why this is the correct result)
Differences to JSON:
• Transforms data to a binary format (not to a string)
• Encodes undefined, NaN, and ArrayBuffer (these can't be represented in JSON)
• Numbers are efficiently encoded either as a variable length integer, as a
32 bit float, as a 64 bit float, or as a 64 bit bigint.
Encoding table:
Data Type
Prefix
Encoding Method
Comment
undefined
127
Functions, symbol, and everything that cannot be identified is encoded as undefined
null
126
integer
125
writeVarInt
Only encodes 32 bit signed integers
float32
124
writeFloat32
float64
123
writeFloat64
bigint
122
writeBigInt64
boolean (false)
121
True and false are different data types so we save the following byte
boolean (true)
120
- 0b01111000 so the last bit determines whether true or false
string
119
writeVarString
object<string,any>
118
custom
Writes {length} then {length} key-value pairs
array
117
custom
Writes {length} then {length} json values
Uint8Array
116
writeVarUint8Array
We use Uint8Array for any kind of binary data
Reasons for the decreasing prefix:
We need the first bit for extendability (later we may want to encode the
prefix with writeVarUint). The remaining 7 bits are divided as follows:
[0-30] the beginning of the data range is used for custom purposes
(defined by the function that uses this library)
[31-127] the end of the data range is used for data encoding by
lib0/encoding.js
new encoding.RleEncoder(writer: function(module:encoding.Encoder, T):void)
Now come a few stateful encoder that have their own classes.
encoding.RleEncoder#s: T|null
Current state
encoding.RleEncoder#write(v: T)
new encoding.IntDiffEncoder(start: number)
Basic diff decoder using variable length encoding.
Encodes the values [3, 1100, 1101, 1050, 0] to [3, 1097, 1, -51, -1050] using writeVarInt.
encoding.IntDiffEncoder#s: number
Current state
encoding.IntDiffEncoder#write(v: number)
new encoding.RleIntDiffEncoder(start: number)
A combination of IntDiffEncoder and RleEncoder.
Basically first writes the IntDiffEncoder and then counts duplicate diffs using RleEncoding.
Encodes the values [1,1,1,2,3,4,5,6] as [1,1,0,2,1,5] (RLE([1,0,0,1,1,1,1,1]) ⇒ RleIntDiff[1,1,0,2,1,5])
encoding.RleIntDiffEncoder#s: number
Current state
encoding.RleIntDiffEncoder#write(v: number)
new encoding.UintOptRleEncoder()
Optimized Rle encoder that does not suffer from the mentioned problem of the basic Rle encoder.
Internally uses VarInt encoder to write unsigned integers. If the input occurs multiple times, we write
write it as a negative number. The UintOptRleDecoder then understands that it needs to read a count.
Encodes [1,2,3,3,3] as [1,2,-3,3] (once 1, once 2, three times 3)
encoding.UintOptRleEncoder#s: number
encoding.UintOptRleEncoder#write(v: number)
encoding.UintOptRleEncoder#toUint8Array()
new encoding.IncUintOptRleEncoder()
Increasing Uint Optimized RLE Encoder
The RLE encoder counts the number of same occurences of the same value.
The IncUintOptRle encoder counts if the value increases.
I.e. 7, 8, 9, 10 will be encoded as [-7, 4]. 1, 3, 5 will be encoded
as [1, 3, 5].
encoding.IncUintOptRleEncoder#s: number
encoding.IncUintOptRleEncoder#write(v: number)
encoding.IncUintOptRleEncoder#toUint8Array()
new encoding.IntDiffOptRleEncoder()
A combination of the IntDiffEncoder and the UintOptRleEncoder.
The count approach is similar to the UintDiffOptRleEncoder, but instead of using the negative bitflag, it encodes
in the LSB whether a count is to be read. Therefore this Encoder only supports 31 bit integers!
Internally uses variable length encoding. Contrary to normal UintVar encoding, the first byte contains:
1 bit that denotes whether the next value is a count (LSB)
1 bit that denotes whether this value is negative (MSB - 1)
1 bit that denotes whether to continue reading the variable length integer (MSB)
Therefore, only five bits remain to encode diff ranges.
Use this Encoder only when appropriate. In most cases, this is probably a bad idea.
encoding.IntDiffOptRleEncoder#s: number
encoding.IntDiffOptRleEncoder#write(v: number)
encoding.IntDiffOptRleEncoder#toUint8Array()
new encoding.StringEncoder()
Optimized String Encoder.
Encoding many small strings in a simple Encoder is not very efficient. The function call to decode a string takes some time and creates references that must be eventually deleted.
In practice, when decoding several million small strings, the GC will kick in more and more often to collect orphaned string objects (or maybe there is another reason?).
This string encoder solves the above problem. All strings are concatenated and written as a single string using a single encoding call.
The lengths are encoded using a UintOptRleEncoder.
export const testMyFirstTest = tc => { t.group('subtest 1', () => { t.describe('this message is part of a collapsible section') }) await t.groupAsync('subtest async 2', async () => { await someaction() t.describe('this message is part of a collapsible section') })}