createExecutionSession accepts a frozen microtest command, a lifecycle module
and factory export, and an environment choice. profile-default reuses the
in-process collected plan or starts a supervised child; fresh-invocation
starts a fresh host for in-process profiles. environmentLifetime reports
reused or fresh before execution.
The lifecycle runs before configuration revalidation and collection, chooses a
selection from the complete collected plan, and observes each case through
awaited cleanup. Invocations accept JSON data, a deadline budget, and an explicit
abort signal or null. Replies preserve native results, lifecycle data, typed
failures, and deadline status. Failure and timeout replies also retain artifacts
already available to the execution owner. Sessions omit ordinary reporters and disk
persistence. dispose() closes admission and awaits active work or owned child
termination.
Config describes authored policy. normalizeConfig produces a
command-independent NormalizedConfig without reading files; loadConfig
returns that policy plus configPath as LoadedConfig. Programmatic commands
accept normalized policy. The old run-prefixed config names are removed.
Configuration loading and authoring are also exposed through
@overkill-dev/run/config for packages that need the config-owned surface
without importing orchestration.
freezePolicy resolves Node microtest discovery and policy without importing
tests or custom engines. It returns an immutable, serializable
FrozenMicrotestPolicy containing the selected profile, loader declarations,
discovered descriptors, baseline request, portable engine, and source identity.
Engine instances and other profile families are unsupported.
Pass moduleBoundary: null for ordinary freezing. A FileModuleBoundary
contains an absolute projectRoot matching discovery and explicit absolute
sharedRoots for installed dependencies and runner bootstrap. Shared roots
must not contain the project root. Frozen runner operations enforce this
additional module constraint in both process models, before config revalidation
and through execution cleanup. It does not grant filesystem permissions.
Older serialized policies without this field normalize it to null.
withFileModuleBoundary(boundary, operation) applies the same constraint to
configuration loading before freezing. Native synchronous module hooks reject
file imports whose real paths escape the allowed roots, including symlink
escapes. Hooks are scoped to the asynchronous operation and removed on success
or failure. A null boundary installs no hook. Native module caching remains
unchanged.
initialize accepts an ordinary or frozen-policy command and prepares discovery
without importing config again, custom engines, or tests. The selected profile
must use in-process execution. Caller bootstrap can run before
await initialized.collect(). Collection validates file-backed frozen policy,
imports modules under ordinary runtime policy, and caches either the collected
handle or the failure. Collection failures may expose RunCollectionError or
an AggregateError containing the original runtime-policy errors.
An initial selection matching no cases rejects collection with
no-tests-collected, as resolve does.
The collected handle exposes immutable facts and a body-free plan catalog.
await collected.run({ selection: { kind: 'all' } }) executes that plan without
reimporting modules. Invocation selection may only narrow the original eligible
cases; seed, shard, order, and policy remain fixed. Empty selection and empty
shards retain ordinary runner semantics. runWithReporterDelivery returns the
usual result and reporter-delivery tracking.
Each invocation creates fresh scopes, assertions, cleanup, runtime observers,
reporter instances, and result collection. Observers close before the handle
becomes idle, including on failures. Overlapping invocations of the same handle
reject with invalid-request. Module state remains shared between invocations.
Explicit ordinary coverage creates a fresh session and run record each time;
previous collection imports are not repeated for coverage. No disposal call is
required for an idle handle.
Native policy observations follow run and attempt ownership. Runner transport,
lifecycle callbacks, and reporter work do not acquire test-policy attribution.
Unknown observations remain run errors. Late test diagnostics retain their
original identity as attribution drift instead of blaming a later attempt.
Use source: { kind: 'memory' } for supplied config, or
{ kind: 'config-file', request: { cwd, configPath }, resolvedPath } for loaded
config. resolvedPath is the loaded config's configPath, including null for
the no-config fallback. File-backed policy is revalidated in the execution host
before collection using native module caching. Conflicting effective policy
fails before test imports. Discovery and loader declarations are compared even
when they resolve to the same files. Reporters and unrelated valid profiles are
excluded from the comparison.
resolve, run, and runWithReporterDelivery also accept
{ kind: 'frozen-policy', policy, invocation: { seed, selection }, reporting }.
The invocation selection narrows the frozen baseline selection. Supply
reporting: { reporters, outputRenderer } explicitly; callbacks and the seed
are excluded from the snapshot. Ordinary runtime guards and process usage policies
still apply. parseFrozenMicrotestPolicy validates transported data, including
descriptor roots and file-set ownership, and freezes the result.
The selection, result, and runner-error parsers validate transported values
against the runner's existing schemas. createRandomRunSeed() generates a
nonnegative bigint for integrations that resolve a seed before crossing an
execution boundary.
run and runWithReporterDelivery accept { timing: null, signal } through
RunInvocationOptions. A non-null signal requires a supervised-process test
profile; other process models reject it before discovery or user imports.
Existing timing-only calls remain valid, and signal: null disables cancellation.
Aborting kills the owned test child and awaits shutdown and output draining.
The result retains completed cases and earlier diagnostics, records interrupted
active cases as crashed, and includes an unattributed crash runner error.
A pre-aborted invocation imports no tests and starts no child. Abort after child
shutdown has no effect. Hard termination cannot guarantee child-side scope
cleanup. This programmatic interruption is separate from graceful CLI signal
handling. Signals are not part of frozen policy, run facts, or IPC messages.
runIfMain(import.meta, testNode, options?) is the supported companion for
bare node path/to/file.test.ts execution. It returns immediately when the
module was imported, loads config from process.cwd() only for the main
module, selects the matching profile from profiles.<name>.files, falls back
to the configured microtest profile, and uses the default line reporter when
neither options, the selected profile, nor project config provides reporters.
Direct execution always runs in the current process. When the selected profile
uses supervised-process, it prints a warning and preserves the profile's
scheduling and run facts while disabling child-process isolation for that
direct Node invocation.
Command-line business logic is exposed through @overkill-dev/run/command-line:
The binary recognizes overkill bench run [paths...] and
overkill bench list [paths...]. Run accepts CommandLineBenchmarkRunRequest
with { cwd, configPath, runRequest }; list accepts
CommandLineBenchmarkListRequest with { cwd, configPath, listRequest }.
Nested requests share the ordinary selection and execution fields, with
profile: string | null. An omitted profile selects the sole benchmark profile.
Unknown, missing, ambiguous, or ordinary-family selections return argument error
3. Ordinary run/list and direct-file execution reject the benchmark family.
Benchmark profiles use testFamily: 'benchmark' in the same registry and support
execution, reporters, attachments, processUsage, timings,
timeouts, and baselines. orchestrator.bench.list(command, { timing: null }) returns a
ResolvedRun; bench.run and bench.runWithReporterDelivery return the regular
result and reporter-delivery shapes. These methods require an explicit profile
in RunCommand and explicit invocation timing options. Ordinary methods reject
benchmark profiles before discovery. CLI benchmark commands infer the sole
benchmark profile when --profile is omitted.
Benchmark execution supports worker pools and supervised processes. It defaults
to reused workers, file work units, serial scheduling, maxConcurrency: 1, and
hedging off. Placement permits parallel lanes for eligible artifact-size
measurements when host calibration allows it; sensitive or unknown work stays
serial. Worker requests remain upper bounds. Concurrent groups and hedging are rejected. Timeout
defaults are collection 5000 ms, soft 40000 ms, and hard 60000 ms; hard overrides
cannot exceed 60000 ms. Resources, runtime descriptors, artifacts, assertion
outcomes, and reporter selection follow the regular runner flow. Listing imports
definitions without executing bodies or acquiring resources. See the
benchmark guide for measurement and budget authoring.
orchestrator.bench.baseline provides update, apply, bootstrap, diff,
and list. Execution methods accept a RunCommand and explicit timing options;
results include baseline changes and the write outcome. list accepts config,
cwd, and { paths, profile }, and reads persisted files without importing
benchmarks. See benchmark baseline commands for write and
comparison behavior, and adapter contracts for profile
configuration.
@overkill-dev/run/benchmark-calibration exposes calibrateBenchmarkHost and
normalizeBenchmarkDuration, also available from
@overkill-dev/bench/calibration. Baseline runs collect real reference samples
in a separate process; adapters choose whether to normalize their observations.
Programmatic microtest requests accept execution: { mode: 'serial' } to run
cases serially through cleanup while preserving the selected profile's process
model and other policy. mode: 'profile-default' retains configured scheduling.
Serial overrides are rejected for other test families before discovery. The
resolved request records the override; RunFacts.execution.scheduling records
effective scheduling while the configured profile stays unchanged. Coverage
remains a separate request field. There is no CLI scheduling override flag.
Programmatic selection helpers are exposed through @overkill-dev/run/filters:
all(filters)
any(filters)
not(filter)
caseId(id)
equals(field, value)
contains(field, value)
glob(field, pattern)
file(pattern)
owner(value)
params(value)
workload(name)
parseRunFilterExpression(expression)
runtime(name)
runtimeDimension(name, dimensionName, value)
runtimeScenario(runtime, scenario, value)
runtimeVariant(name, variantId)
suite(value)
tag(value)
title(value)
Resource wrapper lifecycle coordination is exposed through
@overkill-dev/run/resource-lifecycle for packages that compose
runner-managed resource scopes with test-body authoring wrappers.
Scoped transcript storage is exposed through
@overkill-dev/run/transcript-store for runner and test integrations. Test
authors should use the higher-level transcript API from @overkill-dev/test.
The current runner accepts explicit file paths through RunRequest.paths and
profile file discovery through profiles.<name>.files. Profile files may use
top-level include and exclude, or mutually exclusive named sets whose
entries use the same include/exclude shape. Named sets must be non-empty and
non-overlapping. Explicit files bypass top-level discovery, but when the
selected profile uses sets, each explicit file must match exactly one set.
Each discovered or explicit file is imported as a native Node ESM module and
must export a named testNode value created by the selected engine.
Attach broad module annotations to that exported top-level node.
commandLineRunner.listTests(...) resolves those modules and prints a plain
plan tree without executing tests or loading fallback reporters.
RunCommand.engine may be { kind: 'default' } to use the shared public engine,
{ kind: 'instance', engine } for in-process programmatic callers that also
create their test nodes with that engine, or { kind: 'module', moduleUrl, exportName, exportKind } for supervised programmatic callers that need the
child process to load the engine without parent-side user-module execution.
Programmatic selection filters are supported through RunRequest.selection.
The current helpers select by stable case id, file, title, suite, table params,
workload name, tag, ownership, public runtime key, runtime matrix variant id, runtime
dimension, and exact runner-visible runtime scenario bindings. Runtime identity
filters use case-sensitive equality. Scenario filtering is programmatic and has
no CLI grammar. Test family matching is intentionally absent because one run is
already bound to one profile test family.
parseRunFilterExpression(expression) parses the CLI filter grammar into the
same RunFilter tree. Runs use seeded ordering by default. Pass
RunRequest.order: 'lexical' for deterministic source-stable order, or
RunRequest.order: 'plan' for programmatic callers that need an already
materialized order. RunRequest.shard uses one-based { index, total }
values and partitions the filtered work-unit set by stable identity hash.
General recording, replay, and --last-failed are separate runner milestones. Direct
prebuilt TestPlan execution belongs to @overkill-dev/engine through
execute(testPlan).
The command methods other than runTests and listTests are fixed first-party
entrypoints and currently return argument errors until their command
implementations land.
Coverage is explicit per run. RunRequest.coverage defaults to false in
first-party callers, and the @overkill-dev/test binary maps --coverage to
true. Coverage requests are valid only for microtest profiles and are
recorded in RunFacts.execution.coverage, with the resolved policy in
RunFacts.coveragePolicy. Microtest profiles may configure
coverage outputs, loaded or all-files sources, line/function/branch
thresholds, and a config-file-relative outputDir. The default outputs are
V8 and LCOV. An empty outputs array keeps raw coverage without rendering
reports.
Type-only TypeScript files, including modules containing documentation comments
or empty export markers, do not contribute to coverage totals. Executable
imports and runtime declarations remain covered.
Source patterns apply to original files when scripts declare source maps.
Generated script paths do not need to match those patterns. Broken declared
maps or references fail coverage; scripts without declarations use JavaScript
coverage. Known test sources are excluded automatically. Mixed test bundles
use coverage.sources.exclude to exclude their test originals.
All-files reporting emits 0% when selected runtime sources were never loaded.
Native TypeScript coverage uses Node's type stripping and preserves source
offsets. Never-loaded TypeScript files need no generated source maps.
A scope containing no executable sources fails, including scopes matching only
types or no files. Native data is retained on reporting failures. Replay and
automatic coverage cleanup await the replay and record-retention workflows.
Integration runs with resource or runtime wrappers also persist a RunRecord.
The record starts after collection and before resource acquisition. Attachment
files are copied under <runtimeStateDir>/runs/<id>/artifacts/; artifact paths
are relative to the project root. Completed and interrupted records retain
attachment metadata and available content. Reporter completion events include
attachments from their case attempt.
Integration profiles accept attachments with these default limits:
Setting
Default
maxInlineBytes
1 MiB
maxArtifactBytes
10 MiB
maxScopeBytes
10 MiB
maxScopeAttachments
100
Scope budgets cover the full retry chain. Retry artifact policy also determines
which attachment files remain. Cancelled or matching hedged peers are discarded;
conflicting executions retain their attachments in the conflict evidence.
Microtest profiles reject attachment configuration.
Coverage runs persist a RunRecord at <runtimeStateDir>/runs/<id>.json.
The same ULID names the default <runtimeStateDir>/runs/<id>/coverage
directory. Configured output directories keep raw data under raw/<id>.
Records preserve the request, decimal seed, resolved facts, execution policy,
project-relative coverage paths, and result artifacts. The started record
precedes coverage setup and imports; early failures may have facts: null.
Completion includes reporter and cleanup errors. Required persistence failures
fail the run. Bare integration runs, microtest runs without coverage, and
orchestrator.resolve() do not create records.
Resolved runtime identities contain selected scenario bindings. They flow into
RunFacts, duration-history keys, reporter labels, and artifact IDs so replay
metadata and captured evidence identify the selected scenario.
Process usage measurement is explicit. Project config can enable it under
profiles.<name>.processUsage.measure; RunRequest.measureProcessUsage
can override that policy for one run. processUsage.budgets are thresholds
and require measurement, while RunRequest.processBudgetOverrides changes
individual thresholds for one run. CLI commands use the selected profile's
process usage policy.
Runner timing collection is explicit beyond the always-present summary.
Project config can set profiles.<name>.timings.collection to summary or
precise; omitted timing policy defaults to summary. RunRequest uses
timingCollection: 'profile-default' by default and can upgrade one run with
timingCollection: 'precise'. The @overkill-dev/test binary parses
--timings into that precise request. Strategies that consume timing-derived
duration facts also require precise collection.
Output capture is run-level intent by default. RunRequest.capture is
buffered by default; live passes capture-capable stdout and stderr through
without creating log-capture artifacts. Non-microtest test controls may
override capture per test. Microtest profiles reject live capture and per-test
capture controls. There is no project config field for capture mode.
Runner profile names are project-owned. Names such as microtest,
backend-http, ui-browser, ui.browser, and unit_fast select profile
entries exactly. Behavior comes from the selected profile config, not from the
name. Profile names must be non-empty and contain only letters, numbers, dots,
underscores, and hyphens. The name benchmark is valid. Profiles declare
testFamily: 'microtest', 'integration', or 'benchmark'; ordinary commands
accept microtest and integration families. The selected
profile's test family is recorded in RunFacts.execution.testFamily.
Configured microtest profiles may set files.include and files.exclude.
include is required when files is present; exclude defaults to [].
With no run paths, the selected profile's files policy discovers test modules.
Explicit file paths bypass the files policy. Directory paths require a files
policy, filter the profile-discovered file set, and cannot be mixed with file
paths.
Configured integration profiles require a files policy. Integration tests
are not governed by RunRequest.capabilityRestrictions; that request field is
part of the microtest runtime model only.
Integration profiles can opt into retries: { maxAttempts: 3 }. The limit
includes the initial attempt; omission or maxAttempts: 1 runs once.
retries.artifacts accepts first-failure-and-final (default),
last-failure-and-final, or all. Microtest profiles reject this setting.
Only completed assertion failures, ordinary body errors, and cooperative soft
timeouts can retry. Cleanup, contract, process-exhaustion, runtime-policy, and crash
failures are terminal. Each attempt gets fresh per-case resources and timeout
state; shared resources and the selected work identity stay unchanged.
Results retain every attempt's outcome, verdict, and duration. retried is
null for one attempt and otherwise records the actual attempt count and final
verdict. Counts and duration history record one logical case, with durations
summed across attempts. Case artifacts carry a zero-based AttemptId;
run-scoped artifacts carry null. Captured output remains capped at 1 MiB per
logical work item across all attempts.
Microtest profile execution is modeled with two independent fields:
execution.processModel is in-process or supervised-process, and
execution.scheduling is concurrent or serial.
execution.maxConcurrency is a positive safe integer or 'unlimited' and
limits active cases per executor. It defaults to 5; serial scheduling still
admits one case at a time. The default microtest profile uses supervised
concurrent execution. The selected values are recorded in RunFacts.execution
and drive runner planning.
Integration profiles default to worker-pool with concurrent scheduling.
worker-pool uses bounded Node worker threads, collects once in a worker,
then resolves selected cases into WorkUnits. workDistribution controls
whether those units are packed by file, by individual case, or by named
profile file-set groups. Group distribution requires files.sets. Each group
names one or more file sets and can override granularity, in-unit scheduling,
local order, and worker lifecycle. Unmatched selected file sets are rejected
by default, or run as plain file units with profile defaults when
unmatched: 'file' is configured. RunFacts.execution contains the frozen
PlacementPlan: work units, resolved per-unit policy, local worker lanes, and
the deterministic initial lane assignment used by execution. The plan also
freezes resource ownership for every selected per-run, per-file, and
per-suite boundary. Shared resources either stay on one reusable executor
lane or use a dedicated infrastructure worker that projects handles to the
assigned work. Local-only handles are rejected when the resolved placement
requires projection. Worker-pool
placement defaults to assignmentPolicy: 'case-count-balanced', which places
larger selected work units first and balances lane load by selected case count
and resource capacity weight.
Use assignmentPolicy: 'stable' to preserve source-order round-robin
placement.
Worker-pool size defaults to one fewer than host parallelism, with a minimum of
one and a maximum of eight. The planner caps that target by host parallelism,
execution.maxWorkers, and the work-unit count. RunRequest.workers replaces
the automatic target before those caps are applied. A serial or single-worker
key shared by every selected unit caps the resolved executor count at one.
Resolved worker-count facts are recorded in RunFacts.execution.workerCount;
resource-owner workers are not included.
dispatchPolicy: 'dynamic-lease' is the default worker-pool dispatch policy.
It leases pending work to compatible idle lanes and may split eligible
not-started units along WorkId boundaries. Split children keep derived trace
identities linked to the frozen parent WorkUnitId; the parent remains the
identity used for sharding and initial placement. Serial scheduling, group
granularity, fresh-worker units, serial keys, and single-worker keys make a unit
indivisible. dispatchPolicy: 'static-assignment' follows the frozen lane
assignment without splitting.
Execution records a complete attempt-centric placement trace for later run
record persistence. Each attempt includes its exact WorkId subset, lane,
opaque executor identity, completion duration, and explicit recovery outcome.
Runner boundary and worker phase details remain in run timings.
Worker-pool defaults to . duplicates only explicitly
safe single-case stragglers under ; rejects
hedging. Slower duplicates are cancelled, matching duplicate outcomes are
discarded, and conflicting outcomes fail the case with a
artifact.
reuses worker threads between units.
creates disposable isolation per
unit. remains available when a single process-isolated
child boundary is preferred.
Resource requirements resolve after work-unit formation. A serial or
exclusive-resource requirement makes its containing file or group unit
serial without serializing unrelated units. single-worker pins matching work
to one lane without changing in-unit scheduling. For supervised-process, a
serial or exclusive requirement anywhere in the selected plan makes the
containing process plan serial.
Planning rejects incompatible runtime or resource definitions, invalid resource
dependency scopes, mixed worker lifecycles for one hard constraint, and worker
capacity that cannot satisfy the resolved lifecycle lanes. These failures are
reported as RunExecutionPlanError with deterministic structured conflicts
before test execution begins.
Direct NormalizedConfig values may set execution.hostProcess for worker-pool
profiles. { kind: 'direct' } keeps the worker-thread pool in the coordinator
process. { kind: 'child', nodeArguments: [...] } starts one supervised host
process around the worker-thread pool, applies validated Node/V8 arguments to
that host only, and records the derived host reasons in RunFacts.execution.
Loaded project config files reject hostProcess for now.
RunRequest.capabilityRestrictions.mode controls the current microtest
restriction policy. The programmatic default is enabled; the command-line
runner also sets enabled explicitly. supervised-process microtests are
enforced in child processes started with Node's permission model. The parent
process remains unrestricted and owns reporters, output files, scheduling, and
supervision. Supervised children receive bootstrap read permission for the
project cwd and runner runtime files, receive no reporter write permission, and
drop fs.read before test bodies run.
in-process means no child process is spawned. Capability restrictions in this
mode are best-effort diagnostics only: Overkill observes native diagnostics,
async_hooks resources, and final global-state snapshots where possible, but it
cannot add --permission after the caller process has started. Runtime methods are never monkey-patched. Native events observe user IPC
listener registration, and native diagnostics observe process.execve() attempts.
Registration is not prevented. Premature in-process exit receives a synchronous
diagnostic and changes successful exit status to failure; abort reporting and
arbitrary process.kill() detection are not guaranteed. Supervised parents
validate shared IPC payloads, terminate children on unexpected IPC, and require
valid completion even after exit code zero. Receipt does not identify the sending
test. Terminal child shutdown and output draining are bounded to one second. Final attempt capture remains open during draining without reviving execution. The CLI bin skeleton starts with --permission-audit, so CLI
in-process microtests can observe extra permission-model diagnostics.
Programmatic in-process callers get those audit diagnostics only if their own
Node process was started with --permission-audit.
Capability results are classified as blocked, observed, or native-gap. Blocked
effects are denied by native permissions.
Observed effects are reported as runtime-policy runner errors and fail the
owning case, all active cases when concurrent attribution is ambiguous, or the
out-of-test boundary when no case is active. Native gaps are documented runtime
limitations; current examples include sync bootstrap reads inside the cwd grant,
Date, Math.random(), sync crypto randomness, arbitrary process signaling,
in-process outgoing IPC without an owned parent, and SQLite execution.
Live instance engines are supported for in-process runs. They are rejected
for supervised-process and worker-pool runs because an object with
executable functions cannot cross the execution boundary. Custom engines for
those process models must use a module selection whose file URL is inside the
run cwd and whose export is either an engine value or a synchronous getter
returning an engine.
Config loading is common runner infrastructure, not plugin discovery.
The command-line runner loads native Node config files, selects the default
line reporter when the selected profile and project config both omit reporters,
defaults managed reporter output to the plain renderer, returns fallback
diagnostics for the binary wrapper to write, and maps run outcomes to stable
exit codes. Fallback diagnostics contain runner errors that were not delivered
to any terminal-capable reporter callback during orchestrator.runWithReporterDelivery(command).
Raw terminal reporters are counted as delivered when their callback succeeds
because their writes are intentionally opaque to the dispatcher. Process
exhaustion maps to exit code 5 before generic runner errors because
process-exhaustion is also a runner error subtype. Profile reporters replace
top-level project reporters for the selected run; otherwise top-level reporters
are the fallback.
Installing a package does not add commands, and there is no installed-package
scan, dynamic command registry, or command plugin lifecycle.
Project config may set outputRenderer to adapt managed line intents, for
example to render GitHub Actions annotations from a brief stdout reporter.
Config files must export config as a named export. Default exports are
configuration errors.
Executing benchmark commands collect one host snapshot before collection and
placement. RunFacts.benchmarkHost records observations, relative admission
limits, and their results; other families and listing operations use null.
RunFacts.benchmarkPlacement records the adaptive worker ceiling and its reasons;
it is null for other families. Listing records an uncalibrated one-worker ceiling.
Collector-free artifact-size measurements can use parallel lanes; sensitive or
unknown measurements remain serial. Requested worker and profile caps remain
visible and may be restricted further by calibration. Configured unavailable
signals block execution. See Host conditions for limits, formulas, probe availability, and
snapshot timing. Baseline normalization remains in benchmarkCalibration.
Built-in timing and CPU metrics normalize automatically. Custom domains select
representative workloads with calibratedMeasurement. Calibration is cached within
an execution context; ordinary runs require no captured files. Normalized budgets
and snapshots use a shared reference-workload scale across hardware.
overkill bench calibrate optionally captures display references for the selected
benchmark profile. orchestrator.bench.calibrate(command, { timing: null }) exposes
the same operation. Capture skips benchmark bodies and unrelated fixtures. Reference
files live under the baseline directory's calibration subdirectory. Version 4
benchmark-run artifacts and RunFacts.benchmarkNormalization retain reference
inputs. See the benchmark guide
for scaling, uncertainty, saved expectations, and stale-reference maintenance.
runCpuCalibrationWorkload() from @overkill-dev/run/benchmark-calibration runs
the fixed integer-mix reference workload and returns its checksum. The built-in
CPU calibration resource measures this same workload.
readBenchmarkMachine() reads host identity and metadata without running a
reference workload. metricReadingSchema validates metric readings shared by
collectors and persisted samples. Both are exported by
@overkill-dev/run/benchmark-calibration.
benchmarkTimelinePolicySchema, benchmarkTimelineEvidenceSchema, and
benchmarkTimelineReportSchema validate timeline policy, retention metadata,
and saved event payloads across benchmark and runner packages.
serializeRunDiagnostics projects artifacts, case results, and runner errors
into bounded JSON-safe evidence without writing a run record.
parseSerializedRunDiagnostics validates that representation without serializing
its values again.
createNodeFileStore provides atomic runtime state writes and create-only
publication, with temporary-file cleanup on failure.