package
Local prompt classification CLI — route prompts to the right model tier using on-device ONNX models
npm install knowmaticGitHub-flavored package documentation.
Across all versions, last 30 days.
937 in 30 days
Runtime metadata.
https://github.com/user-attachments/assets/7f721cf5-b935-4a93-8c13-73a980b47bd0
Fast prompt classification that routes prompts to the appropriate Claude model tier (Ex: Haiku, Sonnet, or Opus) using on-device ONNX models. Works as both a CLI with an interactive TUI and a library you can import into any Node.js project. Optimize API costs by intelligently selecting the right model based on prompt complexity and reasoning effort.
Try the models in your browser at knowmatic-lab.xyz.
npm install knowmaticimport { classifyDifficulty } from "knowmatic";
const result = await classifyDifficulty("Explain quantum entanglement");
console.log(result.top); // { label: "Hard", score: 0.95 }| Function | Signature | Returns | Description |
|---|---|---|---|
classifyDifficulty | (text: string, opts?: ClassifyOptions) => Promise<ClassifyResult> | ClassifyResult | Routes to Haiku / Sonnet / Opus |
classifyReasoningEffort | (text: string, opts?: ClassifyOptions) => Promise<ClassifyResult> | ClassifyResult | Estimates low / medium / high effort |
classifyCode | (text: string, opts?: ClassifyOptions) => Promise<ClassifyResult> | ClassifyResult | Detects programming language |
autocomplete | (text: string, opts?: AutocompleteOptions) => Promise<AutocompleteResult> | AutocompleteResult | Generates a full completion |
autocompleteStream | (text: string, opts?: AutocompleteOptions) => AsyncGenerator<string> | AsyncGenerator<string> | Streams tokens one at a time |
ClassifyOptions
| Field | Type | Default | Description |
|---|---|---|---|
modelsDir | string | bundled models | Custom path to ONNX model directory |
AutocompleteOptions
| Field | Type | Default | Description |
|---|---|---|---|
maxNewTokens | number | engine default | Maximum tokens to generate |
temperature | number | engine default | Sampling temperature |
topK | number | engine default | Top-K filtering |
minConfidence | number | engine default | Minimum token confidence to continue |
repetitionPenalty | number | engine default | Repetition penalty factor |
signal | AbortSignal | -- | Abort signal to cancel generation |
modelsDir | string | bundled models | Custom path to ONNX model directory |
ClassifyResult
{
predictions: { label: string; score: number }[];
top: { label: string; score: number };
latencyMs: number;
}AutocompleteResult
{
text: string;
tokens: number[];
}import { autocompleteStream } from "knowmatic";
for await (const token of autocompleteStream("How do I")) {
process.stdout.write(token);
}For low-level access, the package also re-exports:
InferenceEngine, Tokenizer, AutocompleteEngine -- core engine classescontainsCode, extractCode -- code detection utilitiesapplyRepetitionPenalty, temperatureScale, topKFilter, softmaxMax, sampleFromLogits -- sampling primitivesMODELS_DIR -- resolved path to the bundled model directory# Build
npm run build
# Run
npm start
# Development (runs from source)
npm run dev| Key | Action |
|---|---|
| Type | Enter your prompt |
| Tab | Accept autocomplete suggestion |
| Enter | Classify the prompt |
| Alt+Enter | Insert newline |
| Escape | Clear suggestion |
| Ctrl+U | Clear input |
| Ctrl+C | Quit |
knowmatic loads four quantized ONNX models at startup:
| Model | Size | Purpose |
|---|---|---|
| Difficulty classifier | 14M | Routes to Haiku / Sonnet / Opus |
| Reasoning effort classifier | 14M | Estimates low / medium / high effort |
| Code classifier | 14M | Detects programming language |
| SFT autocomplete | 28M | Generates word-level completions |
All inference runs on CPU via onnxruntime-node. A BPE tokenizer handles text encoding with a 512-token context window.
src/
index.ts # CLI entry point and TUI state machine
lib.ts # Public library API
paths.ts # Model directory resolution
inference.ts # ONNX model inference engine
tokenizer.ts # BPE tokenizer implementation
autocomplete.ts # Generative autocomplete engine
sampling.ts # Token sampling strategies
codeDetection.ts # Code block detection
ui.ts # Terminal rendering
models/
difficulty_classifier/
reasoning_effort/
code_classifier/
sft/
tokenizer/See TERMS_OF_USE.md for full terms. Models are provided under a limited license and may not be redistributed. See the terms document for commercial use provisions.
All models, weights, code, and associated materials are the exclusive property of knowmatic hobby lab.
Estimated unpacked size across 10 packages.
367.3 MB
Includes dependencies.