Skip to main content

​
Local ONNX decision models

The installable ONNX plugin runs classifiers locally through the decision model API. It uses a separate, persistent Node process with ONNX Runtime’s CPU backend. Inference does not send state or questions to a remote service. Models download only when you explicitly run the download command. See Decision models for the model role, rubric examples, and provider-neutral plugin API.

​
Setup

The decision-provider API was added after released OpenClaw 2026.9.5. Packaged ONNX installs require a host and plugin API of at least 2026.9.6; the installer rejects 2026.9.5 before loading the plugin. The native runtime belongs to the plugin package and is not bundled into core.

​
Current development checkout

Until a supporting release is available, use an OpenClaw source checkout that contains both the decision-provider API and extensions/onnx. Build that checkout with pnpm install --frozen-lockfile and pnpm build, then enable the plugin and select its model using the configuration below. Run commands from the checkout:
The checkout’s co-versioned source plugins use the host’s development API. This does not make a packaged plugin compatible with a released 2026.9.5 host.

​
Packaged installation

On a compatible host, install the locally built package and prepare a model:

​
Configuration

Select the role globally or for one agent:
The default artifact directory is <stateDir>/models/onnx. Set the plugin’s modelDir to use another directory; the download, verify, and probe commands also accept --model-dir <path>. Artifacts are grouped by model ID. Downloads use fixed repository revisions, sizes, and SHA256 hashes. Existing mismatched files are refused rather than overwritten. openclaw onnx verify <model> checks an installation. Verification and cached-download checks stream the files, so checking a large graph does not require a graph-sized memory buffer. Inference still loads verified graph bytes into its worker.

​
Models

The hosted presets use FP32 graphs. Model licenses and conversion sources are linked from the original GLiClass, GLiNER2, and DeBERTa model pages. Downloaded weights remain separate from OpenClaw’s package. For Instruct models, use the installed plugin’s dist/scripts/export-gliclass-instruct.py helper with the pinned Python packages listed by --help. Supply a local copy of the exact official checkpoint revision printed by openclaw onnx models:
The helper verifies the source files, runs without remote model code or downloads, and creates a fresh directory containing the graph, tokenizer, and model.json export manifest. Local exports are operator-supplied artifacts: the runtime checks their declared source revision and file hashes. Their graph hash is not a published third-party attestation. Few-shot example sections are not supported by this export.

​
Question semantics

These models classify text against a rubric. JSON state and rubric entries are serialized as text; instructions and criterion descriptions condition classification. Use descriptive criteria rather than opaque IDs when possible.
  • Choice: softmax over the model’s complete label logits; highest-probability label wins.
  • Score: classify the ordered criterion levels, then return their expected zero-based index.
  • Boolean: supply both criteria.true and criteria.false as meaningful predicate descriptions. Bare Boolean questions are unsupported because labels such as true and false do not reliably express a zero-shot predicate.
Probabilities are model estimates, not calibrated guarantees. These classifiers are not interchangeable with Jev on every reasoning task. Validate the rubric on representative examples before relying on its decision quality. The plugin supports up to 32 questions and 64 labels per question, with a one-MiB limit on compiled inputs across the batch. Each encoded input, including its rubric, must fit 512 tokens. It rejects unsupported input instead of truncating it. GLiClass reserves its label, separator, and example markers. GLiNER2.5 also rejects the schema’s reserved markers and parentheses in rubrics; ordinary state text can contain punctuation.

​
Lifecycle and runtime

Up to maxLoadedModels selected models warm during plugin service startup. Later requests reuse native sessions; the resident cache evicts the least recently used model when full. Missing files and failed artifact-integrity checks leave warm sessions available. Eviction happens after artifact verification and tokenizer preparation, before loading the replacement native session. threads sets CPU intra-operation parallelism from 1 to 8. Cold-loading a large model can exhaust a request’s deadline on slower machines. The host allows up to 30 seconds; consumers can request less. When agents use several models, set maxLoadedModels to hold the active models if memory permits, or select a smaller model. The default cache holds two models; the maximum is five. The host still enforces its 30-second decision deadline and four-call provider limit. The worker serializes native operations. A queued cancellation leaves the warm process intact; cancelling active native work terminates and joins that process before releasing the request. The next live request starts a fresh worker. Plugin retirement stops the worker and its owned work. The runtime requires supported Node.js and the optional onnxruntime-node native package. CPU inference is the initial backend; GPU and WASM execution are not enabled by this plugin. Native libraries and model tokenizers load inside the worker, keeping them out of Gateway discovery and ordinary plugin registration.