Skip to main content
OpenClaw exports diagnostics through the official diagnostics-otel plugin using OTLP/HTTP (protobuf). Logs can also be written as stdout JSONL for container and sandbox log pipelines. Any collector or backend that accepts OTLP/HTTP works without code changes. For local file logs, see Logging.
  • Diagnostics events are structured, in-process records emitted by the Gateway and bundled plugins for model runs, message flow, sessions, queues, and exec.
  • diagnostics-otel subscribes to those events and exports them as OpenTelemetry metrics, traces, and logs over OTLP/HTTP, and can mirror log records to stdout JSONL.
  • Provider calls receive a W3C traceparent header from OpenClaw’s trusted model-call span context when the provider transport accepts custom headers. Plugin-emitted trace context is not propagated.
  • Exporters attach only when both the diagnostics surface and the plugin are enabled, so in-process cost stays near zero by default.

​
Quick start

openclaw plugins install clawhub:@openclaw/diagnostics-otel

{
  plugins: {
    allow: ["diagnostics-otel"],
    entries: {
      "diagnostics-otel": { enabled: true },
    },
  },
  diagnostics: {
    enabled: true,
    otel: {
      enabled: true,
      endpoint: "http://otel-collector:4318",
      protocol: "http/protobuf",
      serviceName: "openclaw-gateway",
      traces: true,
      metrics: true,
      logs: true,
      sampleRate: 0.2,
      flushIntervalMs: 60000,
    },
  },
}
Or enable the plugin from the CLI: openclaw plugins enable diagnostics-otel.
protocol supports http/protobuf only. Since traces and metrics default to enabled, any other value (including grpc) aborts the entire diagnostics-otel subscription with an unsupported protocol warning - this also stops stdout log export. Explicitly set traces: false and metrics: false if you only want logsExporter: "stdout" with a non-OTLP protocol value.

​
Signals exported

SignalWhat goes in it
MetricsCounters/histograms for token usage, cost, run duration, failover, skill usage, message flow, Talk events, queue lanes, session state/recovery, tool execution, exec, memory, liveness, and exporter health.
TracesSpans for model usage, model calls, harness lifecycle, skill usage, tool execution, exec, webhook/message processing, context assembly, and tool loops.
LogsStructured logging.file records exported over OTLP or stdout JSONL when diagnostics.otel.logs is enabled; log bodies are withheld unless content capture is explicitly enabled.
Toggle traces, metrics, and logs independently. Traces and metrics default to on when diagnostics.otel.enabled is true; logs default to off and export only when diagnostics.otel.logs is explicitly true. Log export defaults to OTLP; set diagnostics.otel.logsExporter to stdout for JSONL on stdout, or both for both.

​
Configuration reference

{
  diagnostics: {
    enabled: true,
    otel: {
      enabled: true,
      endpoint: "http://otel-collector:4318",
      tracesEndpoint: "http://otel-collector:4318/v1/traces",
      metricsEndpoint: "http://otel-collector:4318/v1/metrics",
      logsEndpoint: "http://otel-collector:4318/v1/logs",
      protocol: "http/protobuf", // grpc disables OTLP export
      serviceName: "openclaw-gateway", // unset falls back to OTEL_SERVICE_NAME, then "openclaw"
      headers: { "x-collector-token": "..." },
      traces: true,
      metrics: true,
      logs: true,
      logsExporter: "otlp", // otlp | stdout | both
      sampleRate: 0.2, // root-span sampler, 0.0..1.0
      flushIntervalMs: 60000, // metric export interval (min 1000ms)
      captureContent: {
        enabled: false,
        inputMessages: false,
        outputMessages: false,
        toolInputs: false,
        toolOutputs: false,
        systemPrompt: false,
        toolDefinitions: false,
      },
    },
  },
}

​
Environment variables

VariablePurpose
OTEL_EXPORTER_OTLP_ENDPOINTFallback for diagnostics.otel.endpoint when the config key is unset.
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT / OTEL_EXPORTER_OTLP_METRICS_ENDPOINT / OTEL_EXPORTER_OTLP_LOGS_ENDPOINTSignal-specific endpoint fallbacks used when the matching diagnostics.otel.*Endpoint config key is unset. Signal-specific config wins over signal-specific env, which wins over the shared endpoint.
OTEL_SERVICE_NAMEFallback for diagnostics.otel.serviceName when the config key is unset. Default service name is openclaw.
OTEL_EXPORTER_OTLP_PROTOCOLFallback for the wire protocol when diagnostics.otel.protocol is unset. Only http/protobuf enables export.
OTEL_SEMCONV_STABILITY_OPT_INSet to gen_ai_latest_experimental to emit the latest GenAI inference span shape: {gen_ai.operation.name} {gen_ai.request.model} span names, CLIENT span kind, and gen_ai.provider.name instead of the legacy gen_ai.system. GenAI metrics always use bounded, low-cardinality attributes regardless.
OPENCLAW_OTEL_PRELOADEDSet to 1 when another preload or host process already registered the global OpenTelemetry SDK. The plugin then skips its own NodeSDK lifecycle but still wires diagnostic listeners and honors traces/metrics/logs.

​
Privacy and content capture

Raw model/tool content is not exported by default. Spans carry bounded identifiers (channel, provider, model, error category, hash-only request ids, tool source, tool owner, skill name/source) and never include prompt text, response text, tool inputs, tool outputs, skill file paths, or session keys. Values that look like scoped agent session keys (for example starting with agent:) are replaced with unknown on low-cardinality attributes. OTLP log records keep severity, logger, code location, trusted trace context, and sanitized attributes by default; the raw log message body is exported only when diagnostics.otel.captureContent is boolean true. Granular captureContent.* subkeys never enable log bodies. Talk metrics export only bounded event metadata (mode, transport, provider, event type) - no transcripts, audio payloads, session ids, turn ids, call ids, room ids, or handoff tokens. Outbound model requests may include a W3C traceparent header generated only from OpenClaw-owned diagnostic trace context for the active model call. Existing caller-supplied traceparent headers are replaced, so plugins or custom provider options cannot spoof cross-service trace ancestry. Set diagnostics.otel.captureContent.* to true only when your collector and retention policy are approved for prompt, response, tool, or system-prompt text. Each subkey is independent:
  • inputMessages - user prompt content.
  • outputMessages - model response content.
  • toolInputs - tool argument payloads.
  • toolOutputs - tool result payloads.
  • systemPrompt - assembled system/developer prompt.
  • toolDefinitions - model tool names, descriptions, and schemas.
When any subkey is enabled, model and tool spans get bounded, redacted openclaw.content.* attributes for that class only.
Boolean captureContent: true enables inputMessages, outputMessages, toolInputs, toolOutputs, toolDefinitions, and OTLP log bodies together, but not systemPrompt - set captureContent.systemPrompt: true explicitly if you also need the assembled system prompt.
toolInputs/toolOutputs content is captured for the built-in agent runtime’s tool executions (openclaw.content.tool_input on completed/error spans, openclaw.content.tool_output on completed spans). External harness tool calls (Codex, Claude CLI) emit tool.execution.* spans without content payloads. Captured content travels on a trusted, listener-only channel and is never placed on the public diagnostic event bus.

​
Sampling and flushing

  • Traces: diagnostics.otel.sampleRate sets a TraceIdRatioBasedSampler on the root span only (0.0 drops all, 1.0 keeps all). Unset uses the OpenTelemetry SDK default (always-on).
  • Metrics: diagnostics.otel.flushIntervalMs (clamped to a minimum of 1000); unset uses the SDK’s periodic-export default.
  • Logs: OTLP logs respect logging.level (file log level) and use the diagnostic log-record redaction path, not console formatting. High-volume installs should prefer OTLP collector sampling/filtering over local sampling. Set diagnostics.otel.logsExporter: "stdout" when your platform already ships stdout/stderr to a log processor and you have no OTLP logs collector. Stdout records are one JSON object per line with ts, signal, service.name, severity, body, redacted attributes, and trusted trace fields when available.
  • File-log correlation: JSONL file logs include top-level traceId, spanId, parentSpanId, and traceFlags when the log call carries a valid diagnostic trace context, letting log processors join local log lines with exported spans.
  • Request correlation: Gateway HTTP requests and WebSocket frames create an internal request trace scope. Logs and diagnostic events inside that scope inherit the request trace by default, while agent run and model-call spans are created as children so provider traceparent headers stay on the same trace.
  • Model-call correlation: openclaw.model.call spans include safe prompt component sizes by default and per-call token attributes when the provider result exposes usage. openclaw.model.usage remains the run-level accounting span for aggregate cost, context, and channel dashboards, and stays on the same diagnostic trace when the emitting runtime has trusted trace context.

​
Exported metrics

​
Model usage

  • openclaw.tokens (counter, attrs: openclaw.token, openclaw.channel, openclaw.provider, openclaw.model, openclaw.agent)
  • openclaw.cost.usd (counter, attrs: openclaw.channel, openclaw.provider, openclaw.model)
  • openclaw.run.duration_ms (histogram, attrs: openclaw.channel, openclaw.provider, openclaw.model)
  • openclaw.context.tokens (histogram, attrs: openclaw.context, openclaw.channel, openclaw.provider, openclaw.model)
  • gen_ai.client.token.usage (histogram, GenAI semantic-conventions metric, attrs: gen_ai.token.type = input/output, gen_ai.provider.name, gen_ai.operation.name, gen_ai.request.model)
  • gen_ai.client.operation.duration (histogram, seconds, GenAI semantic-conventions metric, attrs: gen_ai.provider.name, gen_ai.operation.name, gen_ai.request.model, optional error.type)
  • openclaw.model_call.duration_ms (histogram, attrs: openclaw.provider, openclaw.model, openclaw.api, openclaw.transport, plus openclaw.errorCategory and openclaw.failureKind on classified errors)
  • openclaw.model_call.request_bytes (histogram, UTF-8 byte size of the final model request payload; no raw payload content)
  • openclaw.model_call.response_bytes (histogram, UTF-8 byte size of streamed response chunk payloads; high-frequency text, thinking, and tool-call deltas count only incremental delta bytes; no raw response content)
  • openclaw.model_call.time_to_first_byte_ms (histogram, elapsed time before the first streamed response event)
  • openclaw.model.failover (counter, attrs: openclaw.provider, openclaw.model, openclaw.failover.to_provider, openclaw.failover.to_model, openclaw.failover.reason, openclaw.failover.suspended, openclaw.lane)
  • openclaw.skill.used (counter, attrs: openclaw.skill.name, openclaw.skill.source, openclaw.skill.activation, optional openclaw.agent, optional openclaw.toolName)

​
Message flow

  • openclaw.webhook.received (counter, attrs: openclaw.channel, openclaw.webhook)
  • openclaw.webhook.error (counter, attrs: openclaw.channel, openclaw.webhook)
  • openclaw.webhook.duration_ms (histogram, attrs: openclaw.channel, openclaw.webhook)
  • openclaw.message.queued (counter, attrs: openclaw.channel, openclaw.source)
  • openclaw.message.received (counter, attrs: openclaw.channel, openclaw.source)
  • openclaw.message.dispatch.started (counter, attrs: openclaw.channel, openclaw.source)
  • openclaw.message.dispatch.completed (counter, attrs: openclaw.channel, openclaw.outcome, openclaw.reason, openclaw.source)
  • openclaw.message.dispatch.duration_ms (histogram, attrs: openclaw.channel, openclaw.outcome, openclaw.reason, openclaw.source)
  • openclaw.message.processed (counter, attrs: openclaw.channel, openclaw.outcome)
  • openclaw.message.duration_ms (histogram, attrs: openclaw.channel, openclaw.outcome)
  • openclaw.message.delivery.started (counter, attrs: openclaw.channel, openclaw.delivery.kind)
  • openclaw.message.delivery.duration_ms (histogram, attrs: openclaw.channel, openclaw.delivery.kind, openclaw.outcome, openclaw.errorCategory)

​
Talk

  • openclaw.talk.event (counter, attrs: openclaw.talk.event_type, openclaw.talk.mode, openclaw.talk.transport, openclaw.talk.brain, openclaw.talk.provider)
  • openclaw.talk.event.duration_ms (histogram, attrs: same as openclaw.talk.event; emitted when a Talk event reports duration)
  • openclaw.talk.audio.bytes (histogram, attrs: same as openclaw.talk.event; emitted for Talk audio frame events that report byte length)

​
Queues and sessions

  • openclaw.queue.lane.enqueue (counter, attrs: openclaw.lane)
  • openclaw.queue.lane.dequeue (counter, attrs: openclaw.lane)
  • openclaw.queue.depth (histogram, attrs: openclaw.lane or openclaw.channel=heartbeat)
  • openclaw.queue.wait_ms (histogram, attrs: openclaw.lane)
  • openclaw.session.state (counter, attrs: openclaw.state, openclaw.reason)
  • openclaw.session.stuck (counter, attrs: openclaw.state; emitted for recoverable stale session bookkeeping)
  • openclaw.session.stuck_age_ms (histogram, attrs: openclaw.state; emitted for recoverable stale session bookkeeping)
  • openclaw.session.turn.created (counter, attrs: openclaw.agent, openclaw.channel, openclaw.trigger)
  • openclaw.session.recovery.requested (counter, attrs: openclaw.state, openclaw.action, openclaw.active_work_kind, openclaw.reason)
  • openclaw.session.recovery.completed (counter, attrs: openclaw.state, openclaw.action, openclaw.status, openclaw.active_work_kind, openclaw.reason)
  • openclaw.session.recovery.age_ms (histogram, attrs: same as the matching recovery counter)
  • openclaw.run.attempt (counter, attrs: openclaw.attempt)

​
Session liveness telemetry

diagnostics.stuckSessionWarnMs is the no-progress age threshold for session liveness diagnostics. A processing session does not age toward this threshold while OpenClaw observes reply, tool, status, block, or ACP runtime progress. Typing keepalives do not count as progress, so a silent model or harness can still be detected. OpenClaw classifies sessions by the work it can still observe:
  • session.long_running: active embedded work, model calls, or tool calls are still making progress. Owned model calls that stay silent past diagnostics.stuckSessionWarnMs also report as long-running before diagnostics.stuckSessionAbortMs, so slow or non-streaming model providers do not look like stalled gateway sessions while abort-observable.
  • session.stalled: active work exists, but the active run has not reported recent progress. Owned model calls switch from session.long_running to session.stalled at or after diagnostics.stuckSessionAbortMs; ownerless stale model/tool activity is not treated as harmless long-running work. Stalled embedded runs stay observe-only at first, then abort-drain after diagnostics.stuckSessionAbortMs with no progress so queued turns behind the lane can resume. When unset, the abort threshold defaults to the safer extended window of at least 5 minutes and 3x diagnostics.stuckSessionWarnMs.
  • session.stuck: stale session bookkeeping with no active work, or an idle queued session with stale ownerless model/tool activity. This releases the affected session lane immediately after recovery gates pass.
Recovery emits structured session.recovery.requested and session.recovery.completed events. Diagnostic session state is marked idle only after a mutating recovery outcome (aborted or released) and only if the same processing generation is still current. Only session.stuck emits the openclaw.session.stuck counter, the openclaw.session.stuck_age_ms histogram, and the openclaw.session.stuck span. Repeated session.stuck diagnostics back off while the session remains unchanged, so dashboards should alert on sustained increases rather than every heartbeat tick. For the config knob and defaults, see Configuration reference. Liveness warnings also emit:
  • openclaw.liveness.warning (counter, attrs: openclaw.liveness.reason)
  • openclaw.liveness.event_loop_delay_p99_ms (histogram, attrs: openclaw.liveness.reason)
  • openclaw.liveness.event_loop_delay_max_ms (histogram, attrs: openclaw.liveness.reason)
  • openclaw.liveness.event_loop_utilization (histogram, attrs: openclaw.liveness.reason)
  • openclaw.liveness.cpu_core_ratio (histogram, attrs: openclaw.liveness.reason)

​
Harness lifecycle

  • openclaw.harness.duration_ms (histogram, attrs: openclaw.harness.id, openclaw.harness.plugin, openclaw.outcome, openclaw.harness.phase on errors)

​
Tool execution and loop detection

  • openclaw.tool.execution.duration_ms (histogram, attrs: gen_ai.tool.name, openclaw.toolName, openclaw.tool.source, openclaw.tool.owner, openclaw.tool.params.kind, plus openclaw.errorCategory on errors)
  • openclaw.tool.execution.blocked (counter, attrs: gen_ai.tool.name, openclaw.toolName, openclaw.tool.source, openclaw.tool.owner, openclaw.tool.params.kind, openclaw.deniedReason)
  • openclaw.tool.loop (counter, attrs: openclaw.toolName, openclaw.loop.level, openclaw.loop.action, openclaw.loop.detector, openclaw.loop.count, optional openclaw.loop.paired_tool; emitted when a repetitive tool-call loop is detected)

​
Exec

  • openclaw.exec.duration_ms (histogram, attrs: openclaw.exec.target, openclaw.exec.mode, openclaw.outcome, openclaw.failureKind)

​
Diagnostics internals (memory, payloads, exporter health)

  • openclaw.payload.large (counter, attrs: openclaw.payload.surface, openclaw.payload.action, openclaw.channel, openclaw.plugin, openclaw.reason)
  • openclaw.payload.large_bytes (histogram, attrs: same as openclaw.payload.large)
  • openclaw.memory.rss_bytes / openclaw.memory.heap_used_bytes / openclaw.memory.heap_total_bytes / openclaw.memory.external_bytes / openclaw.memory.array_buffers_bytes (histograms, no attrs; process memory samples)
  • openclaw.memory.pressure (counter, attrs: openclaw.memory.level, openclaw.memory.reason)
  • openclaw.diagnostic.async_queue.dropped (counter, attrs: openclaw.diagnostic.async_queue.drop_class; internal diagnostic-queue backpressure drops)
  • openclaw.telemetry.exporter.events (counter, attrs: openclaw.exporter, openclaw.signal, openclaw.status, optional openclaw.reason, optional openclaw.errorCategory; exporter lifecycle/failure self-telemetry)

​
Exported spans

  • openclaw.model.usage
    • openclaw.channel, openclaw.provider, openclaw.model
    • openclaw.tokens.* (input/output/cache_read/cache_write/total)
    • gen_ai.system by default, or gen_ai.provider.name when the latest GenAI semantic conventions are opted in
    • gen_ai.request.model, gen_ai.operation.name, gen_ai.usage.*
  • openclaw.run
    • openclaw.outcome, openclaw.channel, openclaw.provider, openclaw.model, openclaw.errorCategory
  • openclaw.model.call
    • gen_ai.system by default, or gen_ai.provider.name when the latest GenAI semantic conventions are opted in
    • gen_ai.request.model, gen_ai.operation.name, openclaw.provider, openclaw.model, openclaw.api, openclaw.transport
    • openclaw.errorCategory, error.type, and optional openclaw.failureKind on errors
    • openclaw.model_call.request_bytes, openclaw.model_call.response_bytes, openclaw.model_call.time_to_first_byte_ms
    • openclaw.model_call.prompt.input_messages_count, openclaw.model_call.prompt.input_messages_chars, openclaw.model_call.prompt.system_prompt_chars, openclaw.model_call.prompt.tool_definitions_count, openclaw.model_call.prompt.tool_definitions_chars, openclaw.model_call.prompt.total_chars (safe component sizes only, no prompt text)
    • openclaw.model_call.usage.* and gen_ai.usage.* when the model-call result carries provider usage for that individual call
    • Span event openclaw.provider.request with attribute openclaw.upstreamRequestIdHash (bounded, hash-based) when the upstream provider result exposes a request id; raw ids are never exported
    • With OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental, model-call spans use the latest GenAI inference span name {gen_ai.operation.name} {gen_ai.request.model} and CLIENT span kind instead of openclaw.model.call.
  • openclaw.harness.run
    • openclaw.harness.id, openclaw.harness.plugin, openclaw.outcome, openclaw.provider, openclaw.model, openclaw.channel
    • On completion: openclaw.harness.result_classification, openclaw.harness.yield_detected, openclaw.harness.items.started, openclaw.harness.items.completed, openclaw.harness.items.active
    • On error: openclaw.harness.phase, openclaw.errorCategory, optional openclaw.harness.cleanup_failed
  • openclaw.tool.execution
    • gen_ai.tool.name, openclaw.toolName, openclaw.tool.source, optional openclaw.tool.owner, openclaw.tool.params.*
    • Optional openclaw.errorCategory/openclaw.errorCode on errors, openclaw.deniedReason and openclaw.outcome=blocked when denied by policy or sandbox
  • openclaw.exec
    • openclaw.exec.target, openclaw.exec.mode, openclaw.outcome, openclaw.failureKind, openclaw.exec.command_length, openclaw.exec.exit_code, openclaw.exec.exit_signal, openclaw.exec.timed_out
  • openclaw.webhook.processed
    • openclaw.channel, openclaw.webhook
  • openclaw.webhook.error
    • openclaw.channel, openclaw.webhook, openclaw.error
  • openclaw.message.processed
    • openclaw.channel, openclaw.outcome, openclaw.reason
  • openclaw.message.delivery
    • openclaw.channel, openclaw.delivery.kind, openclaw.outcome, openclaw.errorCategory, openclaw.delivery.result_count
  • openclaw.session.stuck
    • openclaw.state, openclaw.ageMs, openclaw.queueDepth
  • openclaw.context.assembled
    • openclaw.prompt.size, openclaw.history.size, openclaw.context.tokens, openclaw.errorCategory (no prompt, history, response, or session-key content)
  • openclaw.tool.loop
    • openclaw.toolName, openclaw.loop.level, openclaw.loop.action, openclaw.loop.detector, openclaw.loop.count, optional openclaw.loop.paired_tool (no loop messages, params, or tool output)
  • openclaw.memory.pressure
    • openclaw.memory.level, openclaw.memory.reason, openclaw.memory.rss_bytes, openclaw.memory.heap_used_bytes, openclaw.memory.heap_total_bytes, openclaw.memory.external_bytes, openclaw.memory.array_buffers_bytes, optional openclaw.memory.threshold_bytes/openclaw.memory.rss_growth_bytes/openclaw.memory.window_ms
When content capture is explicitly enabled, model and tool spans can also include bounded, redacted openclaw.content.* attributes for the specific content classes you opted into.

​
Diagnostic event catalog

The events below back the metrics and spans above. Plugins can also subscribe to them directly without OTLP export. Model usage
  • model.usage - tokens, cost, duration, context, provider/model/channel, session ids. usage is provider/turn accounting for cost and telemetry; context.used is the current prompt/context snapshot and can be lower than provider usage.total when cached input or tool-loop calls are involved.
Message flow
  • webhook.received / webhook.processed / webhook.error
  • message.queued / message.processed
  • message.delivery.started / message.delivery.completed / message.delivery.error
Queue and session
  • queue.lane.enqueue / queue.lane.dequeue
  • session.state / session.long_running / session.stalled / session.stuck
  • run.attempt / run.progress
  • diagnostic.heartbeat (aggregate counters: webhooks/queue/session)
Harness lifecycle
  • harness.run.started / harness.run.completed / harness.run.error - per-run lifecycle for the agent harness. Includes harnessId, optional pluginId, provider/model/channel, and run id. Completion adds durationMs, outcome, optional resultClassification, yieldDetected, and itemLifecycle counts. Errors add phase (prepare/start/send/resolve/cleanup), errorCategory, and optional cleanupFailed.
Exec
  • exec.process.completed - terminal outcome, duration, target, mode, exit code, and failure kind. Command text and working directories are not included.
  • exec.approval.followup_suppressed - stale approval follow-up dropped after a session rebound. Includes approvalId, reason (session_rebound), phase (direct_delivery or gateway_preflight), and the dispatcher timestamp. Session keys, routes, and command text are not included.

​
Without an exporter

Keep diagnostics events available to plugins or custom sinks without running diagnostics-otel: 
{
  diagnostics: { enabled: true },
}
For targeted debug output without raising logging.level, use diagnostics flags. Flags are case-insensitive and support wildcards (telegram.* or *): 
{
  diagnostics: { flags: ["telegram.http"] },
}
Or as a one-off env override: 
OPENCLAW_DIAGNOSTICS=telegram.http,telegram.payload openclaw gateway
Flag output goes to the standard log file (logging.file) and is still redacted by logging.redactSensitive. Full guide: Diagnostics flags.

​
Disable

{
  diagnostics: { otel: { enabled: false } },
}
Or leave diagnostics-otel out of plugins.allow, or run openclaw plugins disable diagnostics-otel.