Observability

LangSmith tracing auto-wires from env vars. Listen to plugin lifecycle via onPluginStatusChange. Log through ctx.logger for plugin-scoped output.

Turn on LangSmith in one block

LangChain auto-wires tracing when these env vars are present in process.env — the runtime never reads them, only declares them in the base env schema so they show up in qiforge-cli env output.

LANGSMITH_TRACING=true
LANGSMITH_API_KEY=ls-...
LANGSMITH_PROJECT=my-oracle
LANGSMITH_ENDPOINT=https://api.smith.langchain.com   # optional, defaults to LangSmith cloud

Env var	Required for tracing	Purpose
LANGSMITH_TRACING	yes	Set to true to enable tracing. Anything else (or unset) disables it.
LANGSMITH_API_KEY	yes	API key from your LangSmith workspace.
LANGSMITH_PROJECT	yes	Project name traces land under.
LANGSMITH_ENDPOINT	no	Override for self-hosted LangSmith. Defaults to LangSmith cloud.

All four LANGSMITH_* vars are optional() in the base env schema — boot never fails when they’re absent. “Required” above means “required for tracing to work,” not a boot-time requirement.

No code changes needed. Set the vars and every LLM call, tool invocation, and middleware hook produces a span in the LangSmith UI. Unset them to turn it off.

What QiForge gives you out of the box

Three signals — that’s the whole surface. Bring your own log aggregator and metrics.

1. LangSmith traces — env-driven, covered above.
2. Plugin status events — app.onPluginStatusChange(handler).
3. Plugin-scoped logger — ctx.logger (boot) and rtCtx.logger (per-request).

Subscribe to plugin status events

1. Register a handler after createOracleApp returns

   app.onPluginStatusChange((event) => {
     Logger.log(
       `[plugin] ${event.plugin} ${event.from} → ${event.to}` +
         (event.reason ? ` (${event.reason})` : ''),
     );
   });
   

   Live example: apps/qiforge-example/src/main.ts.

2. Listen for the matrix plugin transitions specifically

   The plugin that reliably emits transitions is matrix — it starts pending at boot and flips to either loaded or failed once Matrix init completes in the background.

   app.onPluginStatusChange((event) => {
     if (event.plugin === 'matrix' && event.to === 'loaded') {
       // safe to send messages, mint UCANs, etc.
     }
   });
   

   Use this to gate operations on Matrix being up, or to alert when it fails.

The event shape:

{
  plugin: string;
  from: 'pending' | 'loaded' | 'failed';
  to:   'pending' | 'loaded' | 'failed';
  reason?: string;
}

Capture background errors with onError

app.onError((err, source) => {
  Logger.error(`[runtime] ${source}: ${err.message}`);
});

source is a short label like 'matrix-init'. Use this for background failures (Matrix sync errors, key setup failures). Boot-time errors (env validation, manifest validation, dependency cycles) throw from createOracleApp itself — your try/catch around the call sees them.

Log the boot resolution snapshot

app.plugins.status() returns the loader’s resolution snapshot — log it once at boot so operators see what came up.

const status = app.plugins.status();
Logger.log(`[boot] loaded plugins: ${status.loaded.join(', ') || '(none)'}`);
if (status.excluded.length > 0) {
  Logger.log(
    '[boot] excluded plugins:',
    status.excluded.map((e) => `${e.plugin} (${e.reason})`).join(', '),
  );
}

Shape of the snapshot — the PluginStatusReport defined in packages/oracle-runtime/src/bootstrap/create-oracle-app.ts:

{
  loaded: string[];                                  // plugin names that came up
  excluded: Array<{
    plugin: string;
    reason: string;                                  // e.g. "auto-detect precondition not met (COMPOSIO_API_KEY)"
  }>;
  softDepGaps: Array<{ plugin: string; missing: string }>;
}

The loader tracks an internal cause (feature_false / auto_detect_missing / cascaded) for each excluded plugin, but the public app.plugins.status() report drops it — excluded items are { plugin, reason } only.

Use the plugin-scoped logger

Every PluginContext and RuntimeContext carries a logger field auto-prefixed with the plugin’s name — see Logger in types.ts.

override getTools(ctx: PluginContext): PluginTool[] {
  ctx.logger.log('Building weather tools');
  // ...
}

handler: async (args, rtCtx: RuntimeContext) => {
  rtCtx.logger.log(`Looking up ${args.city}`);
  // ...
}

The Logger interface:

Method	Required	Purpose
log(message, ...optional)	yes	Info-level output.
error(message, ...optional)	yes	Error-level output.
warn(message, ...optional)	yes	Warn-level output.
debug(message, ...optional)	no	Debug-level — implementation-defined.
verbose(message, ...optional)	no	Verbose-level — implementation-defined.
child(bindings)	no	Returns a logger with additional context fields. Falls back to the same logger if not implemented.

Use NestJS’s default Logger format in development (pnpm dev shows it nicely); pipe to your aggregator in production. Override the global logger with createOracleApp({ logger }) if you want a custom format — see createOracleApp reference.

UI-facing tool-call events

For showing the user what the agent did, the runtime emits typed events via rtCtx.emit. The bundled clients (Portal, Slack, Matrix) consume these and render the right UI — you only emit them yourself when writing a custom client.

rtCtx.emit.toolCall({ /* ... */ });
rtCtx.emit.actionCall({ /* ... */ });
rtCtx.emit.renderComponent({ /* ... */ });
rtCtx.emit.reasoning({ /* ... */ });
rtCtx.emit.browserToolCall({ /* ... */ });
rtCtx.emit.router({ /* ... */ });
rtCtx.emit.messageCacheInvalidation({ /* ... */ });

The seven event types are defined on RuntimeContext.emit in types.ts.

Where to read next

Deployment

Logs and probes in production.

Add a middleware

Custom observability via plugin middleware hooks.

Runtime context

Full rtCtx.emit and rtCtx.logger surface.

Environment variables

Every env var the runtime declares.