openclaw/docs/plugins/typesafe.md
Mike 09b55d361a
feat(typesafe): judge V2 image state with hosted Jev (#177636)
Translate one existing V2 PNG/JPEG image into hosted Jev referenced-asset requests. Keep the prepared endpoint/header/auth owner, guarded transport, lifecycle checks, route-bound billing, and text-only local/custom behavior. Do not restore retired transport or core migration code.

Fix the published-catalog consumer test to distinguish explicit image bounds from absent alias token ceilings and pricing.

Validation: 136 focused tests, the real TypeSafe SDK V1/V2 prepared integration, 13 prompt-prefix tests, catalog regression red/green, targeted types/lint/format, and independent review. Authenticated real Jev text and generated-image calls passed. Full-agent live coverage remains a documented limitation; current-head CI/security/review enforcement is required before merge.

Thanks @mikeypaepke-gif for the original feature and design.

Co-authored-by: Mike <262474202+mikeypaepke-gif@users.noreply.github.com>
Co-authored-by: steipete <58493+steipete@users.noreply.github.com>
2026-10-27 07:51:53 +08:00

20 KiB
Raw Permalink Blame History

summary title read_when
Use hosted Jev or a local System One server for typed decisions TypeSafe AI
Configuring a typed decision model
Using TypeSafe with the decision evaluation tool
Running Kev locally through the System One API

TypeSafe AI

The official external typesafe plugin connects OpenClaw's optional decision model role to TypeSafe AI's hosted Jev models or an explicitly configured System One server such as Kev. Its models appear in the separate Decision picker, never in the conversational model picker.

This adapter requires the prepared Decision V2 API. Released OpenClaw 2026.10.4 exposes V1 only. Packaged use requires a supporting paired host/plugin release; the older installation floor alone does not establish V2 compatibility. The normal release process aligns plugin API and build versions with the host.

See Decision models for the model role, available backends, rubric examples, and provider-neutral plugin API.

The plugin is disabled by default. Installing or enabling it does not select a decision model or schedule background work.

Install

TypeSafe AI is packaged separately from core for publication to npm and ClawHub. Its first publication is pending a supporting release. Once published, install it from npm on a compatible host:

openclaw plugins install @openclaw/typesafe

To select ClawHub explicitly:

openclaw plugins install @openclaw/typesafe

Until a supporting release is available, use a source checkout containing the prepared V2 decision-provider API and extensions/typesafe. Build it with pnpm install --frozen-lockfile and pnpm build, then apply the configuration below. Source-checkout plugins use the host's co-versioned development API; that does not make the packaged plugin compatible with a V1-only host.

Enable and configure

For hosted Jev, create a protected credential in Settings → Secrets, then reference it from the plugin configuration. Merge this example into your existing configuration; keep any other entries in plugins.allow.

{
  plugins: {
    allow: ["typesafe"],
    entries: {
      typesafe: {
        enabled: true,
        config: {
          apiKey: { source: "store", provider: "default", id: "TYPESAFE_API_KEY" },
        },
      },
    },
  },
  agents: {
    ownership: "explicit",
    defaults: { decisionModel: "typesafe/jev-latest" },
    entries: {
      research: { decisionModel: "typesafe/jev-1.13.0" },
    },
  },
}

typesafe/jev-latest appears as Jev; the pinned typesafe/jev-1.13.0 appears as Jev 1.13.0. An unset agent override inherits agents.defaults.decisionModel; an empty override disables decisions for that agent. An unset or empty global role leaves decisions off by default.

Hosted mode reads the host's prepared SecretRef value for each request. It does not independently read environment credentials or cache a previous credential. A missing or unavailable credential makes decisions unavailable. Use the normal secret refresh flow after changing a credential.

Selecting a decision model authorizes supported, otherwise-enabled consumers to send their selected evidence to the configured endpoint. Hosted Jev requests incur TypeSafe's normal usage charges. Consumer scheduling and publication permissions remain unchanged. Clearing the role or explicitly disabling the plugin prevents its use by those consumers.

Hosted model facts

The pinned typesafe/jev-1.13.0 catalog row declares these TypeSafe model facts, checked October 16, 2026:

Fact Published value
Input price $0.042 per million tokens
Output price Free
Per-request context 64,000 tokens: state plus all questions
Per-question context 32,000 tokens: state plus each individual question

These native decision limits are separate from chat context/output limits. Billing metadata is labeled provider-docs; it is a dated estimate source, not a guarantee of future charges. At the published rate, 100 observed input tokens cost $0.0000042. An explicit provider-reported charge, including zero, takes precedence over an estimate.

As of October 16, 2026, jev-latest points to jev-1.13.0 and jev-preview points to jev-1.14.0, released October 14. Both aliases can move. Their catalog rows deliberately omit fixed prices and token limits; selecting an alias does not pin or rewrite it. Pin a version when your evaluation thresholds depend on that version's behavior. The API response identifies the model that answered.

Hosted billing and limits apply only to the matching prepared TypeSafe route. A custom endpoint, loopback server, or different API path cannot inherit those facts merely by accepting a Jev model ID. Local Kev pricing stays unknown. Third-party pricing feeds do not fill TypeSafe's missing prices.

TypeSafe also documents an opt-in TypeSafe-Danger-Zone-Long-Context: true request header that raises both context limits to 262k tokens at the same token rate. It is off by default, slower, and less accurate. The static catalog declares the default limits; it does not enable the header or advertise the larger limit for ordinary requests. Prepared requests with the effective header set to true omit the default context limits and retain the hosted token rate. Provider headers, per-model headers, and finally request.headers determine the effective value, with case-insensitive header names.

Local System One server

Run Kev

Kev is an Apache-2.0 family of decision models that serves the System One API through a persistent Python process. It supports Apple Silicon and CUDA. The Qwen3-based Kev-0.6B, Kev-4B, and Kev-8B checkpoints have been tested with this adapter.

For a Mac, start with the Qwen3-based Kev-4B checkpoint. This example requires Python 3.12+ and uv, and pins the tested adapter revision in a local directory. The first server start also downloads its base model weights:

git clone https://github.com/jaredpalmer/kev.git
cd kev
git checkout 5f78968927069eaacc3b2bdb688586989b3933ac
uv sync --frozen --extra serve
uv run python - <<'PY'
from huggingface_hub import snapshot_download

snapshot_download(
    "jaredpalmer/kev-4b",
    revision="c4bfa11b0dc07691884f2d97f1c4c4c05c92e416",
    local_dir="models/kev-4b-qwen3",
)
PY
KEV_DTYPE=bf16 uv run --extra serve python -m kev.serve \
  --run models/kev-4b-qwen3 --port 8009

The newer default Kev-4B checkpoint uses Qwen3.5; its Mac performance differs from the Qwen3 checkpoint above. Follow the upstream model cards when choosing another checkpoint. Kev-0.6B uses less memory; Kev-8B trades more memory and latency for decision quality. All of them use the same OpenClaw model label for the server you configure below.

From the same Kev checkout in a second terminal, verify the loaded checkpoint and run Kev's API tests:

curl --fail http://127.0.0.1:8009/v1/models
KEV_BASE_URL=http://127.0.0.1:8009 \
  uv run --extra serve python -m pytest tests/test_api.py -q

Connect OpenClaw

Start your System One server separately, then set baseUrl to its loopback origin and select typesafe/kev-latest:

{
  plugins: {
    allow: ["typesafe"],
    entries: {
      typesafe: {
        enabled: true,
        config: { baseUrl: "http://127.0.0.1:8009" },
      },
    },
  },
  agents: {
    defaults: { decisionModel: "typesafe/kev-latest" },
  },
}

Merge the example with existing settings, preserving other allowed plugins. Omit apiKey for local inference. The plugin does not read or send the hosted credential on this path; remove a retained SecretRef if the host should also stop preparing it.

The endpoint applies to every request from this plugin, including requests whose model label names Jev. Model selection does not choose between hosted and local endpoints. The kev-latest label requires baseUrl and is never sent to the hosted TypeSafe endpoint.

baseUrl accepts HTTP or HTTPS on localhost, 127.0.0.1, or [::1], with an optional port and trailing slash. Supply the origin, without /v1, credentials, query, or fragment; the plugin appends /v1/systemone. This existing plugin setting remains loopback-only. For a separate inference host or reverse proxy, use the standard model-provider configuration below. Ordinary ambient HTTP proxy variables are not used for configured endpoints; explicitly enabled managed proxy policy still applies. localhost stays pinned to loopback, independently of system DNS.

Kev runs one checkpoint per server process. Its request model label does not load or switch weights. Choose the checkpoint when starting the server and inspect GET /v1/models to verify it. See Kev's serving instructions for installation, model selection, and hardware requirements. OpenClaw does not download weights or start that process. An unavailable server produces an unavailable decision, without automatically switching to hosted Jev.

For lower latency, batch independent questions over the same state in one call. Keep repeated evidence unchanged when possible so Kev can reuse its prefix cache. The tested Kev server serializes inference; more concurrent HTTP calls increase queueing time. Native decisions admit at most four concurrent requests per provider and return overloaded beyond that limit. The core evaluation tool uses the same admission limit.

Cancellation closes OpenClaw's HTTP request, but the Kev server may finish inference already in progress. Avoid immediately resubmitting canceled work; choose a deadline that allows for inference and queueing on your hardware.

For local compatibility, omitted question instructions are sent as null. Structured Score rubric levels are encoded as text; returned legends must match that transmitted rubric before the original level descriptions are restored in decision results. Kev's optional nonnegative latency_ms field is validated and removed; all answer types, labels, probabilities, and rubric bounds retain the same validation as hosted results.

Custom endpoint and provider request settings

Use the same models.providers.<id> configuration as other providers to connect TypeSafe decisions to a self-hosted System One server or HTTPS reverse proxy:

{
  plugins: {
    allow: ["typesafe"],
    entries: { typesafe: { enabled: true } },
  },
  models: {
    providers: {
      typesafe: {
        baseUrl: "https://decision.example.com/v1",
        // Optional: omit when this server does not require authentication.
        apiKey: { source: "store", provider: "default", id: "SYSTEM_ONE_TOKEN" },
        models: [],
      },
    },
  },
  agents: {
    defaults: { decisionModel: "typesafe/kev-latest" },
  },
}

Merge the example with existing configuration. The provider baseUrl is the API root; OpenClaw appends /systemone. An origin-only URL implies /v1, so both https://decision.example.com and https://decision.example.com/v1 target /v1/systemone. Do not include URL credentials, a query, or a fragment.

The adapter uses the shared provider HTTP request and network-policy owners:

  • A configured custom/local origin is trusted for ordinary LAN and tailnet addresses by default. No extra TypeSafe network switch is required.
  • models.providers.typesafe.request.allowPrivateNetwork: false opts out of exact-origin private-network trust. true explicitly permits broader private destinations under the same policy used by other model providers. Use it only when that broader access is intended; it is not needed for an ordinary private HTTPS reverse proxy.
  • headers, authHeader, and request.auth use the shared header/auth contract. Explicit bearer/header auth overrides reject empty or unavailable credentials rather than silently sending an unauthenticated request. request.proxy and request.tls use the shared transport contract, including private CA certificates and client TLS credentials. TLS certificate/hostname verification stays enabled; insecure verification overrides are rejected.
  • The common provider contract accepts HTTP(S) API roots. Prefer HTTPS outside loopback so evaluation content and credentials are encrypted. Redirects remain rejected, and ambient proxies are not implicitly enabled for configured endpoints.
  • timeoutSeconds sets the provider request budget, still bounded by the native decision deadline. It does not extend the consumer's deadline or add retries.

See custom provider network policy for the shared exact-origin and private-network rules.

Trust and credentials: Evaluation state, questions, and rubrics are user content. They leave the Gateway host when the selected endpoint is remote. Only configure a server you trust with that content. Authentication proves OpenClaw's identity to the server; it does not establish that the server is trustworthy. Request content and model output cannot change the destination.

An explicit models.providers.typesafe entry owns the complete route and uses only its configured, host-prepared apiKey or request credentials. It never reads or inherits plugins.entries.typesafe.config.apiKey, which remains hosted-only, and does not search ambient or profile credentials. Unavailable configured secrets fail closed, without unauthenticated or hosted fallback. Removing an unused legacy SecretRef also stops the host preparing it. Without a model-provider entry, existing hosted and plugin loopback configuration behaves as before.

Use the server's routing model ID in decisionModel. The selected model portion is transmitted unchanged; routers such as Llama swap may reject kev-latest if no route has that name. models: [] supplies the ordinary custom-provider config shape without registering conversational models. The native decision role can use custom IDs not among the plugin's declared picker presets. This endpoint support does not relax System One response validation or guarantee compatibility with additional response fields returned by every server implementation.

Decision contract

Consumers call the provider-neutral decision runtime. The host supplies the model selected for the owning agent. The adapter translates the supported question types:

OpenClaw TypeSafe Result
Choice Choice Reported label and probability estimates
Score Score Reported fractional zero-based rubric position and estimates
Boolean Noul Probability of true, preserved from 0 to 1

Choice supports 2–255 alternatives; Score supports 2–10 rubric levels. Unsupported input is rejected before transmission; the adapter does not truncate or split a consumer's rubric. Responses must match the complete question batch, its labels, types, and rubric bounds. Additional top-level response and usage fields outside the specification are accepted but are not exposed in provider-neutral results. Required fields and the plugin’s JSON size, depth, and node limits still apply.

Reported probabilities can be rounded, so they may not sum exactly to one. A reported label or Score can also differ from a calculation over those estimates. OpenClaw preserves the returned values. Normalizing estimates or choosing their largest value is an explicit consumer policy. Probabilities and confidence are not demonstrated accuracy guarantees or permission to act.

The host owns concurrency, circuit health, deadlines, cancellation, and provider lifecycle. Native decisions have a 30-second maximum; shorter consumer or plugin timeouts still apply. Requests use the fixed TypeSafe HTTPS endpoint unless provider configuration or the existing loopback baseUrl selects a self-hosted server. Both paths reject redirects, and do not retry automatically. Consumers decide what to do with unavailable decisions; caller cancellation must not start fallback work.

Experimental image decisions

Hosted Jev accepts one tagged PNG or JPEG image with optional accompanying text through Decision V2. The Jev, Jev Preview, and Jev 1.13.0 catalog rows advertise image input. Local Kev, custom endpoints, and other model IDs remain text-only.

For example, pass this shape to decision_evaluate, replacing the placeholder with the base64 bytes of a real PNG:

{
  "contractVersion": 2,
  "state": {
    "type": "image",
    "dataUri": "data:image/png;base64,<PNG_BYTES>",
    "text": "Inspect the delivery area."
  },
  "questions": {
    "package": { "type": "boolean", "instructions": "Is a package visible?" }
  }
}

The adapter sends the image once in TypeSafe's referenced assets map and enables TypeSafe-Danger-Zone-Images: true only for image requests. Text-only requests do not send this header, even when configured manually. Prepared credentials, headers, deadlines, cancellation, and usage accounting follow the same transport as text decisions; there is no separate image credential or endpoint.

OpenClaw limits the decoded image to 4 MiB. PNG and JPEG must be still images; WebP, remote image URLs, Box/Tags questions, and images embedded in instructions or criteria are not supported by this adapter. TypeSafe additionally requires 280–1568 pixels on each side and rejects invalid or animated images rather than resizing them. Ordinary text/JSON guards remain unchanged; the serialized image request uses TypeSafe's 32 MiB body limit to allow base64 expansion.

This is TypeSafe's experimental image API, not a production accuracy guarantee. Its format, limits, and accuracy may change. See the current TypeSafe image documentation before use. Only send images you are authorized to share with the hosted service.

Agent evaluation tool

Core provides decision_evaluate automatically when an agent has an effective decisionModel selection. Normal tool policy, including explicit denies, and the active harness's capabilities still apply. No TypeSafe tool registration or additional enablement setting is required. See the provider-neutral tool contract for its request shape and results.

The tool accepts explicit shared state and independent boolean, choice, and score questions. The calling agent's trusted identity selects its inherited or per-agent decisionModel; there is no per-call provider or model override. With a TypeSafe selection, the adapter translates Boolean questions to Noul and sends only the supplied evidence to hosted Jev or the configured endpoint.

A temporary credential or provider failure returns an actionable unavailable result and leaves the configured tool available. Clearing the agent's effective selection removes eligibility through the normal tool/context refresh lifecycle. Execution rechecks the selection and current authority. timeoutMs defaults to 30,000 ms and caps provider requests at the shorter of this setting and the host's remaining deadline. Typed answers supply evidence, not permission to publish, send messages, or change durable state.

HTTP 400 (Bad Request), HTTP 413 (Content Too Large), and TypeSafe's documented 422 request-validation response return unsupported-input, not a provider outage. A request rejection does not establish context overflow specifically. The adapter cancels error bodies without reading them because they can reflect credentials or submitted evidence. Authentication (401/403), rate limits (429), and other HTTP/transport failures retain their existing classifications; no automatic retry is added.

Existing external installation

The official package keeps the typesafe plugin ID used by the prototype and earlier development checkouts. Preserve plugins.entries.typesafe, its protected credential, and agent decisionModel selections when switching. Use the supported plugin management flow to replace the old installation, and remove an explicit prototype path from plugins.load.paths if it would override the installed package. Do not configure two copies as independent providers. Installing the package does not delete prototype files or credentials.