Product

One decision per task. Written down, gated, enforced.

sllim is two processes on your network. A control plane measures your fleet on your task contracts and writes a versioned policy. A small enforcer applies it on the request path and refuses anything the policy did not prove.

See it run Talk to us

The contract

A task you can name, with a bar you can write down.

A pack holds one task: its prompt, the labels a model may answer, the quality floor, the latency ceiling, the off-topic limit, and fixtures split into a dev slice and a sealed slice. The check is a program, not a judge: an answer passes only when it is exactly a label. The triage starter pack has 160 fixtures.

packs/triage/pack.json · excerpt160 fixtures

{
  "task_id": "support-triage",
  "labels": [
    "billing_refund",
    "billing_invoice",
    "account_access",
    "account_delete",
    "bug_report",
    "feature_request",
    "integration_help",
    "plan_change",
    "out_of_scope"
  ],
  "floor_pass_rate": 0.9,
  "latency_p95_ms_max": 2000,
  "out_of_scope_accept_rate_max": 0.1,
  "sealed_samples_min": 100,
  "fixtures": [
    {
      "fixture_id": "refund-01",
      "split": "dev",
      "label": "billing_refund",
      "input": "I was charged twice for October. Please refund the duplicate payment."
    }
  ]
}

The fleet

Only the models you already pay for.

Each entry names an endpoint, the decoding settings it is measured and served with, its prices, and the environment variables that hold its key and workspace. The file never holds a key. Models that bill per call or spend plan quota say so, and run only when paid calls are allowed. Local engines, Anthropic's API, and Ollama-native servers are spoken over plain HTTP.

config/fleet.demo.json · 2 of 14 modelsno keys

{
  "models": [
    {
      "model_id": "local-gemma-e4b",
      "provider": "openai_compatible",
      "base_url": "http://127.0.0.1:11434/v1",
      "model_name": "gemma4:e4b-mlx",
      "api_key_env": null,
      "workspace_id_env": null,
      "is_billed": false,
      "effort": null,
      "max_output_tokens": 16,
      "concurrency_max": 1,
      "price_input_nanousd_per_token": 15,
      "price_output_nanousd_per_token": 30,
      "note": "Local Ollama. Price is an operator-set hardware cost of $0.015 in and $0.03 out per million tokens."
    },
    {
      "model_id": "claude-haiku-5-5",
      "provider": "anthropic",
      "base_url": "https://api.anthropic.com/v1",
      "model_name": "claude-haiku-5-5",
      "api_key_env": "ANTHROPIC_API_KEY",
      "workspace_id_env": "ANTHROPIC_WORKSPACE_ID",
      "is_billed": true,
      "effort": "low",
      "max_output_tokens": 1024,
      "concurrency_max": 4,
      "price_input_nanousd_per_token": 100,
      "price_output_nanousd_per_token": 500,
      "note": "List price $0.10 in and $0.50 out per million tokens, up to 100K-token prompts."
    }
  ]
}

The statistics

Cheap is easy to claim. The gate makes it hard to fake.

The screen and the gate ask the same questions at two confidence levels, on fixtures that never overlap.

Compile · dev slice

Screened on an 80% bound.

Every model answers the dev fixtures. It stays eligible only if its 80% Wilson lower bound clears the floor, its upper bound on off-topic accepts stays under the limit, its p95 is under the ceiling, and no call errored. The cheapest by measured cost per call wins the draft.

Gate · sealed slice

Certified on a 95% bound.

Only the drafted model runs on the sealed slice. The pass-rate lower bound must clear the floor with a minimum sample count. Wilson, not the normal interval, because the normal interval undercovers below a few hundred items.

Drift · fingerprint

A changed model is a new model.

A policy rests on a fingerprint: the fleet settings, the wire version, and every model name the provider reported. A price change, a settings change, or a different reported model makes it stale, and the gate refuses a draft whose fingerprint moved.

The policy

The choice, with its reasons.

A versioned document per task: the model and prompt, the fallback for an outage, the constraints that decided it, the batch and fingerprint it rests on, and every model that lost, with why. Gateways can consume it as a task-to-model map.

policy.json · 9 more rejected models not shownpassed

{
  "id": "support-triage-policy",
  "version": 1,
  "status": "passed",
  "assignments": [
    {
      "task_id": "support-triage",
      "model_id": "local-gemma-e4b",
      "prompt_id": "triage-support-prompt-1",
      "fallback_model_id": null,
      "constraint_ids": [
        "floor",
        "out_of_scope",
        "latency"
      ],
      "batch_id": "batch-20261023T165405Z",
      "model_fingerprint": "sha256:e3cfce764ed72c018420d2b7a4dddf216c14489c79d90ed3218f5726c7e62c01",
      "rejected": [
        {
          "model_id": "local-qwen-2b",
          "reason": "pass rate 0.72 (36/50), 80% lower bound 0.66, is below the floor 0.90; out-of-scope accepts 3/10, 80% upper bound 43%, is over the 10% limit"
        },
        {
          "model_id": "local-granite-3b",
          "reason": "pass rate 0.90 (45/50), 80% lower bound 0.86, is below the floor 0.90; out-of-scope accepts 2/10, 80% upper bound 32%, is over the 10% limit"
        },
        {
          "model_id": "local-qwen-4b",
          "reason": "out-of-scope accepts 2/10, 80% upper bound 32%, is over the 10% limit"
        }
      ]
    }
  ]
}

The enforcer

An OpenAI-shaped call, and a refusal for everything unproven.

A sidecar with no third-party dependencies on the request path. The caller sends messages and a task id; the policy chooses the model and its settings. When the assigned endpoint is down, it tries the named fallback once.

a call from the measured run190 ms

POST /v1/chat/completions
x-sllim-task-id: support-triage

{ "messages": [ { "role": "user", "content": "Hi team, our card was charged twice for the Octo…" } ] }

→ 200
x-sllim-model-id: local-gemma-e4b
x-sllim-policy-id: support-triage-policy
x-sllim-policy-version: 1
x-sllim-fallback: false

{ "choices": [ { "message": { "content": "billing_refund" } } ] }

Code	Status	When
policy_missing	409	No policy file is loaded.
policy_not_passed	409	The policy is still a draft: its gate has not passed.
task_unassigned	409	The task id has no assignment in the policy.
fleet_missing	409	No fleet file is loaded, so no endpoint is known.
field_unsupported	400	The caller set sampling, token limits, tools, or a format the evidence was not measured with.
stream_unsupported	400	Streaming was requested; this cut answers whole replies.
provider_key_missing	503	The environment variable the fleet names for the key is not set.
provider_unavailable	503	The assigned endpoint was down, and there was no fallback or it was down too.
provider_rejected	502	The provider refused the request; the fallback would too, so it is not tried.

Where it runs

Self-hosted, and boring on purpose.

The parts that touch your traffic are small and readable, so a security review can finish.

Prompts, fixtures, and outputs stay on your network.
Keys and workspace ids are read from the environment at call time and never written to evidence.
A policy that names a model the fleet lacks stops the enforcer at startup.
A missing policy or fleet file fails every call closed instead of guessing.
Evidence is plain JSON: every call, every verdict, every reason.

Two depths, one artifact

Start with one task. Grow without a rewrite.

Developer · built

Compile, gate, and enforce one task.

A pack, a fleet, the compiler, the gate, the policy file, and the sidecar, self-hosted. What the evidence page ran.

Team · in development

Reviewers, canary, rollback, drift watch.

Human reviewers for tasks a check cannot settle, shadow and canary rollout, rollback to the last passed policy, and reruns when a price or a model changes. Not built yet.

Bring a task list.

We will tell you which of your tasks a cheaper model can prove, and what that is worth.

Talk to us