Inference Engineering
Speed you can check.
Tessen is the inference system I built for one desk-sized machine: a planner, a supervisor, and a measuring instrument around an existing llama.cpp backend. It is not a new engine — it is the discipline layer that plans a launch before it happens, refuses one that doesn't fit, and keeps the raw receipt for every number it publishes.
The measured recipe
One published figure set, produced on a single DGX Spark (GB10, 121 GiB unified memory) serving Qwen3.6-35B-A3B — a 35B mixture-of-experts model with 3B active parameters — at the UD-Q8_K_XL quantisation, and quoted identically in the README, the recipes doc, the release notes, and the ledger:
| Single-stream decode | 54.0 tok/sShallow prompt, Q8 quality tier |
|---|---|
| Single-stream, long prompt | 50.3 tok/s15.6K-token prompt |
| Prefill | 1,722.8 tok/s~15.6K-token prompt |
| Aggregate — 2 agents | 75.1 tok/sSame model, same box |
| Aggregate — 4 agents | 104.8 tok/sSame model, same box |
| Evidence base | 981 raw receipt filesLifecycle, startup, milestone, reliability, and packaging gates |
The honest comparison: a public GB10 bench of the same model shows 62.1 tok/s single-stream — at Q4. The 54.0 above is at Q8, a higher quality tier. Tessen's compare command exists for exactly this reason: it refuses to compare measurement records that are not comparable.
What Tessen does
| doctor | Measure the hostWhat the machine actually has, before anything is planned |
|---|---|
| plan | Size the launch — or refuse itWeights + buffers + KV cache + reserve, accounted against the host. A launch that doesn't fit is refused, not attempted |
| serve | Launch under the planllama-server supervised, bearer-token auth, OpenAI-compatible surface, ready-record JSON |
| bench | Measure against a manifestWorkload manifests carry mandatory provenance fields; raw output is kept |
| compare | The honesty gateRefuses to compare records whose conditions differ |
Qualified surface today: dense llama / qwen2 / qwen3 GGUF models, text chat and streaming, Linux x64 — with a test base of contract, OpenAI-surface, real-model, and managed-process suites, a Docker zero-install demo path, and the managed-control patch for llama.cpp itself. Nobody else in the public set ships the fit-refusing planner, the provenance-mandatory benchmarks, and the compare gate together. The position is trustworthy measurement and safe launch on one machine — not engine speed.