Deliver optimized tokens on your infrastructure.
License Zyphra Optimization Engine (ZOE) to power the inference services you deliver to customers. Run ZOE on your own hardware and infrastructure, tuned to supported models and accelerator configurations.
Make more of the compute you operate.
Bring ZOE’s serving optimizations into your infrastructure. Focus on the performance your customers need and the economics of delivering it.
Throughput
Optimize how many tokens your infrastructure serves within a target latency budget.
Serving cost
Evaluate cost per token across the models, hardware and traffic patterns you support.
Agent workloads
Support the long contexts and repeated model calls that agent applications demand.
Measured gains. From kernels to serving.
Selected optimization results on AMD Instinct MI355X.
- Faster time to first token¹GLM-5.3 · 512K context 90% prefix-cache hits
- 3.6×
- Higher single-user decode throughput²DeepSeek-V4-Pro · 1M context
- 40%
- Faster top-K selection kernel³Sparse attention · 1M context Kernel-only measurement
- 11.1×
Results reflect specific test configurations.
¹ GLM-5.3, 512K context, 90% prefix-cache hits: vLLM v0.20.0 baseline vs. ZOE, 42.3 → 11.8 s. Single request; one output token.
² DeepSeek-V4-Pro, 1M context: SGLang v0.5.19 baseline vs. ZOE’s CSA top-K optimization, 43.5 → 61.1 tokens/s. Decode throughput only.
³ CSA top-K selection, 1M context, one request: SGLang baseline vs. Zyphra’s HIP kernel, 748.0 → 67.2 µs. This is an individual-kernel result.
All tests: AMD MI355X, one node, no live traffic. Point-in-time measurements; results vary by workload and configuration. Source: Zyphra Inference Wave 2, 20 October 2026.
Optimization across the serving stack.
ZOE brings together model- and hardware-specific kernels, algorithms, execution optimizations and routing. It addresses how workloads use processing power, memory and communication across accelerators.
Kernels
Model- and chip-specific computation.
Parallelism
Work distributed to fit the interconnect.
Execution
Serving tuned for long-context, multi-step work.
Routing
Routing tuned to the deployment.
Your hardware. Your service. ZOE underneath.
You provide the infrastructure and deliver inference to your customers. License ZOE as the optimization software within that environment, alongside your existing orchestration.
- 01Your infrastructure
- 02Licensed ZOE
- 03Your inference service
Deliver inference for enterprise coding agents on your GPU infrastructure. License ZOE to optimize serving for large codebases and parallel agent sessions, working with the orchestration layer you operate. Licensed components depend on the deployment.
Models and accelerators
Identify the model and hardware combinations you want to serve.
Workload profile
Define context lengths, concurrency and the mix of interactive and agent requests.
Performance targets
Evaluate throughput, latency and cost per token against your current serving baseline.
Bring ZOE to your infrastructure.
Tell us about your hardware, models and the inference service you want to deliver. Let’s discuss a ZOE license for your deployment.