More intelligence from every accelerator.
Zyphra’s Optimization Engine includes model and hardware specific kernels, algorithms and routing to deliver maximum intelligence and efficiency, in training and inference.
Measured gains. From kernels to serving.
Selected optimization results on AMD Instinct.
- Faster time to first token¹
- 3.6×
- Higher per-user output speed²
- 1.9×
- Lower cost per session³
- 60%
Results reflect specific test configurations.
¹ GLM-5.3, 8x MI355X, 512K context, 90% prefix-cache hits, single request. vLLM v0.29.0 baseline 42.3 s vs. ZOE 11.8 s. Source: Zyphra Inference Wave 2.
² GLM-5.3-Flash, 8x MI300X, SemiAnalysis AgentX agentic coding benchmark, 64 concurrent sessions. Stock SGLang v0.5.22 with MTP speculative decoding: 43.9 tok/s per user. ZOE: 83.5 tok/s per user. Internal benchmark, October 2026.
³ Same setup. At about 90 tok/s per user, stock SGLang serves 24 concurrent sessions per node and ZOE serves about 60, so each session costs about 60% less. Internal benchmark, October 2026.
Makes models run efficiently on the hardware you choose.
ZOE brings together kernels, parallelism, speculative decoding, quantization and tuning. These address how a workload uses processing power, memory and communication across accelerators.
- 01
Kernels
Custom kernels for each model on each accelerator.
- 02
Parallelism
Novel parallelism schemes for each model and interconnect topology.
- 03
Speculative decoding
Speculators chosen or trained from scratch for each served model.
- 04
Quantization
Custom schemes for lower-precision weights and KV cache with rigorous quality checks.
- 05
Tuning
Vendor libraries, containers, and serving configurations. Tuned for performance and stability.
Built to excel in what agents need most.
Long context
Handle extremely long contexts without degradation.
Long horizon workloads
Coherence across thousands of steps, tool calls and decisions.
Throughput at scale
A high-throughput stack for real-time multi-agent systems.
Experience ZOE on Zyphra’s stack or yours.
Zyphra’s model endpoints
Frontier model API endpoints powered by Zyphra’s Optimization Engine deliver best in class throughput and latency for developers.
Driving your sovereign Zyphra stack
Zyphra’s Optimization Engine maximizes the intelligence and efficiency of dedicated sovereign deployments.
Your stack optimized
Deploy Zyphra’s Optimization Engine in your stack to get the most performance out of your accelerators.