ZAYA1-8B
Efficient small language model for mathematical and coding problems.
About
ZAYA1-8B is an open-source small MoE model by Zyphra that pushes the frontier of intelligence density through its novel architecture and unique test-time compute inference. ZAYA1-8B excels at challenging mathematical and coding tasks that require deep reasoning expertise.
At under 1 billion active parameters, ZAYA1-8B performs strongly on reasoning, mathematics and coding benchmarks, matching or exceeding the performance of models many times its size such as Mistral-Small-4-119B, and remaining competitive with substantially larger first-generation frontier reasoning models such as DeepSeek-R1-0528, Gemini-2.5-Pro and Claude 4.5 Sonnet. With our novel Markovian-RSA test-time compute methodology, we achieve significant additional performance gains — exceeding GPT-5-High on HMMT'25 (89.6 vs 88.3) and closing in on frontier open-weight models such as DeepSeek-V3.2 on mathematics benchmarks.
Performance
ZAYA1-8B also performs competitively against recent SOTA OS models in the same weight class and against many substantially larger Western OS models across a wide range of evaluations such as mathematics (AIME and HMMT), coding (LCB), reasoning and knowledge retrieval (GPQA-Diamond) and instruction following (IFEval and IFBench).
