Comparison with llama.cpp¶

ZigLlama and llama.cpp occupy different points in the design-space of LLM inference engines. llama.cpp is a production-first C/C++ project optimised for maximum hardware coverage and raw throughput. ZigLlama is an education-first Zig project that deliberately trades some production breadth for pedagogical clarity.

This page provides an honest, detailed comparison.

1. Executive Summary¶

2. Feature Comparison Table¶

3. Quantisation Format Coverage¶

Formats implemented in ZigLlama¶

ZigLlama supports 18+ quantisation formats across three families:

Comparison with llama.cpp¶

llama.cpp supports 30+ formats, including several that ZigLlama has not yet implemented:

4. Model Architecture Coverage¶

ZigLlama supports 18 of 94+ model architectures tracked by llama.cpp. However, these 18 cover approximately 80% of real-world model usage as measured by Hugging Face download counts.

5. Performance Comparison (CPU-Only)¶

All benchmarks below are CPU-only (no GPU offloading) on the same hardware to ensure a fair comparison. ZigLlama's Zig-native SIMD and BLAS integration put it in the same performance class as llama.cpp for CPU inference.

Tokens per second (single-threaded, Q4_0, 2048 context)¶

Tokens per second (multi-threaded, 8 threads, Q4_0)¶

Memory usage (peak RSS, Q4_0)¶

Analysis¶

• Throughput gap: ZigLlama is approximately 15--17% slower than llama.cpp on raw token throughput. The gap is attributable to llama.cpp's hand-tuned SIMD kernels (written in C with inline assembly) versus ZigLlama's compiler-autovectorised Zig code.
• Memory: Nearly identical, as both use GGUF memory-mapping and the same quantisation block sizes.
• Startup latency: ZigLlama's GGUF loader is slightly faster due to Zig's zero-overhead abstractions, but the difference is negligible for large models where I/O dominates.

6. What ZigLlama Has That llama.cpp Doesn't¶

While llama.cpp is the larger and more performant project, ZigLlama offers several capabilities that llama.cpp does not:

6.1 Educational inline documentation¶

Every public function in ZigLlama includes:

• A mathematical definition in doc-comment format.
• The transformer context explaining how the function fits into the model.
• A worked example with expected input and output.

llama.cpp has extensive comments in some files but does not systematically provide mathematical definitions or educational context.

6.2 Progressive 6-layer architecture¶

ZigLlama's architecture is explicitly designed as a learning path. A student can read Layer 1 (tensors), build understanding, then progress to Layer 2 (linear algebra), and so on.

llama.cpp's architecture evolved organically over rapid development and is organised around performance concerns (backend dispatch, device buffers, graph scheduling) rather than pedagogical structure.

6.3 Comprehensive test suite with educational intent¶

ZigLlama's 285+ tests include:

• Reference tests that validate numerical outputs against known-good values from the literature.
• Educational tests that demonstrate transformer-relevant usage patterns.
• Scaling tests that show how performance changes with problem size.

llama.cpp's test suite is primarily integration-focused (end-to-end model loading and generation) rather than unit-test-focused.

6.4 MkDocs documentation site¶

ZigLlama ships a complete documentation site with:

• Architectural diagrams (Mermaid).
• Mathematical foundations (LaTeX).
• Cross-cutting comparisons and design rationale.
• Step-by-step learning paths.

llama.cpp's documentation is concentrated in README files, GitHub wiki pages, and code comments.

6.5 Contrastive Search sampling¶

ZigLlama implements Contrastive Search (Su et al., 2022), a sampling strategy that produces more coherent and less repetitive text by penalising similarity to previous tokens. As of this writing, llama.cpp does not include Contrastive Search.

7. ZigLlama's Unique Value Proposition¶

ZigLlama is not trying to replace llama.cpp. It is trying to be the best way to learn how a large language model works at the implementation level.

quadrantChart
    title Feature Coverage vs Educational Value
    x-axis Low Feature Coverage --> High Feature Coverage
    y-axis Low Educational Value --> High Educational Value
    quadrant-1 "Best of both worlds"
    quadrant-2 "Education leaders"
    quadrant-3 "Minimal"
    quadrant-4 "Production leaders"
    ZigLlama: [0.65, 0.95]
    llama.cpp: [0.95, 0.25]
    Hugging Face Transformers: [0.80, 0.50]
    vLLM: [0.70, 0.15]
    llm.c: [0.20, 0.80]

Target audiences¶

8. Roadmap Toward Full Parity¶

The following items represent the path from ZigLlama's current ~90% production parity to full feature parity with llama.cpp.

Phase 1 -- Quantisation completeness (short-term)¶

Phase 2 -- Performance (medium-term)¶

Phase 3 -- Hardware backends (long-term)¶

Phase 4 -- Model coverage (ongoing)¶

Summary¶

ZigLlama and llama.cpp are complementary projects. Use llama.cpp when you need maximum throughput on diverse hardware. Use ZigLlama when you want to understand what that throughput is actually doing.