ZigLlama -- Educational LLM Inference in Zig¶

What is ZigLlama¶

ZigLlama is an educational implementation of transformer-based large language models written entirely in Zig. It covers the full inference stack -- from raw tensor storage to autoregressive text generation -- organised into six progressively more complex layers. Every module is documented with the mathematical theory it implements, the design trade-offs it embodies, and the connections to production systems such as llama.cpp.

The project targets two audiences simultaneously. For learners, ZigLlama provides a self-contained curriculum: each layer depends only on the layers below it, so you can study attention mechanisms without first understanding quantisation, or explore KV-caching without reading the tokeniser. For practitioners, the same codebase demonstrates production-grade techniques -- SIMD matrix kernels, K-quantisation, grammar-constrained decoding, streaming generation, and memory-mapped model loading -- all expressed in readable, allocation-explicit Zig.

Why Zig for LLM Inference¶

Architecture at a Glance¶

The following diagram shows the six layers of ZigLlama as a dependency stack. Each layer builds exclusively on the layers beneath it.

block-beta
  columns 1
  block:L6["Layer 6 -- Inference"]
    G["Text Generation"] S["Sampling Strategies"] KV["KV Cache"] ST["Streaming"] B["Batching"]
  end
  block:L5["Layer 5 -- Models"]
    LL["LLaMA / Mistral / GPT-2 / ..."] TK["Tokeniser"] GU["GGUF Loader"] CF["Config"]
  end
  block:L4["Layer 4 -- Transformers"]
    ATT["Multi-Head Attention"] FF["Feed-Forward Networks"] TB["Transformer Blocks"]
  end
  block:L3["Layer 3 -- Neural Primitives"]
    ACT["Activations (SwiGLU, GELU, ...)"] NRM["Normalization (RMSNorm, LayerNorm)"] EMB["Embeddings & RoPE"]
  end
  block:L2["Layer 2 -- Linear Algebra"]
    MAT["SIMD Matrix Ops"] QNT["Quantisation (K-quant, IQ)"] BLS["BLAS Integration"]
  end
  block:L1["Layer 1 -- Foundation"]
    TEN["Tensors"] MEM["Memory Mapping"] GGF["GGUF Format"] THR["Threading"]
  end

  L6 --> L5
  L5 --> L4
  L4 --> L3
  L3 --> L2
  L2 --> L1

Learning Paths¶

ZigLlama supports multiple learning trajectories depending on your background and available time.

Fast Track (2--4 hours)¶

For experienced ML engineers who want to understand how inference works at the systems level.

Complete Journey (1--2 weeks)¶

Work through every layer from the ground up. This path assumes familiarity with basic linear algebra (\( Ax = b \), matrix multiplication) and a working knowledge of at least one systems language.

API Reference¶

For developers integrating ZigLlama into their own projects or extending it with new model architectures, the API Reference provides per-module documentation of every public function and type.

Key Statistics¶

The numbers below are computed from the repository at the v0.1.0 tag.

Performance Highlights¶

Quick Navigation¶

Supported Model Architectures¶

ZigLlama implements 18 of the 94 architecture families tracked by llama.cpp. These 18 families cover approximately 80% of real-world model usage.

How to Cite¶

If you use ZigLlama in academic work, please cite:

@software{zigllama2024,
  title   = {ZigLlama: Educational LLM Inference in Zig},
  author  = {Dipankar Sarkar and Contributors},
  year    = {2024},
  url     = {https://github.com/dipankar/zigllama},
  version = {0.1.0}
}

License¶

ZigLlama is released under the MIT License. See LICENSE for the full text.