Layer 1: Foundations¶

The Foundations layer is the bedrock of ZigLlama. Everything above it -- linear algebra, neural primitives, transformer blocks, model loading, and inference -- depends on the abstractions defined here. This layer answers a single architectural question: How do we represent, store, and efficiently manipulate the multi-dimensional numerical data that underlies large language models?

Learning Objectives¶

After working through the six modules in this layer you will be able to:

1. Define tensors formally and implement generic multi-dimensional arrays in Zig with row-major memory layout.
2. Explain Zig's allocator pattern and apply defer / errdefer to manage memory without a garbage collector.
3. Use memory-mapped I/O (mmap) to load multi-gigabyte model files with near-zero copy overhead.
4. Parse the GGUF v3 binary format -- headers, typed metadata, tensor descriptors, and alignment padding -- to extract model weights.
5. Integrate BLAS libraries (OpenBLAS, MKL, Accelerate) behind a vtable-based interface and fall back to a pure-Zig SIMD implementation.
6. Build a work-stealing thread pool with NUMA awareness for parallel matrix and attention operations.

Mathematical Prerequisites¶

Linear Algebra¶

The reader should be comfortable with:

Basic Calculus¶

While this layer is focused on inference (no back-propagation), understanding the chain rule and partial derivatives helps explain why certain operations are fused or reordered for numerical stability (e.g., the log-sum-exp trick inside softmax).

Components Overview¶

The table below lists every module in the Foundations layer, its primary responsibility, and the source file that implements it.

How Foundations Connect to the Transformer¶

The diagram below shows data flow from a GGUF file on disk all the way to a single transformer layer. Every box below the dashed line lives in this layer.

flowchart TD
    subgraph "Layer 1 -- Foundations"
        DISK["GGUF File on Disk"]
        MMAP["MemoryMap.fromFile()"]
        READER["GGUFReader.readFile()"]
        TENSOR["Tensor(f32) weights"]
        BLAS["BlasInterface.gemm()"]
        POOL["ThreadPool (parallel rows)"]
    end

    subgraph "Layer 2+ -- Higher Layers"
        QKV["Q / K / V Projections"]
        ATTN["Attention Scores"]
        FFN["Feed-Forward Network"]
    end

    DISK --> MMAP --> READER --> TENSOR
    TENSOR --> BLAS
    BLAS --> POOL
    POOL --> QKV --> ATTN --> FFN

Dependency Graph¶

Within the Foundations layer the modules depend on each other as follows:

graph LR
    T["tensor.zig"] --> MM["memory_mapping.zig"]
    T --> BLAS["blas_integration.zig"]
    T --> THR["threading.zig"]
    T --> GGUF["gguf_format.zig"]
    MM --> GGUF

tensor.zig is the only module with no intra-layer dependencies; it is imported by every other Foundation module.

Suggested Reading Order¶

For newcomers, we recommend the following sequence:

1. Tensor Operations -- understand the data structure everything else manipulates.
2. Memory Management -- learn Zig's ownership and allocation model.
3. Memory-Mapped I/O -- see how large files get into the address space.
4. GGUF Binary Format -- parse actual model files.
5. BLAS Integration -- accelerate the critical matrix multiply.
6. CPU Threading & NUMA -- parallelise across cores.

Each page is self-contained but forward-references are noted with links.

Key Design Decisions¶

References¶