Quick Start Guide¶

This page gets you from a verified installation to interactive text generation in under ten minutes. It assumes you have already completed the Installation steps and that zig build test passes.

Running Your First Demo¶

The fastest way to see ZigLlama in action is the bundled simple_demo:

zig run examples/simple_demo.zig

The demo walks through every layer of the architecture, printing educational commentary as it goes:

1. Tensor creation and matrix multiplication (Layer 1).
2. SIMD-accelerated operations and quantisation (Layer 2).
3. Activation functions and normalisation (Layer 3).
4. Multi-head attention and feed-forward networks (Layer 4).
5. LLaMA model initialisation and tokenisation (Layer 5).
6. Autoregressive text generation with sampling (Layer 6).

Basic Usage Example¶

The following program is a minimal but complete inference pipeline. It initialises a LLaMA-7B configuration, creates a model and tokeniser, and generates text from a prompt.

const std = @import("std");
const zigllama = @import("src/main.zig");

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    const config = zigllama.models.config.ModelConfig.llama(.LLaMA_7B);
    var model = try zigllama.models.llama.LLaMAModel.init(config, allocator);
    defer model.deinit();

    var tokenizer = try zigllama.models.tokenizer.SimpleTokenizer.init(allocator, config.vocab_size);
    defer tokenizer.deinit();

    var generator = zigllama.inference.generation.TextGenerator.init(&model, &tokenizer, allocator, null);
    const result = try generator.generate("The future of AI is");
    defer result.deinit(allocator);

    std.debug.print("Generated: {s}\n", .{result.text orelse ""});
}

Step-by-step Walkthrough¶

sequenceDiagram
    participant User
    participant Generator as TextGenerator
    participant Model as LLaMAModel
    participant Tokeniser as SimpleTokenizer

    User ->> Generator: generate("The future of AI is")
    Generator ->> Tokeniser: encode(prompt)
    Tokeniser -->> Generator: token_ids[]
    loop For each new token
        Generator ->> Model: forward(token_ids)
        Model -->> Generator: logits[vocab_size]
        Generator ->> Generator: sample(logits)
        Generator ->> Tokeniser: decode(new_token)
        Tokeniser -->> Generator: text_piece
    end
    Generator -->> User: GenerationResult { text, tokens, timing }

Running Tests Layer by Layer¶

ZigLlama's test suite is organised to match the 6-layer architecture. You can validate each layer independently to build confidence before moving on.

# Layer 1 -- Foundation (tensors, memory management, GGUF)
zig build test-foundation

# Layer 2 -- Linear Algebra (SIMD, quantisation)
zig build test-linear-algebra

# All layers at once
zig build test

Example Files¶

The examples/ directory contains 12 self-contained programs. Each can be run directly with zig run.

Run any example from the repository root:

zig run examples/benchmark_demo.zig

Understanding the Output¶

When you run the simple demo, expect output similar to:

=== ZigLlama Foundation Layer Demo ===
Demonstrating basic tensor operations...

Matrix A (2x3):
[[1.0, 2.0, 3.0],
 [4.0, 5.0, 6.0]]

Matrix B (3x2):
[[1.0, 0.0],
 [0.0, 1.0],
 [0.5, 0.5]]

Result A x B (2x2):
[[2.5, 3.5],
 [7.0, 8.0]]

Loading a Real Model (Optional)¶

If you have a GGUF model file (e.g., downloaded from Hugging Face), you can load it directly:

const std = @import("std");
const zigllama = @import("src/main.zig");

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    // Load model from GGUF file
    const gguf_file = try zigllama.models.gguf.GGUFFile.init(
        allocator,
        "models/llama-2-7b-chat.Q4_K_M.gguf",
    );
    defer gguf_file.deinit();

    // Extract configuration from GGUF metadata
    const config = try zigllama.models.config.ModelConfig.fromGGUF(gguf_file);

    std.debug.print("Model: {s}\n", .{config.name});
    std.debug.print("Parameters: {d:.1}B\n", .{config.parameterCount()});
    std.debug.print("Vocab size: {d}\n", .{config.vocab_size});
}

Exploring the Inference Pipeline¶

The generation pipeline offers several configuration knobs. Here is a slightly more advanced example that demonstrates temperature scaling and top-p sampling:

const sampling_config = zigllama.inference.generation.SamplingConfig{
    .strategy = .top_p,
    .temperature = 0.8,
    .top_p = 0.95,
    .max_tokens = 128,
};

var generator = zigllama.inference.generation.TextGenerator.init(
    &model,
    &tokenizer,
    allocator,
    &sampling_config,
);

const result = try generator.generate("Once upon a time");
defer result.deinit(allocator);

std.debug.print("{s}\n", .{result.text orelse ""});

What to Read Next¶

After completing this quick start, choose the path that fits your goals: