Model Converter¶

The Model Converter (src/tools/model_converter.zig) transforms neural-network weight files between the most common serialisation formats. It optionally applies quantisation during conversion, making it possible to go from a HuggingFace SafeTensors checkpoint directly to a 4-bit GGUF file in a single pass.

Supported Formats¶

ModelFormat Enum¶

pub const ModelFormat = enum {
    PyTorch,     // .pt, .pth
    GGUF,        // .gguf
    SafeTensors, // .safetensors
    ONNX,        // .onnx
    TensorFlow,  // .pb
    Custom,      // .zigllama

    pub fn fromExtension(path: []const u8) ?ModelFormat { ... }
    pub fn toString(self: ModelFormat) []const u8 { ... }
    pub fn extension(self: ModelFormat) []const u8 { ... }
};

ConversionConfig¶

All conversion parameters are captured in a single struct:

pub const ConversionConfig = struct {
    source_format: ModelFormat,
    target_format: ModelFormat,
    quantization_type: ?QuantizationType = null,
    preserve_metadata: bool = true,
    validate_output: bool = true,
    verbose: bool = false,
    chunk_size: usize = 1024 * 1024,  // 1 MB processing chunks
};

Quantization Types¶

The QuantizationType enum covers both legacy and modern K-quant formats:

Conversion Pipeline¶

The ModelConverter.convert method executes five stages:

flowchart LR
    A[Read Source] --> B[Map Tensors]
    B --> C{Quantize?}
    C -- Yes --> D[Apply Quantization]
    C -- No --> E[Write Target]
    D --> E
    E --> F{Validate?}
    F -- Yes --> G[Validate Output]
    F -- No --> H[Done]
    G --> H

1. Read Source -- Dispatch to format-specific loader (loadGGUF, loadSafeTensors, loadPyTorch, loadCustom). Each loader populates a ModelData struct containing ModelMetadata, a list of TensorInfo descriptors, and a raw data buffer.

2. Map Tensors -- Internal normalisation step that reconciles tensor names and data types across formats.

3. Quantize -- If quantization_type is non-null, iterate over every tensor, compute per-block scale factors, and pack weights into the target bit width.

4. Write Target -- Dispatch to format-specific writer (saveGGUF, saveSafeTensors, saveCustom).

5. Validate -- Re-open the output file and verify that it is non-empty and structurally sound.

Progress Reporting¶

A callback-based progress API allows the caller to render a progress bar:

var converter = ModelConverter.init(allocator, config);
converter.setProgressCallback(myProgressFn);
try converter.convert("model.safetensors", "model.gguf");

The converter invokes the callback at each stage with a [0.0, 1.0] progress float and a human-readable message.

Model Metadata¶

Every converted model carries a ModelMetadata record:

pub const ModelMetadata = struct {
    architecture: []const u8,
    vocab_size: u32,
    context_length: u32,
    embedding_dim: u32,
    num_layers: u32,
    num_heads: u32,
    intermediate_size: u32,
    rope_theta: f32,
    created_by: []const u8,
    creation_time: u64,
    source_format: []const u8,
    quantization: []const u8,
    checksum: []const u8,
};

ConversionUtils.validateArchitecture performs sanity checks (non-zero dimensions, embedding_dim % num_heads == 0) on the metadata before writing.

CLI Usage Examples¶

The converter exposes a secondary CLI through ConverterCLI:

Basic conversion (format auto-detected)¶

model_converter model.safetensors model.gguf

Conversion with quantization¶

model_converter --quantization q4_k_m model.safetensors model.gguf

Explicit formats with verbose output¶

model_converter \
    --source-format safetensors \
    --target-format custom \
    --verbose \
    llama-7b.safetensors llama-7b.zigllama

Full help¶

model_converter --help

ZigLlama Model Converter

USAGE:
    model_converter [OPTIONS] <source> <target>

ARGUMENTS:
    <source>    Source model file
    <target>    Target model file

OPTIONS:
    --source-format <fmt>   Source format (gguf, safetensors, pytorch, custom)
    --target-format <fmt>   Target format (gguf, safetensors, custom)
    --quantization <type>   Quantization type (q4_0, q4_k_m, iq2_xs, etc.)
    --verbose, -v           Enable verbose output
    --help, -h              Show this help

Utility Functions¶

ConversionUtils provides helpers that are useful beyond the converter itself:

Source Reference¶