Inside LLM model files: understanding safetensors and GGUF

While working on LLM inference at Stratorys, we spent some time examining the model files themselves. We had a few questions: what is actually stored in a model file? How does a loader locate a particular tensor among several billion parameters? And when a checkpoint is converted to another format, how can we verify that the numerical values have been preserved?

We looked at Safetensors and GGUF, two formats commonly found in the LLM ecosystem. Safetensors is widely used to distribute model checkpoints on Hugging Face, while GGUF is the native model file format used by llama.cpp. Both can represent the weights of the same trained model, but they organize the data differently.

For this post, we used Llama 3.2 3B Instruct. We inspected its original Safetensors files, converted the checkpoint to GGUF in BF16 and compared the resulting tensors with the source.

The files behind an LLM

A checkpoint downloaded from Hugging Face contains more than the weights. For Llama 3.2 3B Instruct, the repository includes:

Llama-3.2-3B-Instruct/
├── config.json
├── generation_config.json
├── model-00001-of-00002.safetensors
├── model-00002-of-00002.safetensors
├── model.safetensors.index.json
├── special_tokens_map.json
├── tokenizer.json
└── tokenizer_config.json

The two .safetensors files contain the learned parameters. The checkpoint is split into two shards, and model.safetensors.index.json maps each tensor name to the shard where it is stored.

The other files provide information that the weight files do not contain. In particular, config.json defines the model architecture, while the tokenizer files describe the vocabulary, tokenization rules and chat template.

This separation matters when implementing or debugging a model loader. A tensor file tells us where numerical values are stored. Those values have no meaning without an implementation of the corresponding architecture.

Llama's config specifies 28 decoder layers, a hidden size of 3,072 and grouped-query attention with 24 query heads and 8 key/value heads. The vocabulary contains 128,256 tokens.

Checking the tensor inventory

We found 254 tensors containing 3,212,749,824 parameters, all stored in BF16.

The tensor count is consistent with the architecture: each decoder layer has nine weight tensors composed of four attention projections, three feed-forward projections and two normalization tensors. With 28 layers, that gives 252 tensors. The embedding table and final normalization account for the remaining two.

The parameter count also gives us an expected storage size. BF16 uses two bytes per element:

The actual size of the two Safetensors files is 6,425,529,048 bytes. The difference is only 29,400 bytes, which corresponds to the two headers and their eight-byte length prefixes.

In the first decoder layer, the query projection has shape [3072, 3072], while the key and value projections have shape [1024, 3072]. These dimensions reflect the model's grouped-query attention configuration, which uses fewer key/value heads than query heads.

The embedding table has shape [128256, 3072], corresponding to 394,002,432 parameters, or 788,004,864 bytes in BF16. On its own, it represents approximately 12.26% of the checkpoint's tensor data. Because the model ties its input and output embeddings, there is no separate lm_head.weight tensor.

Reading the Safetensors structure

Safetensors stores a compact JSON header followed by a binary data section. Unlike traditional pickle-based PyTorch checkpoints, it does not depend on Python object deserialization to reconstruct tensors. The first eight bytes of a Safetensors file specify the length of the JSON header as an unsigned 64-bit little-endian integer. The header then records each tensor's name, data type, shape and byte range within the data section.

For our checkpoint, the two shards have different headers because they contain different numbers of tensors. The offsets stored in the JSON header are relative to the beginning of the tensor data section, not the beginning of the file.

For example, the first shard contains these entries:

{
  "model.layers.0.self_attn.q_proj.weight": {
    "dtype": "BF16",
    "shape": [3072, 3072],
    "data_offsets": [964177920, 983052288]
  },
  "model.layers.0.self_attn.k_proj.weight": {
    "dtype": "BF16",
    "shape": [1024, 3072],
    "data_offsets": [939012096, 945303552]
  }
}

The size of a tensor can be checked directly from these offsets.

For the query projection:

Both calculations give the same result. The key projection occupies 6,291,456 bytes, which also matches its shape and BF16 representation.

To locate a tensor in the actual file, we add its offset to the beginning of the data section. The first input normalization tensor, for example, begins at data offset 788,004,864.

This is the absolute byte position where the tensor begins in the first shard.

Inspecting the header without loading the weights

Since the header is stored at the beginning of the file, we can read it independently of the tensor data. The following script also checks that each BF16 tensor occupies the expected number of bytes.

import json
import math
import struct
from pathlib import Path

path = Path("model-00001-of-00002.safetensors")

with path.open("rb") as f:
    header_size = struct.unpack("<Q", f.read(8))[0]
    header = json.loads(f.read(header_size))

data_start = 8 + header_size
file_size = path.stat().st_size

for name, tensor in header.items():
    if name == "__metadata__":
        continue

    assert tensor["dtype"] == "BF16"

    start, end = tensor["data_offsets"]
    expected_size = math.prod(tensor["shape"]) * 2
    stored_size = end - start

    assert stored_size == expected_size
    assert 0 <= start <= end
    assert data_start + end <= file_size

    print(name, tensor["shape"], stored_size)

This checks the byte ranges described by the header without allocating memory for the complete model. It is a targeted inspection script, not a replacement for the validation performed by the Safetensors library. The header length should also be bounded and validated before reading untrusted files.

The format provides enough information to locate and interpret the tensors, but it does not describe the calculations performed by the network. In the two shards we inspected, the optional metadata contains only {"format": "pt"}. The architecture still comes from the separate configuration file.

GGUF takes a different approach by including much more model information in the binary file itself.

Inside the GGUF file

GGUF is the model format used by llama.cpp and tools built around the GGML ecosystem. It can store the architecture settings, tokenizer, chat template and tensors in one file.

Its binary structure contains four main parts: a fixed header, key-value metadata, tensor descriptors and the tensor data section.

The fixed header starts with the ASCII identifier GGUF, followed by the format version and the counts of tensors and metadata entries. The metadata uses typed key-value fields, while the tensor descriptors provide the names, dimensions, types and data offsets.

The numerical data is stored after these sections, with the data section aligned according to the file's alignment requirements.

We converted the original Llama checkpoint to GGUF without changing the precision of the main weight matrices. The result was a 6,433,688,160-byte BF16 GGUF file.

The converted file was larger than the two original Safetensors shards. This compares a GGUF file containing weights and tokenizer metadata with the source weight files alone; it excludes the source's separate tokenizer and configuration files. GGUF does not automatically make a model smaller; it is a different container with its own metadata and representation rules.

Our file uses GGUF version 3 and contains 37 metadata entries. The architecture field is llama, and the tokenizer metadata includes 128,256 vocabulary entries, 280,147 merge rules and the chat template.

The tokenizer metadata accounts for 7,821,550 bytes, making it the largest part of the information stored before the actual weights.

The file layout can be measured precisely:

The tensor data section starts at a 32-byte-aligned position. As with Safetensors, the offsets inside the tensor descriptors are relative to the beginning of that data section.

The GGUF file contains 255 tensors, compared with 254 in the original checkpoint. That difference is explained by the conversion process, rather than by a change to the number of learned parameters.

Inspecting GGUF metadata and tensors

The gguf Python package supplied with llama.cpp can read the architecture metadata and tensor descriptors. With the package installed from the pinned revision in the reproduction instructions, run this beside the converted file:

from gguf import GGUFReader

reader = GGUFReader("Llama-3.2-3B-Instruct-BF16.gguf")
print(reader.fields["general.architecture"].contents())
print("Tensor count:", len(reader.tensors))

for tensor in reader.tensors[:5]:
    print(tensor.name, [int(d) for d in tensor.shape], tensor.tensor_type.name)

This prints llama, a tensor count of 255, and the names, dimensions and storage types of the first five tensors. We will return to GGUF's dimension convention when comparing the tensors with their Safetensors counterparts.

Reconciling the two file sizes

We can account for the difference in size without making assumptions about compression or hidden data.

The original Safetensors checkpoint contains 6,425,499,648 bytes of BF16 tensor data. The GGUF tensor data section contains 6,425,850,112 bytes, which is 350,464 bytes more.

Two operations explain this increase.

First, the converter stores 57 normalization tensors in F32 instead of BF16. Their values are preserved exactly because every BF16 value can be represented in F32, but their storage requirement doubles. This adds 350,208 bytes.

Second, it generates a rope_freqs.weight tensor containing 64 F32 values. This adds another 256 bytes.

Normalization type changes   +350,208 bytes
Generated RoPE tensor             +256 bytes
                             ------------
Additional tensor data       +350,464 bytes

That matches the measured difference between the two tensor data sections exactly.

We can also reconcile the total file sizes:

The GGUF file is 8,159,112 bytes larger. Most of that difference comes from the metadata stored before the weights, particularly the tokenizer. The rest comes from the F32 normalization tensors and generated RoPE values.

Verifying the Safetensors-to-GGUF conversion

A conversion can produce a valid GGUF file without its tensor data being directly comparable to the source checkpoint. Tensor naming, dimension conventions and some internal layouts differ between the two formats.

We therefore treated the comparison as a tensor-level audit rather than a file-level comparison.

The first difference is naming. The first decoder layer includes these mappings:

The second difference concerns dimension ordering. A feed-forward tensor with shape [8192, 3072] in Safetensors is described as [3072, 8192] by GGUF reader.

This highlight the different conventions used to express tensor dimensions. By itself, it does not indicate that the numerical matrix was transposed.

Some attention tensors require an additional transformation. During conversion, the query and key projections are row-permuted to match the rotary position embedding layout expected by llama.cpp. A direct comparison of their stored arrays would therefore report differences.

Our audit matched source and converted tensors by their roles in the architecture and accounted for the GGUF dimension convention. For the query and key projections, it applied the converter's row permutation to the source tensors before comparing them with the GGUF arrays. The normalization tensors were compared after the exact BF16-to-F32 conversion. The generated RoPE tensor was checked separately against the configuration, since it has no learned counterpart in the source checkpoint.

The result was an exact match for all 254 original tensors.

The learned values survived the conversion exactly. Two inference runtimes may still produce different results because they use different kernels, perform operations in a different order or use different generation settings.

The converter version also matters. GGUF metadata and conversion code can change, so checks written for one version may need updating for another.

Our audit uses the pinned llama.cpp revision 620d2225cc436ab0b001e3200e590e3b286ef9ab, with the inspection report, conversion checks and reproduction instructions available alongside the article.