# Llama model-file inspection

Evidence for `inside-llm-model-files-understanding-safetensors-and-gguf.md`.
No inference throughput or model-quality benchmark is performed here.

## Inputs

- Model: `meta-llama/Llama-3.2-3B-Instruct`
- Model revision: `0cb88a4f764b7a12671c53f0838cd831a0843b95`
- llama.cpp: `620d2225cc436ab0b001e3200e590e3b286ef9ab` (0.7.0, build 11669)
- Python 3.14.4; NumPy 2.3.5; Torch 2.13.0; Transformers 5.12.1;
  Safetensors 0.8.0; SentencePiece 0.2.1.
- `gguf-py` comes from that llama.cpp checkout (package version 0.19.0).

The source checkpoint is already available in the author's Hugging Face cache.
To obtain the same inputs elsewhere, download the pinned repository revision
with your own Hugging Face access. The audit checks the hashes of both shards,
the configuration, the index and the tokenizer assets before inspecting them.

## Reproduce

From the website root, prepare the source and a Python environment:

```sh
git clone https://github.com/ggml-org/llama.cpp.git .cache/llama-model-files-source
git -C .cache/llama-model-files-source checkout 620d2225cc436ab0b001e3200e590e3b286ef9ab

uv venv --python 3.14 .cache/model-files-env
uv pip install --python .cache/model-files-env/bin/python \
  numpy==2.3.5 torch==2.13.0 transformers==5.12.1 \
  safetensors==0.8.0 sentencepiece==0.2.1
. .cache/model-files-env/bin/activate
```

Use llama.cpp binaries built from the same commit. The recorded run used
Homebrew's llama.cpp 0.7.0 on macOS arm64, built with AppleClang 21.0.0.
To build the tools yourself:

```sh
cmake -S .cache/llama-model-files-source -B .cache/llama-model-files-build
cmake --build .cache/llama-model-files-build --parallel \
  --target llama-quantize llama-cli llama-completion
export PATH="$PWD/.cache/llama-model-files-build/bin:$PATH"
```

Run the conversion and audit, supplying the local model snapshot directory:

```sh
sh scripts/model-files/reproduce.sh MODEL_DIR \
  .cache/llama-model-files-source .cache/model-files \
  .cache/model-files/inspection.json
```

`reproduce.sh` runs conversion to BF16 and two independent quantizations from
that BF16 file. It records logs under the working directory. Its metadata
override gives the model an explicit basename, fine-tune label and source URL,
so the name of the local snapshot directory is not mistaken for a fine-tune.
It does not modify the input checkpoint or fetch model files.

For an audit of existing outputs, without converting them again:

```sh
python scripts/model-files/audit_model_files.py --model-dir MODEL_DIR \
  --llama-source .cache/llama-model-files-source \
  --gguf-dir .cache/model-files --report .cache/model-files/inspection.json
```

The audit checks every source tensor against GGUF BF16, including Q/K row
permutations and BF16-to-F32 normalization weights. It independently recreates
the generated RoPE factors and decodes the first Q4_0 block of
`blk.0.ffn_gate.weight`, checking it against `gguf-py`'s decoder. The two Q4_0
outputs must differ in tensor content only at `token_embd.weight`.

The report separates tensor payload from total file size. Both bits-per-weight
figures divide by the source's 3,212,749,824 learned parameters; the generated
64-value RoPE tensor is included in the bytes but not in that denominator.
MB and GB in the article use powers of 1,000, not powers of 1,024.

To check loading and invalid tensor values without generating an answer:

```sh
for model_file in .cache/model-files/*.gguf; do
  llama-completion --model "$model_file" --n-gpu-layers 0 \
    --ctx-size 64 --n-predict 0 --prompt Hello --no-conversation \
    --no-warmup --check-tensors > "${model_file%.gguf}-load.log" 2>&1 || exit
done
```

This is a loading check, not a semantic equivalence test. The recorded run used
macOS with Metal available even though GPU layer offload was disabled.

## Published artifacts

The article's public asset directory contains `inspection.json`, one tensor
inventory for each GGUF and `conversion-checks.json` with all 254 comparisons.
`app/assets/blog/inside-llm-model-files-understanding-safetensors-and-gguf/inspection.json`
is a byte-identical copy used by the existing JSON viewer.

The public directory also contains copies of these scripts, this README, the
metadata override, the figure generator, and sanitized conversion, quantization
and loading logs. Paths are replaced by `MODEL_DIR`, `LLAMA_SOURCE` and
`WORK_DIR`; terminal escape sequences and progress updates are removed.
Do not publish the input shards, the GGUF files or virtual environments.

The report records file hashes for the actual run. Other software, compiler or
hardware versions can change a generated file; keep those details when
comparing hashes. An error measured on 32 weights says nothing by itself about
whole-model output quality.

For maintainers, regenerate the public figures and viewer copy after writing
the report into the article's public asset directory:

```sh
python scripts/model-files/prepare_assets.py --model-dir MODEL_DIR \
  --llama-source .cache/llama-model-files-source --work-dir .cache/model-files
```

This also copies the source downloads and sanitizes the six logs. Run the
loading check above first. The figure generator can run independently:

```sh
python scripts/model-files/figures.py PATH_TO_INSPECTION_JSON --output-dir OUTPUT_DIR
```

Review the generated SVGs and article together before running the website's
content validation, build and distribution checks.