Analysis and engineering work
Articles 7
The hidden memory cost of LLM inference: KV cache explained
Calculate KV cache memory and understand how context length, architecture and concurrency affect serving capacity.
13 min read
From safetensors to tokens, executing a Transformer step by step
Follow a prompt through Llama 3.2, from stored weights to attention, softmax and the next token, with measured values and Rust code.
14 min read
Inside LLM model files: understanding safetensors and GGUF
Inspect Llama 3.2 model files, understand how Safetensors and GGUF store tensors, and verify that a BF16 conversion preserves the learned weights.
10 min read
Isolating the Inference Engine to Test Without a GPU
A shared contract and a deterministic stub engine to test orchestration independently of the model weights.
7 min read
Choosing a data query engine is not a speed contest
A practical guide to choosing the right data query engine by starting with workload, benchmarks, operational costs, and long-term tradeoffs.
6 min read
Building an observability platform with ClickHouse: From internal telemetry to customer-facing analytics
Use ClickHouse for internal telemetry and customer-facing analytics while designing ingestion, tenancy and query boundaries.
14 min read
The hidden cost of observability: cardinality, sampling and telemetry at scale
Control telemetry costs through cardinality limits, sampling, retention and measures of investigative usefulness.
6 min read
A technical problem to solve?
Describe your context and the difficulties you are facing.
Talk to Stratorys →