[{"data":1,"prerenderedAt":2686},["ShallowReactive",2],{"blog-\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002F":3,"blog-translation-\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002F":2665,"blog-series-\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002F":2676,"i-simple-icons:github":2677,"i-simple-icons:linkedin":2680,"i-simple-icons:bluesky":2682,"i-simple-icons:x":2684},{"id":4,"title":5,"author":6,"body":7,"cover":2653,"date":2654,"description":2655,"extension":2656,"featured":2657,"kind":2658,"language":2659,"meta":2660,"modified":2661,"navigation":1780,"path":2662,"published":1780,"readingTime":2663,"seo":2664,"seriesKey":2665,"seriesOrder":2665,"seriesTitle":2665,"sitemap":2666,"stem":2672,"tags":2673,"translationKey":2665,"__hash__":2675},"blog\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf.md","Inside LLM model files: understanding safetensors and GGUF","Lucas Jahier",{"type":8,"value":9,"toc":2640},"minimark",[10,14,17,20,25,28,39,50,57,60,63,68,76,79,86,89,553,556,567,578,582,585,588,591,742,745,748,1193,1196,1199,1717,1720,1726,1730,1733,2092,2095,2102,2105,2109,2112,2115,2122,2125,2132,2135,2142,2145,2148,2209,2212,2218,2221,2225,2238,2351,2357,2361,2364,2367,2370,2373,2380,2386,2389,2392,2442,2445,2449,2452,2455,2458,2507,2518,2521,2524,2530,2533,2539,2607,2610,2613,2637],[11,12,13],"p",{},"While working on LLM inference at Stratorys, we spent some time examining the model files themselves. We had a few questions: what is actually stored in a model file? How does a loader locate a particular tensor among several billion parameters? And when a checkpoint is converted to another format, how can we verify that the numerical values have been preserved?",[11,15,16],{},"We looked at Safetensors and GGUF, two formats commonly found in the LLM ecosystem. Safetensors is widely used to distribute model checkpoints on Hugging Face, while GGUF is the native model file format used by llama.cpp. Both can represent the weights of the same trained model, but they organize the data differently.",[11,18,19],{},"For this post, we used Llama 3.2 3B Instruct. We inspected its original Safetensors files, converted the checkpoint to GGUF in BF16 and compared the resulting tensors with the source.",[21,22,24],"h2",{"id":23},"the-files-behind-an-llm","The files behind an LLM",[11,26,27],{},"A checkpoint downloaded from Hugging Face contains more than the weights. For Llama 3.2 3B Instruct, the repository includes:",[29,30,36],"pre",{"className":31,"code":33,"language":34,"meta":35},[32],"language-text","Llama-3.2-3B-Instruct\u002F\n├── config.json\n├── generation_config.json\n├── model-00001-of-00002.safetensors\n├── model-00002-of-00002.safetensors\n├── model.safetensors.index.json\n├── special_tokens_map.json\n├── tokenizer.json\n└── tokenizer_config.json\n","text","",[37,38,33],"code",{"__ignoreMap":35},[11,40,41,42,45,46,49],{},"The two ",[37,43,44],{},".safetensors"," files contain the learned parameters. The checkpoint is split into two shards, and ",[37,47,48],{},"model.safetensors.index.json"," maps each tensor name to the shard where it is stored.",[11,51,52,53,56],{},"The other files provide information that the weight files do not contain. In particular, ",[37,54,55],{},"config.json"," defines the model architecture, while the tokenizer files describe the vocabulary, tokenization rules and chat template.",[11,58,59],{},"This separation matters when implementing or debugging a model loader. A tensor file tells us where numerical values are stored. Those values have no meaning without an implementation of the corresponding architecture.",[11,61,62],{},"Llama's config specifies 28 decoder layers, a hidden size of 3,072 and grouped-query attention with 24 query heads and 8 key\u002Fvalue heads. The vocabulary contains 128,256 tokens.",[64,65,67],"h3",{"id":66},"checking-the-tensor-inventory","Checking the tensor inventory",[11,69,70,71,75],{},"We found ",[72,73,74],"strong",{},"254 tensors containing 3,212,749,824 parameters",", all stored in BF16.",[11,77,78],{},"The tensor count is consistent with the architecture: each decoder layer has nine weight tensors composed of four attention projections, three feed-forward projections and two normalization tensors. With 28 layers, that gives 252 tensors. The embedding table and final normalization account for the remaining two.",[11,80,81],{},[82,83],"img",{"alt":84,"src":85},"Tensor organization across the decoder layers","\u002Fassets\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002Flayer-pattern.svg",[11,87,88],{},"The parameter count also gives us an expected storage size. BF16 uses two bytes per element:",[90,91,103,108,220,224,226],"svg",{"style":92,"className":93,"viewBox":95,"width":96,"height":97,"dataWidth":98,"dataHeight":99,"xmlns":100,"xmlnsXLink":101,"xmlns:h5":102},"vertical-align: -NaNem;; display: block; margin: 0 auto;",[94],"typst-doc","0 0 173.000 27.000","15.727272727272727em","2.4545454545454546em","173.000","27.000","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","http:\u002F\u002Fwww.w3.org\u002F1999\u002Fxlink","http:\u002F\u002Fwww.w3.org\u002F1999\u002Fxhtml",[104,105,107],"style",{"type":106},"text\u002Fcss",".typst-text {\n  pointer-events: bounding-box;\n}\n\n.tsel span,\n.tsel {\n  left: 0;\n  position: fixed;\n  text-align: justify;\n  white-space: nowrap;\n  width: 100%;\n  height: 100%;\n  text-align-last: justify;\n  color: transparent;\n  white-space: pre;\n}\n.tsel span::-moz-selection,\n.tsel::-moz-selection {\n  color: transparent;\n  background: #7db9dea0;\n}\n.tsel span::selection,\n.tsel::selection {\n  color: transparent;\n  background: #7db9dea0;\n}\n.pseudo-link {\n  fill: transparent;\n  cursor: pointer;\n  pointer-events: all;\n}\nsvg {\n  fill: none;\n}\n\n.outline_glyph path,\npath.outline_glyph {\n  fill: var(--glyph_fill);\n  stroke: var(--glyph_stroke);\n}\n\n.outline_glyph path,\npath.outline_glyph {\n  transition: 0.2s fill stroke;\n}\n.hover .typst-text {\n  --glyph_fill: #66bab7;\n  --glyph_stroke: #66bab7;\n}\n\n.typst-jump-ripple,\n.typst-debug-react-ripple {\n  width: 0;\n  height: 0;\n  background-color: transparent;\n  position: absolute;\n  border-radius: 50%;\n}\n.typst-jump-ripple {\n  border: 1px solid #66bab7;\n}\n.typst-debug-react-ripple {\n  border: 1px solid #cb1b45;\n}\n@keyframes typst-jump-ripple-effect {\n  to {\n    width: 10vw;\n    height: 10vw;\n    opacity: 0.01;\n    margin: -5vw;\n  }\n}\n@keyframes typst-debug-react-ripple-effect {\n  to {\n    width: 3vw;\n    height: 3vw;\n    opacity: 0.01;\n    margin: -1.5vw;\n  }\n}\n",[109,110,113,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195,200,205,210,215],"defs",{"className":111},[112],"glyph",[114,115],"path",{"id":116,"className":117,"d":119},"gl3Hi1Q0",[118],"outline_glyph","M 139 106 C 107 106 86 82 86 50 C 86 20 109 -5 139 -5 C 153 -5 165 -1 174 8 L 175 0 C 175 -63 154 -117 112 -160 C 105 -168 101 -174 101 -178 C 101 -188 105 -193 114 -193 C 123 -193 135 -181 152 -158 C 186 -110 203 -57 203 0 C 203 53 185 106 139 106 Z ",[114,121],{"id":122,"className":123,"d":124},"gl3Hi1RI",[118],"M 269 666 C 228 624 168 603 89 603 L 89 564 C 141 564 184 572 217 588 L 217 82 C 217 64 213 52 204 47 C 195 42 170 39 130 39 L 95 39 L 95 0 C 120 2 174 3 257 3 C 340 3 394 2 419 0 L 419 39 L 384 39 C 343 39 318 42 310 47 C 302 52 297 64 297 82 L 297 636 C 297 660 295 666 269 666 Z ",[114,126],{"id":127,"className":128,"d":129},"gl3Hi1RM",[118],"M 237 666 C 186 666 143 648 106 612 C 69 576 50 534 50 483 C 50 449 75 424 106 424 C 136 424 161 450 161 480 C 161 513 137 536 105 536 C 102 536 100 536 98 535 C 117 584 161 627 224 627 C 306 627 352 556 352 470 C 352 403 318 331 250 255 L 62 43 C 49 28 50 29 50 0 L 421 0 L 450 180 L 417 180 C 409 129 402 100 396 91 C 391 86 361 84 306 84 L 139 84 L 236 179 C 304 243 390 312 419 365 C 439 400 449 435 449 470 C 449 588 357 666 237 666 Z ",[114,131],{"id":132,"className":133,"d":134},"gl3Hi1RQ",[118],"M 303 353 C 369 378 431 441 431 526 C 431 569 410 604 369 631 C 333 654 292 666 246 666 C 201 666 162 654 127 631 C 88 605 68 571 68 528 C 68 495 90 472 122 472 C 154 472 176 495 176 527 C 176 560 157 578 119 580 C 145 615 186 633 242 633 C 302 633 332 598 332 527 C 332 485 324 450 309 421 C 282 373 245 364 183 364 C 171 362 165 357 165 348 C 165 333 172 333 192 333 L 235 333 C 310 333 348 280 348 173 C 348 88 317 14 241 14 C 176 14 128 36 99 80 C 134 79 160 105 160 139 C 160 173 135 198 101 198 C 62 198 42 178 42 137 C 42 88 64 49 108 18 C 147 -9 193 -22 244 -22 C 301 -22 350 -3 393 34 C 436 71 457 117 457 173 C 457 267 383 332 303 353 Z ",[114,136],{"id":137,"className":138,"d":139},"gl3Hi1RU",[118],"M 353 677 C 344 677 336 672 330 663 L 28 199 L 28 163 L 289 163 L 289 81 C 289 63 285 51 278 46 C 271 41 252 39 219 39 L 194 39 L 194 0 C 223 2 269 3 331 3 C 393 3 439 2 468 0 L 468 39 L 443 39 C 410 39 391 41 384 46 C 377 51 373 63 373 81 L 373 163 L 471 163 L 471 202 L 373 202 L 373 660 C 373 670 366 677 353 677 Z M 295 553 L 295 202 L 67 202 Z ",[114,141],{"id":142,"className":143,"d":144},"gl3Hi1RY",[118],"M 118 315 C 123 315 129 319 134 326 C 164 371 205 393 257 393 C 292 393 319 373 337 332 C 348 305 354 264 354 209 C 354 146 346 102 331 76 C 306 35 272 14 229 14 C 162 14 109 62 91 114 C 94 113 96 114 100 114 C 130 114 155 137 155 167 C 155 198 130 219 100 219 C 65 219 50 200 50 163 C 50 63 131 -22 231 -22 C 292 -22 344 0 386 44 C 428 88 449 141 449 202 C 449 260 432 310 398 353 C 361 400 315 423 259 423 C 212 423 171 408 138 378 L 138 556 C 165 548 191 544 218 544 C 264 544 304 555 338 578 C 369 597 390 616 402 634 C 408 642 411 648 411 651 C 411 661 406 666 396 666 C 341 645 294 634 256 634 C 211 634 168 643 127 662 C 122 664 118 665 114 665 C 105 665 100 656 100 637 L 100 345 C 100 324 100 315 118 315 Z ",[114,146],{"id":147,"className":148,"d":149},"gl3Hi1Rc",[118],"M 383 504 C 416 504 432 521 432 555 C 432 627 378 666 304 666 C 221 666 155 627 106 548 C 63 480 42 403 42 316 C 42 189 65 100 112 47 C 152 1 198 -22 251 -22 C 312 -22 362 1 401 47 C 438 91 457 144 457 205 C 457 266 439 318 403 361 C 365 407 316 431 257 431 C 205 431 165 402 138 346 L 138 352 C 138 465 166 561 226 605 C 252 624 279 633 306 633 C 342 633 368 623 385 602 C 351 602 334 583 334 553 C 334 525 355 504 383 504 Z M 344 340 C 355 317 361 272 361 206 C 361 141 356 98 345 76 C 325 35 294 14 251 14 C 222 14 200 24 184 44 C 171 60 162 74 158 85 C 146 116 140 163 140 227 C 140 255 144 282 151 308 C 164 355 201 399 256 399 C 295 399 325 379 344 340 Z ",[114,151],{"id":152,"className":153,"d":154},"gl3Hi1Rg",[118],"M 475 604 C 482 613 485 626 485 644 L 243 644 C 174 644 135 648 128 657 C 125 660 122 667 120 676 L 89 676 L 55 464 L 88 464 C 98 520 106 550 112 555 C 115 558 146 560 205 560 L 401 560 L 295 410 C 214 295 174 171 174 36 C 174 -3 190 -22 223 -22 C 256 -22 272 -3 272 36 L 272 87 C 272 239 296 349 343 416 Z ",[114,156],{"id":157,"className":158,"d":159},"gl3Hi1Rk",[118],"M 250 666 C 201 666 158 650 122 618 C 86 586 68 546 68 497 C 68 456 83 419 113 386 C 120 378 142 361 179 335 C 88 288 42 227 42 153 C 42 100 64 57 107 24 C 147 -7 194 -22 249 -22 C 305 -22 354 -4 395 32 C 436 68 457 115 457 170 C 457 216 441 257 408 294 C 396 307 365 330 316 361 C 393 402 431 454 431 515 C 431 606 344 666 250 666 Z M 379 515 C 379 463 348 418 286 381 L 167 459 C 136 479 120 505 120 536 C 120 596 185 633 249 633 C 318 633 379 584 379 515 Z M 250 14 C 168 14 99 73 99 153 C 99 220 136 274 210 315 L 328 240 C 376 209 400 174 400 134 C 400 62 325 14 250 14 Z ",[114,161],{"id":162,"className":163,"d":164},"gl3Hi1Ro",[118],"M 116 140 C 83 140 67 123 67 89 C 67 14 128 -22 207 -22 C 287 -22 351 18 398 99 C 437 166 457 243 457 329 C 457 456 434 546 388 599 C 349 644 304 666 253 666 C 194 666 145 645 106 602 C 63 557 42 503 42 440 C 42 379 60 327 96 284 C 134 238 183 215 242 215 C 294 215 334 243 361 299 L 361 286 C 361 177 342 103 303 63 C 272 30 239 14 206 14 C 166 14 137 23 118 42 C 149 42 165 60 165 91 C 165 119 144 140 116 140 Z M 243 246 C 204 246 175 266 155 306 C 144 329 138 374 138 439 C 138 504 145 548 158 573 C 180 613 212 633 253 633 C 294 633 324 608 343 558 C 354 531 361 485 361 420 C 361 333 322 246 243 246 Z ",[114,166],{"id":167,"className":168,"d":169},"gl3Hi1R4",[118],"M 698 367 L 80 367 C 64 367 56 359 56 344 C 56 329 64 321 80 321 L 698 321 C 714 321 722 329 722 344 C 722 356 711 367 698 367 Z M 698 179 L 80 179 C 64 179 56 171 56 156 C 56 141 64 133 80 133 L 698 133 C 714 133 722 141 722 156 C 722 169 711 179 698 179 Z ",[114,171],{"id":172,"className":173,"d":174},"gl3Hi1Tg",[118],"M 910 581 C 923 623 956 644 1009 644 L 1009 683 C 968 680 937 679 915 680 L 793 683 L 793 645 C 850 644 879 627 879 596 C 879 593 877 587 874 577 L 729 132 L 574 605 C 573 611 572 616 572 619 C 572 636 600 644 655 644 L 655 683 C 612 680 564 679 510 680 L 385 683 L 385 644 C 419 644 442 642 452 639 C 462 636 469 627 474 612 L 496 544 L 362 132 L 206 607 C 205 611 204 615 204 619 C 204 636 232 644 288 644 L 288 683 C 245 680 197 679 143 680 L 18 683 L 18 644 C 57 644 80 642 88 638 C 96 634 103 624 108 606 L 307 0 C 312 -15 319 -22 329 -22 C 335 -22 339 -21 342 -19 C 349 -11 352 -4 353 1 L 513 491 L 674 0 C 679 -15 686 -22 697 -22 C 702 -22 707 -21 710 -19 C 716 -11 719 -4 720 1 Z ",[114,176],{"id":177,"className":178,"d":179},"gl3Hi1UM",[118],"M 298 -11 C 360 -11 413 11 456 56 C 499 101 521 154 521 217 C 521 278 501 332 460 377 C 419 422 369 445 308 445 C 259 445 215 426 178 387 L 178 694 L 28 683 L 28 645 C 64 645 86 642 94 636 C 102 630 105 616 105 593 L 105 0 L 136 0 L 169 58 C 201 12 244 -11 298 -11 Z M 294 19 C 253 19 219 38 194 77 C 185 90 181 104 181 117 L 181 320 C 181 333 184 344 191 354 C 220 395 257 415 304 415 C 345 415 378 395 404 354 C 422 324 431 279 431 218 C 431 110 393 19 294 19 Z ",[114,181],{"id":182,"className":183,"d":184},"gl3Hi1UY",[118],"M 415 251 C 415 366 348 448 236 448 C 176 448 126 425 85 378 C 47 334 28 281 28 220 C 28 157 49 104 91 58 C 133 12 185 -11 247 -11 C 304 -11 348 9 378 48 C 403 81 415 105 415 121 C 415 131 409 136 398 136 C 390 136 385 131 382 120 C 360 54 317 21 253 21 C 208 21 173 42 146 85 C 127 116 118 163 117 227 L 387 227 C 409 227 415 229 415 251 Z M 326 349 C 337 316 342 284 342 255 L 118 255 C 121 321 138 366 168 389 C 193 409 215 419 236 419 C 281 419 311 396 326 349 Z ",[114,186],{"id":187,"className":188,"d":189},"gl3Hi1Ug",[118],"M 431 453 C 393 453 358 438 326 408 C 296 431 262 442 223 442 C 137 442 59 378 59 294 C 59 252 74 218 104 192 C 85 168 75 141 75 110 C 75 72 88 43 113 24 C 71 10 28 -26 28 -77 C 28 -120 56 -154 111 -178 C 153 -197 199 -206 249 -206 C 300 -206 347 -197 389 -178 C 444 -154 471 -120 471 -75 C 471 -22 449 17 405 42 C 359 67 308 70 234 70 C 190 70 166 70 161 71 C 133 75 113 102 113 133 C 113 148 117 162 126 174 C 154 155 186 145 223 145 C 309 145 386 209 386 293 C 386 332 373 364 347 389 C 372 412 399 423 428 423 C 423 418 420 410 420 400 C 420 378 431 367 453 367 C 474 367 485 378 485 401 C 485 432 461 453 431 453 Z M 223 411 C 277 411 304 372 304 294 C 304 215 277 175 223 175 C 168 175 141 214 141 293 C 141 372 168 411 223 411 Z M 164 4 L 222 4 C 274 4 316 1 348 -6 C 391 -15 412 -39 412 -77 C 412 -109 392 -134 352 -153 C 321 -168 287 -175 250 -175 C 214 -175 180 -168 148 -153 C 107 -134 87 -109 87 -77 C 87 -35 123 4 164 4 Z ",[114,191],{"id":192,"className":193,"d":194},"gl3Hi1Uk",[118],"M 314 415 C 359 415 382 380 382 309 L 382 79 C 382 60 379 48 372 44 C 365 40 343 39 306 39 L 306 0 L 421 3 L 535 0 L 535 39 C 503 39 482 40 472 43 C 462 46 458 53 458 64 L 458 253 C 458 301 457 334 454 353 C 444 414 399 445 320 445 C 259 445 213 417 182 361 L 182 694 L 32 683 L 32 645 C 68 645 90 642 98 636 C 106 630 109 616 109 593 L 109 79 C 109 60 105 48 98 44 C 91 40 69 39 32 39 L 32 0 L 147 3 L 261 0 L 261 39 C 224 39 203 40 196 44 C 189 48 185 60 185 79 L 185 261 C 185 342 236 415 314 415 Z ",[114,196],{"id":197,"className":198,"d":199},"gl3Hi1Uo",[118],"M 194 601 C 194 631 169 657 139 657 C 109 657 83 631 83 601 C 83 571 108 544 138 544 C 169 544 194 570 194 601 Z M 143 3 L 247 0 L 247 39 C 214 39 194 41 188 45 C 182 49 180 60 180 78 L 180 445 L 37 433 L 37 395 C 70 395 91 392 98 387 C 105 382 108 368 108 345 L 108 79 C 108 60 105 48 98 44 C 91 40 69 39 33 39 L 33 0 Z ",[114,201],{"id":202,"className":203,"d":204},"gl3Hi1VQ",[118],"M 360 130 C 360 206 308 254 205 274 C 158 283 128 294 112 305 C 96 316 88 332 88 351 C 88 398 123 421 193 421 C 261 421 298 383 303 306 C 304 298 309 294 319 294 C 330 294 336 303 336 321 L 336 420 C 336 439 331 448 321 448 C 310 448 295 430 285 422 C 260 439 229 448 193 448 C 104 448 33 408 33 323 C 33 289 47 260 76 237 C 111 208 144 202 207 191 C 272 178 305 149 305 104 C 305 47 270 18 199 18 C 130 18 86 64 67 155 C 64 168 59 175 50 175 C 39 175 33 165 33 146 L 33 17 C 33 -2 38 -11 48 -11 C 51 -11 56 -8 64 -1 C 82 15 74 12 92 30 C 121 3 157 -11 199 -11 C 294 -11 360 38 360 130 Z ",[114,206],{"id":207,"className":208,"d":209},"gl3Hi1VU",[118],"M 332 126 L 332 186 L 300 186 L 300 128 C 300 78 282 22 238 22 C 197 22 177 56 177 124 L 177 395 L 316 395 L 316 433 L 177 433 L 177 615 L 145 615 C 144 582 141 553 134 527 C 117 461 78 427 19 424 L 19 395 L 102 395 L 102 126 C 102 67 120 29 155 10 C 182 -4 207 -11 232 -11 C 298 -11 332 55 332 126 Z ",[114,211],{"id":212,"className":213,"d":214},"gl3Hi1Vo",[118],"M 113 -205 C 168 -205 212 -167 243 -90 L 416 332 C 433 372 463 392 508 392 L 508 431 C 475 428 449 427 430 428 L 341 431 L 341 393 C 372 392 387 379 387 354 C 387 348 385 341 381 332 L 286 98 L 180 354 C 177 362 175 368 174 371 C 174 385 193 392 230 392 L 230 431 L 118 428 C 94 427 61 428 19 431 L 19 392 C 72 392 85 394 99 360 L 246 1 L 222 -55 C 195 -119 176 -175 113 -175 C 99 -175 87 -171 76 -164 C 99 -159 110 -144 110 -120 C 110 -90 95 -75 64 -75 C 34 -75 19 -90 19 -121 C 19 -170 64 -205 113 -205 Z ",[114,216],{"id":217,"className":218,"d":219},"gl3Hi1Zg",[118],"M 630 32 C 630 39 628 44 623 49 L 422 250 L 623 451 C 628 456 630 461 630 468 C 630 481 620 491 607 491 C 600 491 595 489 590 484 L 389 283 L 188 484 C 183 489 178 491 171 491 C 158 491 148 481 148 468 C 148 461 150 456 155 451 L 356 250 L 155 49 C 150 44 148 39 148 32 C 148 19 158 9 171 9 C 178 9 183 11 188 16 L 389 217 L 590 16 C 595 11 600 9 607 9 C 620 9 630 19 630 32 Z ",[109,221],{"className":222},[223],"clip-path",[104,225],{"type":106},[227,228,235],"g",{"className":229,"transform":231,"dataTid":232,"dataPageWidth":233,"dataPageHeight":234},[230],"typst-page","translate(0, 0)","pTZB9dv+0mZwm9\u002FtlBpRrgQ","173","27",[227,236,240],{"dataTid":237,"className":238},"gTZB9dv+0mZwm9\u002FtlBpRrgQ",[239],"typst-group",[227,241,244],{"transform":242,"dataTid":243},"translate(0.000,0.000)","pvn7MhlfIzsgTQv2aEqodn0",[227,245,248,459,548],{"dataTid":246,"className":247},"gvn7MhlfIzsgTQv2aEqodn0",[239],[227,249,251],{"transform":242,"dataTid":250},"paDSZwiDlNEIXkJo6nz\u002Fi34",[227,252,255,329,349,412,431,450],{"dataTid":253,"className":254},"gaDSZwiDlNEIXkJo6nz\u002Fi34",[239],[227,256,259],{"transform":257,"dataTid":258},"translate(0.000,7.634)","pjRtK4Clreb08lXiE7u5+xM",[227,260,266,271,275,279,283,287,291,295,299,303,306,309,313],{"dataTid":261,"className":262,"fill":264,"transform":265},"gjRtK4Clreb08lXiE7u5+xM",[263],"typst-text","#000","scale(0.011,-0.011)",[267,268],"use",{"x":269,"y":269,"href":270},"0","#gl3Hi1Tg",[267,272],{"x":273,"y":269,"href":274},"1028","#gl3Hi1UY",[267,276],{"x":277,"y":269,"href":278},"1472","#gl3Hi1Uo",[267,280],{"x":281,"y":269,"href":282},"1750","#gl3Hi1Ug",[267,284],{"x":285,"y":269,"href":286},"2250","#gl3Hi1Uk",[267,288],{"x":289,"y":269,"href":290},"2806","#gl3Hi1VU",[267,292],{"x":293,"y":269,"href":294},"3195","#gl3Hi1QE",[267,296],{"x":297,"y":269,"href":298},"3527","#gl3Hi1UM",[267,300],{"x":301,"y":269,"href":302},"4083","#gl3Hi1Vo",[267,304],{"x":305,"y":269,"href":290},"4611",[267,307],{"x":308,"y":269,"href":274},"5000",[267,310],{"x":311,"y":269,"href":312},"5444","#gl3Hi1VQ",[227,314,316],{"transform":315},"scale(16,-16)",[317,318,322],"foreignObject",{"x":269,"y":319,"width":320,"height":321},"-50.38","364.88","62.50",[323,324,328],"h5:div",{"className":325,"style":327},[326],"tsel","font-size: 62px","Weight bytes",[227,330,333],{"transform":331,"dataTid":332},"translate(67.274,7.634)","pTVgzH5qlGVoRBbtwYAF8r4",[227,334,337,340],{"dataTid":335,"className":336,"fill":264,"transform":265},"gTVgzH5qlGVoRBbtwYAF8r4",[263],[267,338],{"x":269,"y":269,"href":339},"#gl3Hi1R4",[227,341,342],{"transform":315},[317,343,345],{"x":269,"y":319,"width":344,"height":321},"48.62",[323,346,348],{"className":347,"style":327},[326],"=",[227,350,353],{"transform":351,"dataTid":352},"translate(78.887,7.634)","ptmt+HRHGpbI4DvyMBzFtOQ",[227,354,357,360,364,368,372,375,378,382,386,390,393,397,400,403],{"dataTid":355,"className":356,"fill":264,"transform":265},"gtmt+HRHGpbI4DvyMBzFtOQ",[263],[267,358],{"x":269,"y":269,"href":359},"#gl3Hi1RQ",[267,361],{"x":362,"y":269,"href":363},"500","#gl3Hi1Q0",[267,365],{"x":366,"y":269,"href":367},"778","#gl3Hi1RM",[267,369],{"x":370,"y":269,"href":371},"1278","#gl3Hi1RI",[267,373],{"x":374,"y":269,"href":367},"1778",[267,376],{"x":377,"y":269,"href":363},"2278",[267,379],{"x":380,"y":269,"href":381},"2556","#gl3Hi1Rg",[267,383],{"x":384,"y":269,"href":385},"3056","#gl3Hi1RU",[267,387],{"x":388,"y":269,"href":389},"3556","#gl3Hi1Ro",[267,391],{"x":392,"y":269,"href":363},"4056",[267,394],{"x":395,"y":269,"href":396},"4334","#gl3Hi1Rk",[267,398],{"x":399,"y":269,"href":367},"4834",[267,401],{"x":402,"y":269,"href":385},"5334",[227,404,405],{"transform":315},[317,406,408],{"x":269,"y":319,"width":407,"height":321},"364.62",[323,409,411],{"className":410,"style":327},[326],"3,212,749,824",[227,413,416],{"transform":414,"dataTid":415},"translate(145.506,7.634)","pZRSO6c4p+nkXmDMBtidCsg",[227,417,420,423],{"dataTid":418,"className":419,"fill":264,"transform":265},"gZRSO6c4p+nkXmDMBtidCsg",[263],[267,421],{"x":269,"y":269,"href":422},"#gl3Hi1Zg",[227,424,425],{"transform":315},[317,426,427],{"x":269,"y":319,"width":344,"height":321},[323,428,430],{"className":429,"style":327},[326],"×",[227,432,435],{"transform":433,"dataTid":434},"translate(156.508,7.634)","pjxPYURMtyRcgim89k0Pu8I",[227,436,439,441],{"dataTid":437,"className":438,"fill":264,"transform":265},"gjxPYURMtyRcgim89k0Pu8I",[263],[267,440],{"x":269,"y":269,"href":367},[227,442,443],{"transform":315},[317,444,446],{"x":269,"y":319,"width":445,"height":321},"31.25",[323,447,449],{"className":448,"style":327},[326],"2",[227,451,453],{"transform":242,"dataTid":452},"p+nF9nphQ3U0gANvmKrhITw",[227,454],{"dataTid":455,"className":456,"dataHint":458},"g+nF9nphQ3U0gANvmKrhITw",[457],"typst-content-hint","a",[227,460,463],{"transform":461,"dataTid":462},"translate(0.000,17.050)","pdcQOS1nf1Ss2pQaXDIVHDo",[227,464,467,481,543],{"dataTid":465,"className":466},"gdcQOS1nf1Ss2pQaXDIVHDo",[239],[227,468,469],{"transform":331,"dataTid":332},[227,470,472,474],{"dataTid":335,"className":471,"fill":264,"transform":265},[263],[267,473],{"x":269,"y":269,"href":339},[227,475,476],{"transform":315},[317,477,478],{"x":269,"y":319,"width":344,"height":321},[323,479,348],{"className":480,"style":327},[326],[227,482,484],{"transform":351,"dataTid":483},"pS+8+p8fvIWE1uQjoeVYSpY",[227,485,488,491,493,495,497,500,502,504,506,508,510,512,514,516,519,522,525,528,531,534],{"dataTid":486,"className":487,"fill":264,"transform":265},"gS+8+p8fvIWE1uQjoeVYSpY",[263],[267,489],{"x":269,"y":269,"href":490},"#gl3Hi1Rc",[267,492],{"x":362,"y":269,"href":363},[267,494],{"x":366,"y":269,"href":385},[267,496],{"x":370,"y":269,"href":367},[267,498],{"x":374,"y":269,"href":499},"#gl3Hi1RY",[267,501],{"x":377,"y":269,"href":363},[267,503],{"x":380,"y":269,"href":385},[267,505],{"x":384,"y":269,"href":389},[267,507],{"x":388,"y":269,"href":389},[267,509],{"x":392,"y":269,"href":363},[267,511],{"x":395,"y":269,"href":490},[267,513],{"x":399,"y":269,"href":385},[267,515],{"x":402,"y":269,"href":396},[267,517],{"x":518,"y":269,"href":294},"5834",[267,520],{"x":521,"y":269,"href":298},"6166",[267,523],{"x":524,"y":269,"href":302},"6722",[267,526],{"x":527,"y":269,"href":290},"7250",[267,529],{"x":530,"y":269,"href":274},"7639",[267,532],{"x":533,"y":269,"href":312},"8083",[227,535,536],{"transform":315},[317,537,539],{"x":269,"y":319,"width":538,"height":321},"529.81",[323,540,542],{"className":541,"style":327},[326],"6,425,499,648 bytes",[227,544,545],{"transform":242,"dataTid":452},[227,546],{"dataTid":455,"className":547,"dataHint":458},[457],[227,549,550],{"transform":242,"dataTid":452},[227,551],{"dataTid":455,"className":552,"dataHint":458},[457],[11,554,555],{},"The actual size of the two Safetensors files is 6,425,529,048 bytes. The difference is only 29,400 bytes, which corresponds to the two headers and their eight-byte length prefixes.",[11,557,558,559,562,563,566],{},"In the first decoder layer, the query projection has shape ",[37,560,561],{},"[3072, 3072]",", while the key and value projections have shape ",[37,564,565],{},"[1024, 3072]",". These dimensions reflect the model's grouped-query attention configuration, which uses fewer key\u002Fvalue heads than query heads.",[11,568,569,570,573,574,577],{},"The embedding table has shape ",[37,571,572],{},"[128256, 3072]",", corresponding to 394,002,432 parameters, or 788,004,864 bytes in BF16. On its own, it represents approximately 12.26% of the checkpoint's tensor data. Because the model ties its input and output embeddings, there is no separate ",[37,575,576],{},"lm_head.weight"," tensor.",[21,579,581],{"id":580},"reading-the-safetensors-structure","Reading the Safetensors structure",[11,583,584],{},"Safetensors stores a compact JSON header followed by a binary data section. Unlike traditional pickle-based PyTorch checkpoints, it does not depend on Python object deserialization to reconstruct tensors. The first eight bytes of a Safetensors file specify the length of the JSON header as an unsigned 64-bit little-endian integer. The header then records each tensor's name, data type, shape and byte range within the data section.",[11,586,587],{},"For our checkpoint, the two shards have different headers because they contain different numbers of tensors. The offsets stored in the JSON header are relative to the beginning of the tensor data section, not the beginning of the file.",[11,589,590],{},"For example, the first shard contains these entries:",[29,592,596],{"className":593,"code":594,"language":595,"meta":35,"style":35},"language-json shiki shiki-themes github-dark","{\n  \"model.layers.0.self_attn.q_proj.weight\": {\n    \"dtype\": \"BF16\",\n    \"shape\": [3072, 3072],\n    \"data_offsets\": [964177920, 983052288]\n  },\n  \"model.layers.0.self_attn.k_proj.weight\": {\n    \"dtype\": \"BF16\",\n    \"shape\": [1024, 3072],\n    \"data_offsets\": [939012096, 945303552]\n  }\n}\n","json",[37,597,598,607,617,633,653,672,678,686,697,713,730,736],{"__ignoreMap":35},[599,600,603],"span",{"class":601,"line":602},"line",1,[599,604,606],{"class":605},"s95oV","{\n",[599,608,610,614],{"class":601,"line":609},2,[599,611,613],{"class":612},"sDLfK","  \"model.layers.0.self_attn.q_proj.weight\"",[599,615,616],{"class":605},": {\n",[599,618,620,623,626,630],{"class":601,"line":619},3,[599,621,622],{"class":612},"    \"dtype\"",[599,624,625],{"class":605},": ",[599,627,629],{"class":628},"sU2Wk","\"BF16\"",[599,631,632],{"class":605},",\n",[599,634,636,639,642,645,648,650],{"class":601,"line":635},4,[599,637,638],{"class":612},"    \"shape\"",[599,640,641],{"class":605},": [",[599,643,644],{"class":612},"3072",[599,646,647],{"class":605},", ",[599,649,644],{"class":612},[599,651,652],{"class":605},"],\n",[599,654,656,659,661,664,666,669],{"class":601,"line":655},5,[599,657,658],{"class":612},"    \"data_offsets\"",[599,660,641],{"class":605},[599,662,663],{"class":612},"964177920",[599,665,647],{"class":605},[599,667,668],{"class":612},"983052288",[599,670,671],{"class":605},"]\n",[599,673,675],{"class":601,"line":674},6,[599,676,677],{"class":605},"  },\n",[599,679,681,684],{"class":601,"line":680},7,[599,682,683],{"class":612},"  \"model.layers.0.self_attn.k_proj.weight\"",[599,685,616],{"class":605},[599,687,689,691,693,695],{"class":601,"line":688},8,[599,690,622],{"class":612},[599,692,625],{"class":605},[599,694,629],{"class":628},[599,696,632],{"class":605},[599,698,700,702,704,707,709,711],{"class":601,"line":699},9,[599,701,638],{"class":612},[599,703,641],{"class":605},[599,705,706],{"class":612},"1024",[599,708,647],{"class":605},[599,710,644],{"class":612},[599,712,652],{"class":605},[599,714,716,718,720,723,725,728],{"class":601,"line":715},10,[599,717,658],{"class":612},[599,719,641],{"class":605},[599,721,722],{"class":612},"939012096",[599,724,647],{"class":605},[599,726,727],{"class":612},"945303552",[599,729,671],{"class":605},[599,731,733],{"class":601,"line":732},11,[599,734,735],{"class":605},"  }\n",[599,737,739],{"class":601,"line":738},12,[599,740,741],{"class":605},"}\n",[11,743,744],{},"The size of a tensor can be checked directly from these offsets.",[11,746,747],{},"For the query projection:",[90,749,754,756,820,823,825],{"style":92,"className":750,"viewBox":751,"width":752,"height":97,"dataWidth":753,"dataHeight":99,"xmlns":100,"xmlnsXLink":101,"xmlns:h5":102},[94],"0 0 219.000 27.000","19.90909090909091em","219.000",[104,755,107],{"type":106},[109,757,759,762,767,770,773,776,779,782,785,788,791,794,797,800,803,806,809,812,815],{"className":758},[112],[114,760],{"id":116,"className":761,"d":119},[118],[114,763],{"id":764,"className":765,"d":766},"gl3Hi1RE",[118],"M 249 -22 C 390 -22 460 92 460 320 C 460 473 428 575 365 625 C 330 652 291 666 250 666 C 109 666 39 551 39 320 C 39 136 88 -22 249 -22 Z M 361 524 C 368 489 371 425 371 332 C 371 240 367 172 360 128 C 347 48 310 8 249 8 C 226 8 203 17 182 34 C 155 57 139 104 132 176 C 129 201 128 253 128 332 C 128 419 131 480 136 513 C 145 568 163 603 191 618 C 213 630 232 636 249 636 C 314 636 350 583 361 524 Z ",[114,768],{"id":122,"className":769,"d":124},[118],[114,771],{"id":127,"className":772,"d":129},[118],[114,774],{"id":132,"className":775,"d":134},[118],[114,777],{"id":137,"className":778,"d":139},[118],[114,780],{"id":142,"className":781,"d":144},[118],[114,783],{"id":147,"className":784,"d":149},[118],[114,786],{"id":152,"className":787,"d":154},[118],[114,789],{"id":157,"className":790,"d":159},[118],[114,792],{"id":162,"className":793,"d":164},[118],[114,795],{"id":167,"className":796,"d":169},[118],[114,798],{"id":177,"className":799,"d":179},[118],[114,801],{"id":182,"className":802,"d":184},[118],[114,804],{"id":202,"className":805,"d":204},[118],[114,807],{"id":207,"className":808,"d":209},[118],[114,810],{"id":212,"className":811,"d":214},[118],[114,813],{"id":217,"className":814,"d":219},[118],[114,816],{"id":817,"className":818,"d":819},"gl3Hi1dUD",[118],"M 698 270 L 80 270 C 64 270 56 263 56 250 C 56 237 64 230 80 230 L 698 230 C 714 230 722 237 722 250 C 722 262 710 270 698 270 Z ",[109,821],{"className":822},[223],[104,824],{"type":106},[227,826,830],{"className":827,"transform":231,"dataTid":828,"dataPageWidth":829,"dataPageHeight":234},[230],"p4UYCalbkse8m8sIpCCxhQ0","219",[227,831,834],{"dataTid":832,"className":833},"g4UYCalbkse8m8sIpCCxhQ0",[239],[227,835,837],{"transform":242,"dataTid":836},"pE0G+bSSjNE8doh8hELgrok",[227,838,841,1022,1188],{"dataTid":839,"className":840},"gE0G+bSSjNE8doh8hELgrok",[239],[227,842,844],{"transform":242,"dataTid":843},"pHfYc5SPSCakcA2vdAWxOx4",[227,845,848,892,911,949,964,1017],{"dataTid":846,"className":847},"gHfYc5SPSCakcA2vdAWxOx4",[239],[227,849,851],{"transform":257,"dataTid":850},"pQMiv\u002FYlijzEmic6pLXGb4s",[227,852,855,857,859,862,865,868,870,873,876,878,880,883],{"dataTid":853,"className":854,"fill":264,"transform":265},"gQMiv\u002FYlijzEmic6pLXGb4s",[263],[267,856],{"x":269,"y":269,"href":389},[267,858],{"x":362,"y":269,"href":396},[267,860],{"x":861,"y":269,"href":359},"1000",[267,863],{"x":864,"y":269,"href":363},"1500",[267,866],{"x":374,"y":269,"href":867},"#gl3Hi1RE",[267,869],{"x":377,"y":269,"href":499},[267,871],{"x":872,"y":269,"href":367},"2778",[267,874],{"x":875,"y":269,"href":363},"3278",[267,877],{"x":388,"y":269,"href":367},[267,879],{"x":392,"y":269,"href":396},[267,881],{"x":882,"y":269,"href":396},"4556",[227,884,885],{"transform":315},[317,886,888],{"x":269,"y":319,"width":887,"height":321},"316.00",[323,889,891],{"className":890,"style":327},[326],"983,052,288",[227,893,896],{"transform":894,"dataTid":895},"translate(58.060,7.634)","p7TDcO2G7EbAoRTDHdOA0k0",[227,897,900,903],{"dataTid":898,"className":899,"fill":264,"transform":265},"g7TDcO2G7EbAoRTDHdOA0k0",[263],[267,901],{"x":269,"y":269,"href":902},"#gl3Hi1dUD",[227,904,905],{"transform":315},[317,906,907],{"x":269,"y":319,"width":344,"height":321},[323,908,910],{"className":909,"style":327},[326],"−",[227,912,915],{"transform":913,"dataTid":914},"translate(69.063,7.634)","pYIh7mSwgREkQxRpU8N\u002Fc1Y",[227,916,919,921,923,925,927,929,931,933,935,937,939,941],{"dataTid":917,"className":918,"fill":264,"transform":265},"gYIh7mSwgREkQxRpU8N\u002Fc1Y",[263],[267,920],{"x":269,"y":269,"href":389},[267,922],{"x":362,"y":269,"href":490},[267,924],{"x":861,"y":269,"href":385},[267,926],{"x":864,"y":269,"href":363},[267,928],{"x":374,"y":269,"href":371},[267,930],{"x":377,"y":269,"href":381},[267,932],{"x":872,"y":269,"href":381},[267,934],{"x":875,"y":269,"href":363},[267,936],{"x":388,"y":269,"href":389},[267,938],{"x":392,"y":269,"href":367},[267,940],{"x":882,"y":269,"href":867},[227,942,943],{"transform":315},[317,944,945],{"x":269,"y":319,"width":887,"height":321},[323,946,948],{"className":947,"style":327},[326],"964,177,920",[227,950,952],{"transform":951,"dataTid":332},"translate(127.734,7.634)",[227,953,955,957],{"dataTid":335,"className":954,"fill":264,"transform":265},[263],[267,956],{"x":269,"y":269,"href":339},[227,958,959],{"transform":315},[317,960,961],{"x":269,"y":319,"width":344,"height":321},[323,962,348],{"className":963,"style":327},[326],[227,965,968],{"transform":966,"dataTid":967},"translate(139.348,7.634)","pOY\u002FTJ2PWdvMQOHzZQGhdAo",[227,969,972,974,976,978,980,982,984,986,988,990,992,994,997,999,1002,1005,1008],{"dataTid":970,"className":971,"fill":264,"transform":265},"gOY\u002FTJ2PWdvMQOHzZQGhdAo",[263],[267,973],{"x":269,"y":269,"href":371},[267,975],{"x":362,"y":269,"href":396},[267,977],{"x":861,"y":269,"href":363},[267,979],{"x":370,"y":269,"href":396},[267,981],{"x":374,"y":269,"href":381},[267,983],{"x":377,"y":269,"href":385},[267,985],{"x":872,"y":269,"href":363},[267,987],{"x":384,"y":269,"href":359},[267,989],{"x":388,"y":269,"href":490},[267,991],{"x":392,"y":269,"href":396},[267,993],{"x":882,"y":269,"href":294},[267,995],{"x":996,"y":269,"href":298},"4888",[267,998],{"x":311,"y":269,"href":302},[267,1000],{"x":1001,"y":269,"href":290},"5972",[267,1003],{"x":1004,"y":269,"href":274},"6361",[267,1006],{"x":1007,"y":269,"href":312},"6805",[227,1009,1010],{"transform":315},[317,1011,1013],{"x":269,"y":319,"width":1012,"height":321},"449.94",[323,1014,1016],{"className":1015,"style":327},[326],"18,874,368 bytes",[227,1018,1019],{"transform":242,"dataTid":452},[227,1020],{"dataTid":455,"className":1021,"dataHint":458},[457],[227,1023,1026],{"transform":1024,"dataTid":1025},"translate(0.000,17.039)","pzBxM1IRPXQorJi260yVNqc",[227,1027,1030,1057,1072,1095,1110,1125,1139,1183],{"dataTid":1028,"className":1029},"gzBxM1IRPXQorJi260yVNqc",[239],[227,1031,1034],{"transform":1032,"dataTid":1033},"translate(42.169,7.634)","prMV8QhYP98Qrld7J7Tz0rg",[227,1035,1038,1040,1042,1044,1046,1048],{"dataTid":1036,"className":1037,"fill":264,"transform":265},"grMV8QhYP98Qrld7J7Tz0rg",[263],[267,1039],{"x":269,"y":269,"href":359},[267,1041],{"x":362,"y":269,"href":363},[267,1043],{"x":366,"y":269,"href":867},[267,1045],{"x":370,"y":269,"href":381},[267,1047],{"x":374,"y":269,"href":367},[227,1049,1050],{"transform":315},[317,1051,1053],{"x":269,"y":319,"width":1052,"height":321},"142.38",[323,1054,1056],{"className":1055,"style":327},[326],"3,072",[227,1058,1060],{"transform":1059,"dataTid":415},"translate(69.672,7.634)",[227,1061,1063,1065],{"dataTid":418,"className":1062,"fill":264,"transform":265},[263],[267,1064],{"x":269,"y":269,"href":422},[227,1066,1067],{"transform":315},[317,1068,1069],{"x":269,"y":319,"width":344,"height":321},[323,1070,430],{"className":1071,"style":327},[326],[227,1073,1075],{"transform":1074,"dataTid":1033},"translate(80.674,7.634)",[227,1076,1078,1080,1082,1084,1086,1088],{"dataTid":1036,"className":1077,"fill":264,"transform":265},[263],[267,1079],{"x":269,"y":269,"href":359},[267,1081],{"x":362,"y":269,"href":363},[267,1083],{"x":366,"y":269,"href":867},[267,1085],{"x":370,"y":269,"href":381},[267,1087],{"x":374,"y":269,"href":367},[227,1089,1090],{"transform":315},[317,1091,1092],{"x":269,"y":319,"width":1052,"height":321},[323,1093,1056],{"className":1094,"style":327},[326],[227,1096,1098],{"transform":1097,"dataTid":415},"translate(108.176,7.634)",[227,1099,1101,1103],{"dataTid":418,"className":1100,"fill":264,"transform":265},[263],[267,1102],{"x":269,"y":269,"href":422},[227,1104,1105],{"transform":315},[317,1106,1107],{"x":269,"y":319,"width":344,"height":321},[323,1108,430],{"className":1109,"style":327},[326],[227,1111,1113],{"transform":1112,"dataTid":434},"translate(119.179,7.634)",[227,1114,1116,1118],{"dataTid":437,"className":1115,"fill":264,"transform":265},[263],[267,1117],{"x":269,"y":269,"href":367},[227,1119,1120],{"transform":315},[317,1121,1122],{"x":269,"y":319,"width":445,"height":321},[323,1123,449],{"className":1124,"style":327},[326],[227,1126,1127],{"transform":951,"dataTid":332},[227,1128,1130,1132],{"dataTid":335,"className":1129,"fill":264,"transform":265},[263],[267,1131],{"x":269,"y":269,"href":339},[227,1133,1134],{"transform":315},[317,1135,1136],{"x":269,"y":319,"width":344,"height":321},[323,1137,348],{"className":1138,"style":327},[326],[227,1140,1141],{"transform":966,"dataTid":967},[227,1142,1144,1146,1148,1150,1152,1154,1156,1158,1160,1162,1164,1166,1168,1170,1172,1174,1176],{"dataTid":970,"className":1143,"fill":264,"transform":265},[263],[267,1145],{"x":269,"y":269,"href":371},[267,1147],{"x":362,"y":269,"href":396},[267,1149],{"x":861,"y":269,"href":363},[267,1151],{"x":370,"y":269,"href":396},[267,1153],{"x":374,"y":269,"href":381},[267,1155],{"x":377,"y":269,"href":385},[267,1157],{"x":872,"y":269,"href":363},[267,1159],{"x":384,"y":269,"href":359},[267,1161],{"x":388,"y":269,"href":490},[267,1163],{"x":392,"y":269,"href":396},[267,1165],{"x":882,"y":269,"href":294},[267,1167],{"x":996,"y":269,"href":298},[267,1169],{"x":311,"y":269,"href":302},[267,1171],{"x":1001,"y":269,"href":290},[267,1173],{"x":1004,"y":269,"href":274},[267,1175],{"x":1007,"y":269,"href":312},[227,1177,1178],{"transform":315},[317,1179,1180],{"x":269,"y":319,"width":1012,"height":321},[323,1181,1016],{"className":1182,"style":327},[326],[227,1184,1185],{"transform":242,"dataTid":452},[227,1186],{"dataTid":455,"className":1187,"dataHint":458},[457],[227,1189,1190],{"transform":242,"dataTid":452},[227,1191],{"dataTid":455,"className":1192,"dataHint":458},[457],[11,1194,1195],{},"Both calculations give the same result. The key projection occupies 6,291,456 bytes, which also matches its shape and BF16 representation.",[11,1197,1198],{},"To locate a tensor in the actual file, we add its offset to the beginning of the data section. The first input normalization tensor, for example, begins at data offset 788,004,864.",[90,1200,1207,1209,1304,1307,1309],{"style":92,"className":1201,"viewBox":1202,"width":1203,"height":1204,"dataWidth":1205,"dataHeight":1206,"xmlns":100,"xmlnsXLink":101,"xmlns:h5":102},[94],"0 0 195.000 44.000","17.727272727272727em","4em","195.000","44.000",[104,1208,107],{"type":106},[109,1210,1212,1217,1220,1223,1226,1229,1232,1235,1238,1241,1244,1247,1252,1257,1262,1265,1270,1273,1278,1283,1288,1293,1298,1301],{"className":1211},[112],[114,1213],{"id":1214,"className":1215,"d":1216},"gl3Hi1Qw",[118],"M 698 274 L 413 274 L 413 559 C 413 575 405 583 389 583 C 373 583 365 575 365 559 L 365 274 L 80 274 C 64 274 56 266 56 250 C 56 234 64 226 80 226 L 365 226 L 365 -59 C 365 -75 373 -83 389 -83 C 405 -83 413 -75 413 -59 L 413 226 L 698 226 C 714 226 722 234 722 250 C 722 263 711 274 698 274 Z ",[114,1218],{"id":116,"className":1219,"d":119},[118],[114,1221],{"id":764,"className":1222,"d":766},[118],[114,1224],{"id":122,"className":1225,"d":124},[118],[114,1227],{"id":127,"className":1228,"d":129},[118],[114,1230],{"id":137,"className":1231,"d":139},[118],[114,1233],{"id":142,"className":1234,"d":144},[118],[114,1236],{"id":147,"className":1237,"d":149},[118],[114,1239],{"id":152,"className":1240,"d":154},[118],[114,1242],{"id":157,"className":1243,"d":159},[118],[114,1245],{"id":167,"className":1246,"d":169},[118],[114,1248],{"id":1249,"className":1250,"d":1251},"gl3Hi1Sc",[118],"M 275 641 L 393 641 C 464 641 510 626 533 597 C 565 558 569 521 578 450 L 611 450 L 582 680 L 33 680 L 33 641 L 61 641 C 96 641 117 639 124 634 C 131 629 135 617 135 599 L 135 81 C 135 63 131 51 124 46 C 117 41 96 39 61 39 L 33 39 L 33 0 C 59 2 110 3 187 3 C 274 3 330 2 357 0 L 357 39 L 320 39 C 267 39 238 44 233 55 C 231 60 230 69 230 82 L 230 321 L 311 321 C 354 321 382 313 394 298 C 406 283 412 253 412 206 L 444 206 L 444 475 L 412 475 C 412 429 406 398 394 383 C 382 368 354 360 311 360 L 230 360 L 230 606 C 230 642 236 641 275 641 Z ",[114,1253],{"id":1254,"className":1255,"d":1256},"gl3Hi1UI",[118],"M 483 91 L 483 150 L 451 150 L 451 91 C 451 52 440 32 419 32 C 399 32 387 57 387 77 L 387 274 C 387 309 383 335 376 353 C 352 412 283 448 213 448 C 136 448 60 405 60 333 C 60 300 77 283 110 283 C 143 283 159 299 159 332 C 159 361 144 378 113 381 C 135 406 168 419 211 419 C 273 419 311 362 311 297 L 311 264 C 233 259 174 247 133 228 C 66 197 32 154 32 97 C 32 71 42 49 61 32 C 93 3 137 -11 193 -11 C 252 -11 294 14 320 65 C 327 27 355 -6 398 -6 C 451 -6 483 36 483 91 Z M 200 18 C 154 18 116 52 116 98 C 116 191 213 231 311 236 L 311 141 C 311 74 267 18 200 18 Z ",[114,1258],{"id":1259,"className":1260,"d":1261},"gl3Hi1UU",[118],"M 374 -11 L 527 0 L 527 38 C 491 38 470 40 462 46 C 454 52 450 67 450 90 L 450 694 L 301 683 L 301 645 C 336 645 358 642 366 636 C 374 630 377 616 377 593 L 377 390 C 345 427 305 445 257 445 C 195 445 142 423 99 378 C 56 333 34 279 34 216 C 34 155 54 102 95 57 C 136 12 186 -11 247 -11 C 298 -11 341 8 374 47 Z M 261 415 C 304 415 339 396 364 358 C 371 347 374 336 374 323 L 374 121 C 374 108 371 97 364 86 C 336 41 298 19 251 19 C 210 19 177 39 151 80 C 133 110 124 155 124 215 C 124 324 161 415 261 415 Z ",[114,1263],{"id":182,"className":1264,"d":184},[118],[114,1266],{"id":1267,"className":1268,"d":1269},"gl3Hi1Uc",[118],"M 265 705 C 176 705 110 633 110 544 L 110 434 L 33 434 L 33 396 L 110 396 L 110 79 C 110 60 107 48 100 44 C 93 40 71 39 34 39 L 34 0 L 149 3 C 191 4 234 3 277 0 L 277 39 L 253 39 C 220 39 199 41 192 46 C 185 51 182 63 182 81 L 182 396 L 294 396 L 294 434 L 179 434 L 179 545 C 179 605 210 675 265 675 C 273 675 280 674 287 672 C 272 664 264 650 264 631 C 264 600 279 584 310 584 C 341 584 357 600 357 632 C 357 677 310 705 265 705 Z ",[114,1271],{"id":197,"className":1272,"d":199},[118],[114,1274],{"id":1275,"className":1276,"d":1277},"gl3Hi1U0",[118],"M 144 3 L 255 0 L 255 39 C 219 39 197 40 190 44 C 183 48 180 60 180 79 L 180 694 L 33 683 L 33 645 C 68 645 90 642 97 636 C 104 630 108 616 108 593 L 108 79 C 108 60 105 48 98 44 C 91 40 69 39 33 39 L 33 0 Z ",[114,1279],{"id":1280,"className":1281,"d":1282},"gl3Hi1U8",[118],"M 314 413 C 359 413 382 378 382 307 L 382 79 C 382 60 379 48 372 44 C 365 40 343 38 306 38 L 306 0 L 421 3 L 535 0 L 535 38 C 503 38 482 39 472 42 C 462 45 458 52 458 64 L 458 251 C 458 298 457 331 454 350 C 444 411 399 442 320 442 C 257 442 210 412 179 352 L 179 442 L 32 431 L 32 393 C 68 393 90 390 98 384 C 106 378 109 365 109 342 L 109 79 C 109 60 105 48 98 44 C 91 40 69 38 32 38 L 32 0 L 147 3 L 261 0 L 261 38 C 224 38 203 40 196 44 C 189 48 185 60 185 79 L 185 259 C 185 340 236 413 314 413 Z ",[114,1284],{"id":1285,"className":1286,"d":1287},"gl3Hi1VA",[118],"M 249 -11 C 311 -11 363 11 406 55 C 449 99 471 152 471 214 C 471 277 450 332 408 378 C 366 424 313 448 250 448 C 187 448 135 424 92 378 C 49 332 28 277 28 214 C 28 152 49 99 92 55 C 135 11 188 -11 249 -11 Z M 250 21 C 202 21 166 42 141 85 C 125 113 117 159 117 222 C 117 283 125 327 140 355 C 164 398 200 419 249 419 C 296 419 332 398 357 357 C 374 329 382 284 382 222 C 382 106 348 21 250 21 Z ",[114,1289],{"id":1290,"className":1291,"d":1292},"gl3Hi1VE",[118],"M 298 -13 C 360 -13 413 9 456 54 C 499 99 521 152 521 215 C 521 276 501 328 462 372 C 421 419 371 442 311 442 C 260 442 216 422 178 383 L 178 442 L 28 431 L 28 393 C 65 393 87 391 94 386 C 101 381 105 368 105 347 L 105 -115 C 105 -134 101 -146 94 -150 C 87 -154 65 -155 28 -155 L 28 -194 L 142 -191 L 257 -194 L 257 -155 C 220 -155 199 -154 192 -150 C 185 -146 181 -134 181 -115 L 181 44 C 203 13 246 -13 298 -13 Z M 304 409 C 345 409 377 386 402 340 C 421 301 431 260 431 215 C 431 168 420 125 398 86 C 372 39 337 16 294 16 C 252 16 219 36 194 75 C 185 88 181 101 181 114 L 181 333 C 210 384 251 409 304 409 Z ",[114,1294],{"id":1295,"className":1296,"d":1297},"gl3Hi1VM",[118],"M 364 378 C 364 417 327 442 288 442 C 237 442 198 412 172 351 L 172 442 L 28 431 L 28 393 C 63 393 84 390 92 384 C 100 378 104 365 104 342 L 104 79 C 104 60 101 48 94 44 C 87 40 65 38 28 38 L 28 0 L 143 3 C 185 4 228 3 271 0 L 271 38 L 247 38 C 214 38 193 41 186 46 C 179 51 176 63 176 81 L 176 232 C 176 275 184 314 199 347 C 219 390 248 412 287 413 C 277 403 272 391 272 377 C 272 346 287 331 318 331 C 345 331 364 352 364 378 Z ",[114,1299],{"id":202,"className":1300,"d":204},[118],[114,1302],{"id":207,"className":1303,"d":209},[118],[109,1305],{"className":1306},[223],[104,1308],{"type":106},[227,1310,1315],{"className":1311,"transform":231,"dataTid":1312,"dataPageWidth":1313,"dataPageHeight":1314},[230],"pwUp8Vpl2nTEAIgldoSnCVE","195","44",[227,1316,1319],{"dataTid":1317,"className":1318},"gwUp8Vpl2nTEAIgldoSnCVE",[239],[227,1320,1322],{"transform":242,"dataTid":1321},"pd+Ox3kCHpcQFbsBr5mMjSc",[227,1323,1326,1537,1646,1712],{"dataTid":1324,"className":1325},"gd+Ox3kCHpcQFbsBr5mMjSc",[239],[227,1327,1329],{"transform":242,"dataTid":1328},"pqOohgzXa0Rk5Kf4VlZTDgs",[227,1330,1333,1393,1408,1457,1476,1532],{"dataTid":1331,"className":1332},"gqOohgzXa0Rk5Kf4VlZTDgs",[239],[227,1334,1337],{"transform":1335,"dataTid":1336},"translate(0.000,7.755)","pn1u\u002FmiWUvl0wJe0W7qWYns",[227,1338,1341,1344,1347,1351,1354,1357,1361,1365,1368,1371,1374,1377,1380,1384],{"dataTid":1339,"className":1340,"fill":264,"transform":265},"gn1u\u002FmiWUvl0wJe0W7qWYns",[263],[267,1342],{"x":269,"y":269,"href":1343},"#gl3Hi1Sc",[267,1345],{"x":1346,"y":269,"href":278},"653",[267,1348],{"x":1349,"y":269,"href":1350},"931","#gl3Hi1U0",[267,1352],{"x":1353,"y":269,"href":274},"1209",[267,1355],{"x":1356,"y":269,"href":294},"1653",[267,1358],{"x":1359,"y":269,"href":1360},"1985","#gl3Hi1VE",[267,1362],{"x":1363,"y":269,"href":1364},"2541","#gl3Hi1VA",[267,1366],{"x":1367,"y":269,"href":312},"3041",[267,1369],{"x":1370,"y":269,"href":278},"3435",[267,1372],{"x":1373,"y":269,"href":290},"3713",[267,1375],{"x":1376,"y":269,"href":278},"4102",[267,1378],{"x":1379,"y":269,"href":1364},"4380",[267,1381],{"x":1382,"y":269,"href":1383},"4880","#gl3Hi1U8",[227,1385,1386],{"transform":315},[317,1387,1389],{"x":269,"y":319,"width":1388,"height":321},"339.75",[323,1390,1392],{"className":1391,"style":327},[326],"File position",[227,1394,1396],{"transform":1395,"dataTid":332},"translate(62.852,7.755)",[227,1397,1399,1401],{"dataTid":335,"className":1398,"fill":264,"transform":265},[263],[267,1400],{"x":269,"y":269,"href":339},[227,1402,1403],{"transform":315},[317,1404,1405],{"x":269,"y":319,"width":344,"height":321},[323,1406,348],{"className":1407,"style":327},[326],[227,1409,1412],{"transform":1410,"dataTid":1411},"translate(74.465,7.755)","pu4OYQKP\u002FNdch4pBKX9H3dc",[227,1413,1416,1419,1423,1426,1429,1432,1435,1438,1441,1445,1448],{"dataTid":1414,"className":1415,"fill":264,"transform":265},"gu4OYQKP\u002FNdch4pBKX9H3dc",[263],[267,1417],{"x":269,"y":269,"href":1418},"#gl3Hi1UU",[267,1420],{"x":1421,"y":269,"href":1422},"556","#gl3Hi1UI",[267,1424],{"x":1425,"y":269,"href":290},"1056",[267,1427],{"x":1428,"y":269,"href":1422},"1445",[267,1430],{"x":1431,"y":269,"href":294},"1945",[267,1433],{"x":1434,"y":269,"href":312},"2277",[267,1436],{"x":1437,"y":269,"href":290},"2671",[267,1439],{"x":1440,"y":269,"href":1422},"3060",[267,1442],{"x":1443,"y":269,"href":1444},"3560","#gl3Hi1VM",[267,1446],{"x":1447,"y":269,"href":290},"3952",[227,1449,1450],{"transform":315},[317,1451,1453],{"x":269,"y":319,"width":1452,"height":321},"271.31",[323,1454,1456],{"className":1455,"style":327},[326],"data start",[227,1458,1461],{"transform":1459,"dataTid":1460},"translate(124.661,7.755)","pzUvlOB5pueI9GCCk24jGCA",[227,1462,1465,1468],{"dataTid":1463,"className":1464,"fill":264,"transform":265},"gzUvlOB5pueI9GCCk24jGCA",[263],[267,1466],{"x":269,"y":269,"href":1467},"#gl3Hi1Qw",[227,1469,1470],{"transform":315},[317,1471,1472],{"x":269,"y":319,"width":344,"height":321},[323,1473,1475],{"className":1474,"style":327},[326],"+",[227,1477,1480],{"transform":1478,"dataTid":1479},"translate(135.663,7.755)","p5MdGa5aldoEQr5UyRbwU\u002FQ",[227,1481,1484,1486,1489,1492,1495,1498,1501,1504,1507,1511,1514,1517,1520,1523],{"dataTid":1482,"className":1483,"fill":264,"transform":265},"g5MdGa5aldoEQr5UyRbwU\u002FQ",[263],[267,1485],{"x":269,"y":269,"href":290},[267,1487],{"x":1488,"y":269,"href":274},"389",[267,1490],{"x":1491,"y":269,"href":1383},"833",[267,1493],{"x":1494,"y":269,"href":312},"1389",[267,1496],{"x":1497,"y":269,"href":1364},"1783",[267,1499],{"x":1500,"y":269,"href":1444},"2283",[267,1502],{"x":1503,"y":269,"href":294},"2675",[267,1505],{"x":1506,"y":269,"href":1364},"3007",[267,1508],{"x":1509,"y":269,"href":1510},"3507","#gl3Hi1Uc",[267,1512],{"x":1513,"y":269,"href":1510},"3813",[267,1515],{"x":1516,"y":269,"href":312},"4119",[267,1518],{"x":1519,"y":269,"href":274},"4513",[267,1521],{"x":1522,"y":269,"href":290},"4957",[227,1524,1525],{"transform":315},[317,1526,1528],{"x":269,"y":319,"width":1527,"height":321},"334.12",[323,1529,1531],{"className":1530,"style":327},[326],"tensor offset",[227,1533,1534],{"transform":242,"dataTid":452},[227,1535],{"dataTid":455,"className":1536,"dataHint":458},[457],[227,1538,1540],{"transform":1024,"dataTid":1539},"psTi9+RDxh1IaBIjC\u002FcyJpw",[227,1541,1544,1559,1588,1603,1641],{"dataTid":1542,"className":1543},"gsTi9+RDxh1IaBIjC\u002FcyJpw",[239],[227,1545,1547],{"transform":1546,"dataTid":332},"translate(62.852,7.447)",[227,1548,1550,1552],{"dataTid":335,"className":1549,"fill":264,"transform":265},[263],[267,1551],{"x":269,"y":269,"href":339},[227,1553,1554],{"transform":315},[317,1555,1556],{"x":269,"y":319,"width":344,"height":321},[323,1557,348],{"className":1558,"style":327},[326],[227,1560,1563],{"transform":1561,"dataTid":1562},"translate(74.465,7.447)","pYUBVbUK3vPkmceKaLLhEzk",[227,1564,1567,1569,1571,1573,1575,1577,1579],{"dataTid":1565,"className":1566,"fill":264,"transform":265},"gYUBVbUK3vPkmceKaLLhEzk",[263],[267,1568],{"x":269,"y":269,"href":367},[267,1570],{"x":362,"y":269,"href":371},[267,1572],{"x":861,"y":269,"href":363},[267,1574],{"x":370,"y":269,"href":490},[267,1576],{"x":374,"y":269,"href":396},[267,1578],{"x":377,"y":269,"href":396},[227,1580,1581],{"transform":315},[317,1582,1584],{"x":269,"y":319,"width":1583,"height":321},"173.62",[323,1585,1587],{"className":1586,"style":327},[326],"21,688",[227,1589,1591],{"transform":1590,"dataTid":1460},"translate(107.468,7.447)",[227,1592,1594,1596],{"dataTid":1463,"className":1593,"fill":264,"transform":265},[263],[267,1595],{"x":269,"y":269,"href":1467},[227,1597,1598],{"transform":315},[317,1599,1600],{"x":269,"y":319,"width":344,"height":321},[323,1601,1475],{"className":1602,"style":327},[326],[227,1604,1607],{"transform":1605,"dataTid":1606},"translate(118.470,7.447)","p4I13w3V2bzwZpgcflsMq5U",[227,1608,1611,1613,1615,1617,1619,1621,1623,1625,1627,1629,1631,1633],{"dataTid":1609,"className":1610,"fill":264,"transform":265},"g4I13w3V2bzwZpgcflsMq5U",[263],[267,1612],{"x":269,"y":269,"href":381},[267,1614],{"x":362,"y":269,"href":396},[267,1616],{"x":861,"y":269,"href":396},[267,1618],{"x":864,"y":269,"href":363},[267,1620],{"x":374,"y":269,"href":867},[267,1622],{"x":377,"y":269,"href":867},[267,1624],{"x":872,"y":269,"href":385},[267,1626],{"x":875,"y":269,"href":363},[267,1628],{"x":388,"y":269,"href":396},[267,1630],{"x":392,"y":269,"href":490},[267,1632],{"x":882,"y":269,"href":385},[227,1634,1635],{"transform":315},[317,1636,1637],{"x":269,"y":319,"width":887,"height":321},[323,1638,1640],{"className":1639,"style":327},[326],"788,004,864",[227,1642,1643],{"transform":242,"dataTid":452},[227,1644],{"dataTid":455,"className":1645,"dataHint":458},[457],[227,1647,1650],{"transform":1648,"dataTid":1649},"translate(0.000,33.759)","pBSmShSZt3UIZShfABghcqA",[227,1651,1654,1669,1707],{"dataTid":1652,"className":1653},"gBSmShSZt3UIZShfABghcqA",[239],[227,1655,1657],{"transform":1656,"dataTid":332},"translate(62.852,7.436)",[227,1658,1660,1662],{"dataTid":335,"className":1659,"fill":264,"transform":265},[263],[267,1661],{"x":269,"y":269,"href":339},[227,1663,1664],{"transform":315},[317,1665,1666],{"x":269,"y":319,"width":344,"height":321},[323,1667,348],{"className":1668,"style":327},[326],[227,1670,1673],{"transform":1671,"dataTid":1672},"translate(74.465,7.436)","pKK\u002FyAljTQEg4856tvfoUEw",[227,1674,1677,1679,1681,1683,1685,1687,1689,1691,1693,1695,1697,1699],{"dataTid":1675,"className":1676,"fill":264,"transform":265},"gKK\u002FyAljTQEg4856tvfoUEw",[263],[267,1678],{"x":269,"y":269,"href":381},[267,1680],{"x":362,"y":269,"href":396},[267,1682],{"x":861,"y":269,"href":396},[267,1684],{"x":864,"y":269,"href":363},[267,1686],{"x":374,"y":269,"href":867},[267,1688],{"x":377,"y":269,"href":367},[267,1690],{"x":872,"y":269,"href":490},[267,1692],{"x":875,"y":269,"href":363},[267,1694],{"x":388,"y":269,"href":499},[267,1696],{"x":392,"y":269,"href":499},[267,1698],{"x":882,"y":269,"href":367},[227,1700,1701],{"transform":315},[317,1702,1703],{"x":269,"y":319,"width":887,"height":321},[323,1704,1706],{"className":1705,"style":327},[326],"788,026,552",[227,1708,1709],{"transform":242,"dataTid":452},[227,1710],{"dataTid":455,"className":1711,"dataHint":458},[457],[227,1713,1714],{"transform":242,"dataTid":452},[227,1715],{"dataTid":455,"className":1716,"dataHint":458},[457],[11,1718,1719],{},"This is the absolute byte position where the tensor begins in the first shard.",[11,1721,1722],{},[82,1723],{"alt":1724,"src":1725},"Safetensors header and tensor offsets in the first shard","\u002Fassets\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002Fsafetensors-layout.svg",[64,1727,1729],{"id":1728},"inspecting-the-header-without-loading-the-weights","Inspecting the header without loading the weights",[11,1731,1732],{},"Since the header is stored at the beginning of the file, we can read it independently of the tensor data. The following script also checks that each BF16 tensor occupies the expected number of bytes.",[29,1734,1738],{"className":1735,"code":1736,"language":1737,"meta":35,"style":35},"language-python shiki shiki-themes github-dark","import json\nimport math\nimport struct\nfrom pathlib import Path\n\npath = Path(\"model-00001-of-00002.safetensors\")\n\nwith path.open(\"rb\") as f:\n    header_size = struct.unpack(\"\u003CQ\", f.read(8))[0]\n    header = json.loads(f.read(header_size))\n\ndata_start = 8 + header_size\nfile_size = path.stat().st_size\n\nfor name, tensor in header.items():\n    if name == \"__metadata__\":\n        continue\n\n    assert tensor[\"dtype\"] == \"BF16\"\n\n    start, end = tensor[\"data_offsets\"]\n    expected_size = math.prod(tensor[\"shape\"]) * 2\n    stored_size = end - start\n\n    assert stored_size == expected_size\n    assert 0 \u003C= start \u003C= end\n    assert data_start + end \u003C= file_size\n\n    print(name, tensor[\"shape\"], stored_size)\n","python",[37,1739,1740,1749,1756,1763,1776,1782,1798,1802,1822,1848,1858,1862,1878,1889,1894,1909,1927,1933,1938,1958,1963,1978,2001,2018,2023,2036,2056,2073,2078],{"__ignoreMap":35},[599,1741,1742,1746],{"class":601,"line":602},[599,1743,1745],{"class":1744},"snl16","import",[599,1747,1748],{"class":605}," json\n",[599,1750,1751,1753],{"class":601,"line":609},[599,1752,1745],{"class":1744},[599,1754,1755],{"class":605}," math\n",[599,1757,1758,1760],{"class":601,"line":619},[599,1759,1745],{"class":1744},[599,1761,1762],{"class":605}," struct\n",[599,1764,1765,1768,1771,1773],{"class":601,"line":635},[599,1766,1767],{"class":1744},"from",[599,1769,1770],{"class":605}," pathlib ",[599,1772,1745],{"class":1744},[599,1774,1775],{"class":605}," Path\n",[599,1777,1778],{"class":601,"line":655},[599,1779,1781],{"emptyLinePlaceholder":1780},true,"\n",[599,1783,1784,1787,1789,1792,1795],{"class":601,"line":674},[599,1785,1786],{"class":605},"path ",[599,1788,348],{"class":1744},[599,1790,1791],{"class":605}," Path(",[599,1793,1794],{"class":628},"\"model-00001-of-00002.safetensors\"",[599,1796,1797],{"class":605},")\n",[599,1799,1800],{"class":601,"line":680},[599,1801,1781],{"emptyLinePlaceholder":1780},[599,1803,1804,1807,1810,1813,1816,1819],{"class":601,"line":688},[599,1805,1806],{"class":1744},"with",[599,1808,1809],{"class":605}," path.open(",[599,1811,1812],{"class":628},"\"rb\"",[599,1814,1815],{"class":605},") ",[599,1817,1818],{"class":1744},"as",[599,1820,1821],{"class":605}," f:\n",[599,1823,1824,1827,1829,1832,1835,1838,1841,1844,1846],{"class":601,"line":699},[599,1825,1826],{"class":605},"    header_size ",[599,1828,348],{"class":1744},[599,1830,1831],{"class":605}," struct.unpack(",[599,1833,1834],{"class":628},"\"\u003CQ\"",[599,1836,1837],{"class":605},", f.read(",[599,1839,1840],{"class":612},"8",[599,1842,1843],{"class":605},"))[",[599,1845,269],{"class":612},[599,1847,671],{"class":605},[599,1849,1850,1853,1855],{"class":601,"line":715},[599,1851,1852],{"class":605},"    header ",[599,1854,348],{"class":1744},[599,1856,1857],{"class":605}," json.loads(f.read(header_size))\n",[599,1859,1860],{"class":601,"line":732},[599,1861,1781],{"emptyLinePlaceholder":1780},[599,1863,1864,1867,1869,1872,1875],{"class":601,"line":738},[599,1865,1866],{"class":605},"data_start ",[599,1868,348],{"class":1744},[599,1870,1871],{"class":612}," 8",[599,1873,1874],{"class":1744}," +",[599,1876,1877],{"class":605}," header_size\n",[599,1879,1881,1884,1886],{"class":601,"line":1880},13,[599,1882,1883],{"class":605},"file_size ",[599,1885,348],{"class":1744},[599,1887,1888],{"class":605}," path.stat().st_size\n",[599,1890,1892],{"class":601,"line":1891},14,[599,1893,1781],{"emptyLinePlaceholder":1780},[599,1895,1897,1900,1903,1906],{"class":601,"line":1896},15,[599,1898,1899],{"class":1744},"for",[599,1901,1902],{"class":605}," name, tensor ",[599,1904,1905],{"class":1744},"in",[599,1907,1908],{"class":605}," header.items():\n",[599,1910,1912,1915,1918,1921,1924],{"class":601,"line":1911},16,[599,1913,1914],{"class":1744},"    if",[599,1916,1917],{"class":605}," name ",[599,1919,1920],{"class":1744},"==",[599,1922,1923],{"class":628}," \"__metadata__\"",[599,1925,1926],{"class":605},":\n",[599,1928,1930],{"class":601,"line":1929},17,[599,1931,1932],{"class":1744},"        continue\n",[599,1934,1936],{"class":601,"line":1935},18,[599,1937,1781],{"emptyLinePlaceholder":1780},[599,1939,1941,1944,1947,1950,1953,1955],{"class":601,"line":1940},19,[599,1942,1943],{"class":1744},"    assert",[599,1945,1946],{"class":605}," tensor[",[599,1948,1949],{"class":628},"\"dtype\"",[599,1951,1952],{"class":605},"] ",[599,1954,1920],{"class":1744},[599,1956,1957],{"class":628}," \"BF16\"\n",[599,1959,1961],{"class":601,"line":1960},20,[599,1962,1781],{"emptyLinePlaceholder":1780},[599,1964,1966,1969,1971,1973,1976],{"class":601,"line":1965},21,[599,1967,1968],{"class":605},"    start, end ",[599,1970,348],{"class":1744},[599,1972,1946],{"class":605},[599,1974,1975],{"class":628},"\"data_offsets\"",[599,1977,671],{"class":605},[599,1979,1981,1984,1986,1989,1992,1995,1998],{"class":601,"line":1980},22,[599,1982,1983],{"class":605},"    expected_size ",[599,1985,348],{"class":1744},[599,1987,1988],{"class":605}," math.prod(tensor[",[599,1990,1991],{"class":628},"\"shape\"",[599,1993,1994],{"class":605},"]) ",[599,1996,1997],{"class":1744},"*",[599,1999,2000],{"class":612}," 2\n",[599,2002,2004,2007,2009,2012,2015],{"class":601,"line":2003},23,[599,2005,2006],{"class":605},"    stored_size ",[599,2008,348],{"class":1744},[599,2010,2011],{"class":605}," end ",[599,2013,2014],{"class":1744},"-",[599,2016,2017],{"class":605}," start\n",[599,2019,2021],{"class":601,"line":2020},24,[599,2022,1781],{"emptyLinePlaceholder":1780},[599,2024,2026,2028,2031,2033],{"class":601,"line":2025},25,[599,2027,1943],{"class":1744},[599,2029,2030],{"class":605}," stored_size ",[599,2032,1920],{"class":1744},[599,2034,2035],{"class":605}," expected_size\n",[599,2037,2039,2041,2044,2047,2050,2053],{"class":601,"line":2038},26,[599,2040,1943],{"class":1744},[599,2042,2043],{"class":612}," 0",[599,2045,2046],{"class":1744}," \u003C=",[599,2048,2049],{"class":605}," start ",[599,2051,2052],{"class":1744},"\u003C=",[599,2054,2055],{"class":605}," end\n",[599,2057,2059,2061,2064,2066,2068,2070],{"class":601,"line":2058},27,[599,2060,1943],{"class":1744},[599,2062,2063],{"class":605}," data_start ",[599,2065,1475],{"class":1744},[599,2067,2011],{"class":605},[599,2069,2052],{"class":1744},[599,2071,2072],{"class":605}," file_size\n",[599,2074,2076],{"class":601,"line":2075},28,[599,2077,1781],{"emptyLinePlaceholder":1780},[599,2079,2081,2084,2087,2089],{"class":601,"line":2080},29,[599,2082,2083],{"class":612},"    print",[599,2085,2086],{"class":605},"(name, tensor[",[599,2088,1991],{"class":628},[599,2090,2091],{"class":605},"], stored_size)\n",[11,2093,2094],{},"This checks the byte ranges described by the header without allocating memory for the complete model. It is a targeted inspection script, not a replacement for the validation performed by the Safetensors library. The header length should also be bounded and validated before reading untrusted files.",[11,2096,2097,2098,2101],{},"The format provides enough information to locate and interpret the tensors, but it does not describe the calculations performed by the network. In the two shards we inspected, the optional metadata contains only ",[37,2099,2100],{},"{\"format\": \"pt\"}",". The architecture still comes from the separate configuration file.",[11,2103,2104],{},"GGUF takes a different approach by including much more model information in the binary file itself.",[21,2106,2108],{"id":2107},"inside-the-gguf-file","Inside the GGUF file",[11,2110,2111],{},"GGUF is the model format used by llama.cpp and tools built around the GGML ecosystem. It can store the architecture settings, tokenizer, chat template and tensors in one file.",[11,2113,2114],{},"Its binary structure contains four main parts: a fixed header, key-value metadata, tensor descriptors and the tensor data section.",[11,2116,2117,2118,2121],{},"The fixed header starts with the ASCII identifier ",[37,2119,2120],{},"GGUF",", followed by the format version and the counts of tensors and metadata entries. The metadata uses typed key-value fields, while the tensor descriptors provide the names, dimensions, types and data offsets.",[11,2123,2124],{},"The numerical data is stored after these sections, with the data section aligned according to the file's alignment requirements.",[11,2126,2127,2128,2131],{},"We converted the original Llama checkpoint to GGUF without changing the precision of the main weight matrices. The result was a ",[72,2129,2130],{},"6,433,688,160-byte BF16 GGUF file",".",[11,2133,2134],{},"The converted file was larger than the two original Safetensors shards. This compares a GGUF file containing weights and tokenizer metadata with the source weight files alone; it excludes the source's separate tokenizer and configuration files. GGUF does not automatically make a model smaller; it is a different container with its own metadata and representation rules.",[11,2136,2137,2138,2141],{},"Our file uses GGUF version 3 and contains 37 metadata entries. The architecture field is ",[37,2139,2140],{},"llama",", and the tokenizer metadata includes 128,256 vocabulary entries, 280,147 merge rules and the chat template.",[11,2143,2144],{},"The tokenizer metadata accounts for 7,821,550 bytes, making it the largest part of the information stored before the actual weights.",[11,2146,2147],{},"The file layout can be measured precisely:",[2149,2150,2151,2165],"table",{},[2152,2153,2154],"thead",{},[2155,2156,2157,2161],"tr",{},[2158,2159,2160],"th",{},"GGUF component",[2158,2162,2164],{"align":2163},"right","Position or size",[2166,2167,2168,2177,2185,2193,2201],"tbody",{},[2155,2169,2170,2174],{},[2171,2172,2173],"td",{},"Metadata ends",[2171,2175,2176],{"align":2163},"Byte 7,822,906",[2155,2178,2179,2182],{},[2171,2180,2181],{},"Tensor descriptor table",[2171,2183,2184],{"align":2163},"15,130 bytes",[2155,2186,2187,2190],{},[2171,2188,2189],{},"Alignment padding",[2171,2191,2192],{"align":2163},"12 bytes",[2155,2194,2195,2198],{},[2171,2196,2197],{},"Tensor data starts",[2171,2199,2200],{"align":2163},"Byte 7,838,048",[2155,2202,2203,2206],{},[2171,2204,2205],{},"File size",[2171,2207,2208],{"align":2163},"6,433,688,160 bytes",[11,2210,2211],{},"The tensor data section starts at a 32-byte-aligned position. As with Safetensors, the offsets inside the tensor descriptors are relative to the beginning of that data section.",[11,2213,2214],{},[82,2215],{"alt":2216,"src":2217},"GGUF metadata, tensor descriptors and aligned data section","\u002Fassets\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002Fgguf-layout.svg",[11,2219,2220],{},"The GGUF file contains 255 tensors, compared with 254 in the original checkpoint. That difference is explained by the conversion process, rather than by a change to the number of learned parameters.",[64,2222,2224],{"id":2223},"inspecting-gguf-metadata-and-tensors","Inspecting GGUF metadata and tensors",[11,2226,2227,2228,2231,2232,2237],{},"The ",[37,2229,2230],{},"gguf"," Python package supplied with llama.cpp can read the architecture metadata and tensor descriptors. With the package installed from the pinned revision in the ",[458,2233,2236],{"href":2234,"external":35,"trailing-slash":2235},"\u002Fassets\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002FREADME.md","remove","reproduction instructions",", run this beside the converted file:",[29,2239,2241],{"className":1735,"code":2240,"language":1737,"meta":35,"style":35},"from gguf import GGUFReader\n\nreader = GGUFReader(\"Llama-3.2-3B-Instruct-BF16.gguf\")\nprint(reader.fields[\"general.architecture\"].contents())\nprint(\"Tensor count:\", len(reader.tensors))\n\nfor tensor in reader.tensors[:5]:\n    print(tensor.name, [int(d) for d in tensor.shape], tensor.tensor_type.name)\n",[37,2242,2243,2255,2259,2274,2288,2306,2310,2328],{"__ignoreMap":35},[599,2244,2245,2247,2250,2252],{"class":601,"line":602},[599,2246,1767],{"class":1744},[599,2248,2249],{"class":605}," gguf ",[599,2251,1745],{"class":1744},[599,2253,2254],{"class":605}," GGUFReader\n",[599,2256,2257],{"class":601,"line":609},[599,2258,1781],{"emptyLinePlaceholder":1780},[599,2260,2261,2264,2266,2269,2272],{"class":601,"line":619},[599,2262,2263],{"class":605},"reader ",[599,2265,348],{"class":1744},[599,2267,2268],{"class":605}," GGUFReader(",[599,2270,2271],{"class":628},"\"Llama-3.2-3B-Instruct-BF16.gguf\"",[599,2273,1797],{"class":605},[599,2275,2276,2279,2282,2285],{"class":601,"line":635},[599,2277,2278],{"class":612},"print",[599,2280,2281],{"class":605},"(reader.fields[",[599,2283,2284],{"class":628},"\"general.architecture\"",[599,2286,2287],{"class":605},"].contents())\n",[599,2289,2290,2292,2295,2298,2300,2303],{"class":601,"line":655},[599,2291,2278],{"class":612},[599,2293,2294],{"class":605},"(",[599,2296,2297],{"class":628},"\"Tensor count:\"",[599,2299,647],{"class":605},[599,2301,2302],{"class":612},"len",[599,2304,2305],{"class":605},"(reader.tensors))\n",[599,2307,2308],{"class":601,"line":674},[599,2309,1781],{"emptyLinePlaceholder":1780},[599,2311,2312,2314,2317,2319,2322,2325],{"class":601,"line":680},[599,2313,1899],{"class":1744},[599,2315,2316],{"class":605}," tensor ",[599,2318,1905],{"class":1744},[599,2320,2321],{"class":605}," reader.tensors[:",[599,2323,2324],{"class":612},"5",[599,2326,2327],{"class":605},"]:\n",[599,2329,2330,2332,2335,2338,2341,2343,2346,2348],{"class":601,"line":688},[599,2331,2083],{"class":612},[599,2333,2334],{"class":605},"(tensor.name, [",[599,2336,2337],{"class":612},"int",[599,2339,2340],{"class":605},"(d) ",[599,2342,1899],{"class":1744},[599,2344,2345],{"class":605}," d ",[599,2347,1905],{"class":1744},[599,2349,2350],{"class":605}," tensor.shape], tensor.tensor_type.name)\n",[11,2352,2353,2354,2356],{},"This prints ",[37,2355,2140],{},", a tensor count of 255, and the names, dimensions and storage types of the first five tensors. We will return to GGUF's dimension convention when comparing the tensors with their Safetensors counterparts.",[64,2358,2360],{"id":2359},"reconciling-the-two-file-sizes","Reconciling the two file sizes",[11,2362,2363],{},"We can account for the difference in size without making assumptions about compression or hidden data.",[11,2365,2366],{},"The original Safetensors checkpoint contains 6,425,499,648 bytes of BF16 tensor data. The GGUF tensor data section contains 6,425,850,112 bytes, which is 350,464 bytes more.",[11,2368,2369],{},"Two operations explain this increase.",[11,2371,2372],{},"First, the converter stores 57 normalization tensors in F32 instead of BF16. Their values are preserved exactly because every BF16 value can be represented in F32, but their storage requirement doubles. This adds 350,208 bytes.",[11,2374,2375,2376,2379],{},"Second, it generates a ",[37,2377,2378],{},"rope_freqs.weight"," tensor containing 64 F32 values. This adds another 256 bytes.",[29,2381,2384],{"className":2382,"code":2383,"language":34,"meta":35},[32],"Normalization type changes   +350,208 bytes\nGenerated RoPE tensor             +256 bytes\n                             ------------\nAdditional tensor data       +350,464 bytes\n",[37,2385,2383],{"__ignoreMap":35},[11,2387,2388],{},"That matches the measured difference between the two tensor data sections exactly.",[11,2390,2391],{},"We can also reconcile the total file sizes:",[2149,2393,2394,2407],{},[2152,2395,2396],{},[2155,2397,2398,2401,2404],{},[2158,2399,2400],{},"Component",[2158,2402,2403],{"align":2163},"Safetensors shards",[2158,2405,2406],{"align":2163},"GGUF BF16",[2166,2408,2409,2420,2431],{},[2155,2410,2411,2414,2417],{},[2171,2412,2413],{},"Tensor data",[2171,2415,2416],{"align":2163},"6,425,499,648",[2171,2418,2419],{"align":2163},"6,425,850,112",[2155,2421,2422,2425,2428],{},[2171,2423,2424],{},"Everything before tensor data",[2171,2426,2427],{"align":2163},"29,400",[2171,2429,2430],{"align":2163},"7,838,048",[2155,2432,2433,2436,2439],{},[2171,2434,2435],{},"Total file size",[2171,2437,2438],{"align":2163},"6,425,529,048",[2171,2440,2441],{"align":2163},"6,433,688,160",[11,2443,2444],{},"The GGUF file is 8,159,112 bytes larger. Most of that difference comes from the metadata stored before the weights, particularly the tokenizer. The rest comes from the F32 normalization tensors and generated RoPE values.",[21,2446,2448],{"id":2447},"verifying-the-safetensors-to-gguf-conversion","Verifying the Safetensors-to-GGUF conversion",[11,2450,2451],{},"A conversion can produce a valid GGUF file without its tensor data being directly comparable to the source checkpoint. Tensor naming, dimension conventions and some internal layouts differ between the two formats.",[11,2453,2454],{},"We therefore treated the comparison as a tensor-level audit rather than a file-level comparison.",[11,2456,2457],{},"The first difference is naming. The first decoder layer includes these mappings:",[2149,2459,2460,2469],{},[2152,2461,2462],{},[2155,2463,2464,2467],{},[2158,2465,2466],{},"Safetensors",[2158,2468,2120],{},[2166,2470,2471,2483,2495],{},[2155,2472,2473,2478],{},[2171,2474,2475],{},[37,2476,2477],{},"model.layers.0.mlp.gate_proj.weight",[2171,2479,2480],{},[37,2481,2482],{},"blk.0.ffn_gate.weight",[2155,2484,2485,2490],{},[2171,2486,2487],{},[37,2488,2489],{},"model.layers.0.self_attn.q_proj.weight",[2171,2491,2492],{},[37,2493,2494],{},"blk.0.attn_q.weight",[2155,2496,2497,2502],{},[2171,2498,2499],{},[37,2500,2501],{},"model.layers.0.self_attn.k_proj.weight",[2171,2503,2504],{},[37,2505,2506],{},"blk.0.attn_k.weight",[11,2508,2509,2510,2513,2514,2517],{},"The second difference concerns dimension ordering. A feed-forward tensor with shape ",[37,2511,2512],{},"[8192, 3072]"," in Safetensors is described as ",[37,2515,2516],{},"[3072, 8192]"," by GGUF reader.",[11,2519,2520],{},"This highlight the different conventions used to express tensor dimensions. By itself, it does not indicate that the numerical matrix was transposed.",[11,2522,2523],{},"Some attention tensors require an additional transformation. During conversion, the query and key projections are row-permuted to match the rotary position embedding layout expected by llama.cpp. A direct comparison of their stored arrays would therefore report differences.",[11,2525,2526],{},[82,2527],{"alt":2528,"src":2529},"Tensor naming, dimension conventions and attention row permutations during conversion","\u002Fassets\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002Fconversion-mapping.svg",[11,2531,2532],{},"Our audit matched source and converted tensors by their roles in the architecture and accounted for the GGUF dimension convention. For the query and key projections, it applied the converter's row permutation to the source tensors before comparing them with the GGUF arrays. The normalization tensors were compared after the exact BF16-to-F32 conversion. The generated RoPE tensor was checked separately against the configuration, since it has no learned counterpart in the source checkpoint.",[11,2534,2535,2536,2131],{},"The result was an exact match for ",[72,2537,2538],{},"all 254 original tensors",[2149,2540,2541,2551],{},[2152,2542,2543],{},[2155,2544,2545,2548],{},[2158,2546,2547],{},"Audit check",[2158,2549,2550],{},"Result",[2166,2552,2553,2561,2569,2576,2583,2591,2599],{},[2155,2554,2555,2558],{},[2171,2556,2557],{},"Original tensors accounted for",[2171,2559,2560],{},"254 \u002F 254",[2155,2562,2563,2566],{},[2171,2564,2565],{},"Tensor names mapped",[2171,2567,2568],{},"Complete",[2155,2570,2571,2574],{},[2171,2572,2573],{},"Dimension conventions handled",[2171,2575,2568],{},[2155,2577,2578,2581],{},[2171,2579,2580],{},"Converter's query\u002Fkey row permutations applied to source",[2171,2582,2568],{},[2155,2584,2585,2588],{},[2171,2586,2587],{},"BF16-to-F32 normalization values",[2171,2589,2590],{},"Exact",[2155,2592,2593,2596],{},[2171,2594,2595],{},"Learned tensor values",[2171,2597,2598],{},"Exact match",[2155,2600,2601,2604],{},[2171,2602,2603],{},"Generated RoPE tensor",[2171,2605,2606],{},"Checked separately",[11,2608,2609],{},"The learned values survived the conversion exactly. Two inference runtimes may still produce different results because they use different kernels, perform operations in a different order or use different generation settings.",[11,2611,2612],{},"The converter version also matters. GGUF metadata and conversion code can change, so checks written for one version may need updating for another.",[11,2614,2615,2616,2624,2625,647,2629,2633,2634,2636],{},"Our audit uses the pinned llama.cpp revision ",[458,2617,2621],{"href":2618,"rel":2619},"https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Ftree\u002F620d2225cc436ab0b001e3200e590e3b286ef9ab",[2620],"nofollow",[37,2622,2623],{},"620d2225cc436ab0b001e3200e590e3b286ef9ab",", with the ",[458,2626,2628],{"href":2627,"external":35,"trailing-slash":2235},"\u002Fassets\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002Finspection.json","inspection report",[458,2630,2632],{"href":2631,"external":35,"trailing-slash":2235},"\u002Fassets\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002Fconversion-checks.json","conversion checks"," and ",[458,2635,2236],{"href":2234,"external":35,"trailing-slash":2235}," available alongside the article.",[104,2638,2639],{},"html pre.shiki code .s95oV, html code.shiki .s95oV{--shiki-default:#E1E4E8}html pre.shiki code .sDLfK, html code.shiki .sDLfK{--shiki-default:#79B8FF}html pre.shiki code .sU2Wk, html code.shiki .sU2Wk{--shiki-default:#9ECBFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .snl16, html code.shiki .snl16{--shiki-default:#F97583}",{"title":35,"searchDepth":609,"depth":609,"links":2641},[2642,2645,2648,2652],{"id":23,"depth":609,"text":24,"children":2643},[2644],{"id":66,"depth":619,"text":67},{"id":580,"depth":609,"text":581,"children":2646},[2647],{"id":1728,"depth":619,"text":1729},{"id":2107,"depth":609,"text":2108,"children":2649},[2650,2651],{"id":2223,"depth":619,"text":2224},{"id":2359,"depth":619,"text":2360},{"id":2447,"depth":609,"text":2448},"\u002Fassets\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf\u002Fcover.png","2026-09-20","Inspect Llama 3.2 model files, understand how Safetensors and GGUF store tensors, and verify that a BF16 conversion preserves the learned weights.","md",false,"concept","en",{},"2026-10-27","\u002Fblog\u002Finside-llm-model-files-understanding-safetensors-and-gguf","10 min read",{"title":5,"description":2655},null,{"loc":2662,"images":2667},[2668,2669,2670,2671],{"loc":85},{"loc":1725},{"loc":2217},{"loc":2529},"blog\u002Finside-llm-model-files-understanding-safetensors-and-gguf",[2466,2120,2674],"model formats","VJ8lTcArEPbBovszHcLjgyZJLXa6XC9hLZasseKDxFY",[],{"left":2678,"top":2678,"width":2020,"height":2020,"rotate":2678,"vFlip":2657,"hFlip":2657,"body":2679},0,"\u003Cpath fill=\"currentColor\" d=\"M12 .297c-6.63 0-12 5.373-12 12c0 5.303 3.438 9.8 8.205 11.385c.6.113.82-.258.82-.577c0-.285-.01-1.04-.015-2.04c-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729c1.205.084 1.838 1.236 1.838 1.236c1.07 1.835 2.809 1.305 3.495.998c.108-.776.417-1.305.76-1.605c-2.665-.3-5.466-1.332-5.466-5.93c0-1.31.465-2.38 1.235-3.22c-.135-.303-.54-1.523.105-3.176c0 0 1.005-.322 3.3 1.23c.96-.267 1.98-.399 3-.405c1.02.006 2.04.138 3 .405c2.28-1.552 3.285-1.23 3.285-1.23c.645 1.653.24 2.873.12 3.176c.765.84 1.23 1.91 1.23 3.22c0 4.61-2.805 5.625-5.475 5.92c.42.36.81 1.096.81 2.22c0 1.606-.015 2.896-.015 3.286c0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12\"\u002F>",{"left":2678,"top":2678,"width":2020,"height":2020,"rotate":2678,"vFlip":2657,"hFlip":2657,"body":2681,"hidden":1780},"\u003Cpath fill=\"currentColor\" d=\"M20.447 20.452h-3.554v-5.569c0-1.328-.027-3.037-1.852-3.037c-1.853 0-2.136 1.445-2.136 2.939v5.667H9.351V9h3.414v1.561h.046c.477-.9 1.637-1.85 3.37-1.85c3.601 0 4.267 2.37 4.267 5.455v6.286zM5.337 7.433a2.06 2.06 0 0 1-2.063-2.065a2.064 2.064 0 1 1 2.063 2.065m1.782 13.019H3.555V9h3.564zM22.225 0H1.771C.792 0 0 .774 0 1.729v20.542C0 23.227.792 24 1.771 24h20.451C23.2 24 24 23.227 24 22.271V1.729C24 .774 23.2 0 22.222 0z\"\u002F>",{"left":2678,"top":2678,"width":2020,"height":2020,"rotate":2678,"vFlip":2657,"hFlip":2657,"body":2683},"\u003Cpath fill=\"currentColor\" d=\"M5.202 2.857C7.954 4.922 10.913 9.11 12 11.358c1.087-2.247 4.046-6.436 6.798-8.501C20.783 1.366 24 .213 24 3.883c0 .732-.42 6.156-.667 7.037c-.856 3.061-3.978 3.842-6.755 3.37c4.854.826 6.089 3.562 3.422 6.299c-5.065 5.196-7.28-1.304-7.847-2.97c-.104-.305-.152-.448-.153-.327c0-.121-.05.022-.153.327c-.568 1.666-2.782 8.166-7.847 2.97c-2.667-2.737-1.432-5.473 3.422-6.3c-2.777.473-5.899-.308-6.755-3.369C.42 10.04 0 4.615 0 3.883c0-3.67 3.217-2.517 5.202-1.026\"\u002F>",{"left":2678,"top":2678,"width":2020,"height":2020,"rotate":2678,"vFlip":2657,"hFlip":2657,"body":2685},"\u003Cpath fill=\"currentColor\" d=\"M14.234 10.162L22.977 0h-2.072l-7.591 8.824L7.251 0H.258l9.168 13.343L.258 24H2.33l8.016-9.318L16.749 24h6.993zm-2.837 3.299l-.929-1.329L3.076 1.56h3.182l5.965 8.532l.929 1.329l7.754 11.09h-3.182z\"\u002F>",1793168446805]