Hugging Face's logo

ggml-org
/
Laguna-XS-2.1-GGUF

Text Generation
GGUF
quantized
conversational
Model card Files
xet
 Community
Laguna-XS-2.1-GGUF / convert.log
ggerganov's picture
ggerganov HF Staff
Upload folder using huggingface_hub
273068c verified
Raw
History Blame Contribute Delete
325 kB
+ OUTPUT_DIR=./upload-Laguna_XS_2.1
+ LLAMA_CPP=./llama.cpp
+ DISPLAY_NAME=Laguna-XS-2.1
+ QUANTIZE=./llama.cpp/build/bin/llama-quantize
+ python3 ./llama.cpp/convert_hf_to_gguf.py ./model-temp-Laguna_XS_2.1-PRIMARY --outtype bf16 --outfile ./upload-Laguna_XS_2.1/Laguna-XS-2.1-BF16.gguf --model-name Laguna-XS-2.1
INFO:hf-to-gguf:Loading model: model-temp-Laguna_XS_2.1-PRIMARY
INFO:hf-to-gguf:Model architecture: LagunaForCausalLM
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: indexing model part 'model-00001-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00002-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00003-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00004-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00005-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00006-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00007-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00008-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00009-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00010-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00011-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00012-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00013-of-00014.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00014-of-00014.safetensors'
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:output.weight, torch.bfloat16 --> BF16, shape = {2048, 100352}
INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> BF16, shape = {2048, 100352}
INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.0.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.0.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 48}
INFO:hf-to-gguf:blk.0.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> BF16, shape = {6144, 2048}
INFO:hf-to-gguf:blk.0.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 6144}
INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.1.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.1.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.1.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.1.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.1.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.1.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.1.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.10.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.10.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.10.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.10.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.10.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.10.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.10.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.10.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.10.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.11.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.11.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.11.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.11.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.11.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.11.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.11.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.11.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.12.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.12.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.12.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.12.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.12.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.12.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.12.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 48}
INFO:hf-to-gguf:blk.12.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> BF16, shape = {6144, 2048}
INFO:hf-to-gguf:blk.12.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 6144}
INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.13.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.13.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.13.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.13.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.13.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.13.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.13.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.13.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.13.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.14.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.14.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.14.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.14.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.14.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.14.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.14.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.14.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.15.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.15.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.15.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.15.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.15.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.15.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.15.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.15.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.15.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.16.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.16.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.16.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.16.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.16.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.16.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 48}
INFO:hf-to-gguf:blk.16.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> BF16, shape = {6144, 2048}
INFO:hf-to-gguf:blk.16.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 6144}
INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.17.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.17.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.17.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.17.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.17.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.17.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.17.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.17.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.17.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.18.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.18.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.18.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.18.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.18.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.18.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.18.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.18.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.18.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.19.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.19.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.19.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.19.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.19.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.19.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.19.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.19.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.2.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.2.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.2.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.2.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.2.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.2.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.2.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.2.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.2.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.20.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.20.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.20.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.20.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.20.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.20.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.20.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 48}
INFO:hf-to-gguf:blk.20.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> BF16, shape = {6144, 2048}
INFO:hf-to-gguf:blk.20.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 6144}
INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.21.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.21.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.21.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.21.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.21.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.21.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.21.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.21.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.22.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.22.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.22.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.22.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.22.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.22.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.22.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.22.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.22.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.23.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.23.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.23.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.23.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.23.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.23.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.23.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.23.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.23.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.24.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.24.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.24.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.24.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.24.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.24.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 48}
INFO:hf-to-gguf:blk.24.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> BF16, shape = {6144, 2048}
INFO:hf-to-gguf:blk.24.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 6144}
INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.25.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.25.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.25.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.25.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.25.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.25.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.25.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.25.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.25.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.26.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.26.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.26.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.26.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.26.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.26.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.26.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.26.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.26.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.26.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.26.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.26.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.26.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.27.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.27.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.27.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.27.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.27.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.27.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.27.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.27.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.27.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.27.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.27.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.27.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.27.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.27.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.28.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.28.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.28.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.28.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.28.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.28.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.28.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.28.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 48}
INFO:hf-to-gguf:blk.28.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.28.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.28.attn_output.weight, torch.bfloat16 --> BF16, shape = {6144, 2048}
INFO:hf-to-gguf:blk.28.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.28.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 6144}
INFO:hf-to-gguf:blk.28.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.29.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.29.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.29.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.29.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.29.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.29.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.29.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.29.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.29.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.29.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.29.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.29.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.29.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.3.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.3.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.3.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.3.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.3.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.3.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.3.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.3.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.3.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.30.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.30.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.30.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.30.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.30.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.30.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.30.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.30.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.30.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.30.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.30.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.30.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.30.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.30.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.31.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.31.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.31.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.31.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.31.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.31.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.31.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.31.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.31.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.31.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.31.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.31.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.31.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.32.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.32.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.32.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.32.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.32.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.32.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.32.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.32.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 48}
INFO:hf-to-gguf:blk.32.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.32.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.32.attn_output.weight, torch.bfloat16 --> BF16, shape = {6144, 2048}
INFO:hf-to-gguf:blk.32.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.32.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 6144}
INFO:hf-to-gguf:blk.32.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.33.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.33.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.33.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.33.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.33.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.33.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.33.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.33.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.33.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.33.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.33.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.33.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.33.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.33.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.34.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.34.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.34.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.34.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.34.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.34.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.34.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.34.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.34.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.34.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.34.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.34.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.34.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.35.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.35.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.35.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.35.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.35.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.35.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.35.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.35.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.35.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.35.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.35.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.35.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.35.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.35.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.36.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.36.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.36.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.36.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.36.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.36.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.36.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.36.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 48}
INFO:hf-to-gguf:blk.36.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.36.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.36.attn_output.weight, torch.bfloat16 --> BF16, shape = {6144, 2048}
INFO:hf-to-gguf:blk.36.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.36.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 6144}
INFO:hf-to-gguf:blk.36.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.37.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.37.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.37.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.37.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.37.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.37.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.37.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.37.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.37.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.37.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.37.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.37.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.37.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.37.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.38.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.38.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.38.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.38.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.38.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.38.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.38.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.38.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.38.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.38.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.38.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.38.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.38.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.38.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.39.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.39.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.39.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.39.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.39.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.39.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.39.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.39.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.39.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.39.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.39.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.39.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.39.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.39.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.4.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.4.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.4.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.4.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.4.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.4.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.4.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 48}
INFO:hf-to-gguf:blk.4.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> BF16, shape = {6144, 2048}
INFO:hf-to-gguf:blk.4.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 6144}
INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.5.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.5.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.5.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.5.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.5.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.5.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.5.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.5.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.5.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.6.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.6.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.6.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.6.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.6.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.6.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.6.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.6.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.7.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.7.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.7.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.7.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.7.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.7.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.7.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.7.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.7.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.8.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.8.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.8.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.8.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.8.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.8.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.8.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 48}
INFO:hf-to-gguf:blk.8.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> BF16, shape = {6144, 2048}
INFO:hf-to-gguf:blk.8.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 6144}
INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.exp_probs_b.bias, torch.bfloat16 --> F32, shape = {256}
INFO:hf-to-gguf:blk.9.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {2048, 256}
INFO:hf-to-gguf:blk.9.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {512, 2048}
INFO:hf-to-gguf:blk.9.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.9.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {2048, 512}
INFO:hf-to-gguf:blk.9.ffn_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.9.attn_gate.weight, torch.bfloat16 --> BF16, shape = {2048, 64}
INFO:hf-to-gguf:blk.9.attn_k_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> BF16, shape = {8192, 2048}
INFO:hf-to-gguf:blk.9.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> BF16, shape = {2048, 8192}
INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> BF16, shape = {2048, 1024}
INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {2048}
INFO:hf-to-gguf:blk.1.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.1.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.1.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.2.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.2.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.2.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.3.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.3.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.3.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.4.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.4.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.4.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.5.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.5.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.5.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.6.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.6.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.6.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.7.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.7.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.7.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.10.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.10.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.10.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.8.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.8.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.8.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.9.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.9.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.9.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.11.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.11.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.11.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.12.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.12.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.12.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.13.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.13.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.13.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.14.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.14.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.14.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.15.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.15.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.15.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.16.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.16.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.16.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.17.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.17.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.17.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.18.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.18.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.18.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.19.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.19.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.19.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.20.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.20.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.20.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.21.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.21.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.21.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.22.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.22.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.22.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.23.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.23.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.23.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.24.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.24.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.24.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.25.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.25.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.25.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.26.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.26.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.26.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.27.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.27.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.27.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.28.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.28.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.28.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.29.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.29.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.29.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.30.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.30.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.30.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.31.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.31.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.31.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.32.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.32.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.32.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.33.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.33.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.33.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.34.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.34.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.34.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.35.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.35.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.35.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.36.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.36.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.36.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.37.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.37.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.37.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.38.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.38.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.38.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:blk.39.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.39.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {2048, 512, 256}
INFO:hf-to-gguf:blk.39.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {512, 2048, 256}
INFO:hf-to-gguf:Set meta model
INFO:hf-to-gguf:Set model parameters
INFO:hf-to-gguf:gguf: context length = 262144
INFO:hf-to-gguf:gguf: embedding length = 2048
INFO:hf-to-gguf:gguf: feed forward length = 8192
INFO:hf-to-gguf:gguf: head count = 48
INFO:hf-to-gguf:gguf: key-value head count = 8
INFO:hf-to-gguf:gguf: rope scaling type = YARN
INFO:hf-to-gguf:gguf: rope theta = 500000.0
INFO:hf-to-gguf:gguf: rope theta swa = 10000.0
INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-06
INFO:hf-to-gguf:gguf: expert count = 256
INFO:hf-to-gguf:gguf: experts used count = 8
INFO:hf-to-gguf:gguf: file type = 32
WARNING:gguf.gguf_writer:Duplicated key name 'laguna.attention.head_count', overwriting it with new value [48, 64, 64, 64, 48, 64, 64, 64, 48, 64, 64, 64, 48, 64, 64, 64, 48, 64, 64, 64, 48, 64, 64, 64, 48, 64, 64, 64, 48, 64, 64, 64, 48, 64, 64, 64, 48, 64, 64, 64] of type ARRAY
INFO:hf-to-gguf:Set model quantization version
INFO:hf-to-gguf:Set model tokenizer
[transformers] The tokenizer you are loading from 'model-temp-Laguna_XS_2.1-PRIMARY' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
INFO:hf-to-gguf:gguf: marked </assistant> (id 24) as CONTROL token
INFO:gguf.vocab:Adding 100026 merge(s).
INFO:gguf.vocab:Setting special token type bos to 2
INFO:gguf.vocab:Setting special token type eos to 2
INFO:gguf.vocab:Setting special token type unk to 0
INFO:gguf.vocab:Setting special token type sep to 8
INFO:gguf.vocab:Setting special token type pad to 9
WARNING:gguf.vocab:No handler for special token type cls with id 10 - skipping
INFO:gguf.vocab:Setting special token type mask to 12
INFO:gguf.vocab:Setting add_bos_token to True
INFO:gguf.vocab:Setting add_sep_token to True
INFO:gguf.vocab:Setting chat_template to {#- Iteration on laguna_glm_thinking_v8/chat_template.jinja -#}
{#- No formatting instructions -#}
{{- "γ€ˆ|EOS|〉" -}}
{%- set enable_thinking = enable_thinking | default(false) -%}
{%- set add_generation_prompt = add_generation_prompt | default(false) -%}
{#- ───── header (system message) ───── -#}
{#- A caller-supplied system message with empty content opts out of the default below, producing no <system> block β€” used to train without a system message. -#}
{%- set system_message = "You are a helpful, conversationally-fluent assistant made by Poolside. You are here to be helpful to users through natural language conversations." -%}
{%- if messages and messages[0].role == "system" -%}
 {%- set system_message = messages[0].content -%}
 {%- set messages = messages[1:] -%}
{%- endif -%}
{%- set has_sys = system_message and system_message.strip() -%}
{%- if has_sys or tools or enable_thinking -%}
 {{- "<system>" -}}
 {%- if has_sys -%}
 {{- system_message.rstrip() -}}
 {%- if tools -%}{{- "\n\n" -}}{%- endif -%}
 {%- endif -%}
 {%- if tools -%}
 {{- "### Tools\n\n" -}}
 {{- "You may call functions to assist with the user query.\n" -}}
 {{- "All available function signatures are listed below:\n" -}}
 {{- "<available_tools>\n" -}}
 {%- for tool in tools -%}
 {{- (tool | tojson) ~ "\n" -}}
 {%- endfor -%}
 {{- "</available_tools>" -}}
 {%- endif -%}
 {{- "</system>\n" -}}
{%- endif -%}
{#- ───── main loop ───── -#}
{%- for message in messages -%}
 {%- set content = message.content if message.content is string else "" -%}
 {%- if message.role == "user" -%}
 {{- "<user>" + content + "</user>\n" -}}
 {%- elif message.role == "assistant" -%}
 {%- generation -%}
 {{- "<assistant>" -}}
 {#- Extract reasoning content from message.reasoning (vLLM field name) or message.reasoning_content -#}
 {%- set reasoning_content = '' -%}
 {%- if message.reasoning is string -%}
 {%- set reasoning_content = message.reasoning -%}
 {%- elif message.reasoning_content is string -%}
 {%- set reasoning_content = message.reasoning_content -%}
 {%- endif -%}
 {#- Display reasoning content for all messages if enable_thinking -#}
 {%- if enable_thinking -%}
 {{- '<think>' + reasoning_content + '</think>' -}}
 {%- else -%}
 {{- '</think>' -}}
 {%- endif -%}
 {#- Display main content (trailing newline only when no tool_calls follow) -#}
 {%- if content -%}
 {{- content -}}
 {%- endif -%}
 {%- if message.tool_calls -%}
 {%- for tool_call in message.tool_calls -%}
 {%- set function_data = tool_call.function -%}
 {{- '<tool_call>' + function_data.name -}}
 {%- set _args = function_data.arguments -%}
 {%- for k, v in _args.items() -%}
 {{- "<arg_key>" ~ k ~ "</arg_key>" -}}
 {{- "<arg_value>" -}}{{- v | tojson(ensure_ascii=False) if v is not string else v -}}{{- "</arg_value>" -}}
 {%- endfor -%}
 {{- "</tool_call>" -}}
 {%- endfor -%}
 {%- endif -%}
 {{- "</assistant>\n" -}}
 {%- endgeneration -%}
 {%- elif message.role == "tool" -%}
 {{- "<tool_response>" + content + "</tool_response>\n" -}}
 {%- elif message.role == "system" -%}
 {#- Render additional system messages (the first one, if any, is handled separately in the header and was sliced off above) -#}
 {{- "<system>" + content + "</system>\n" -}}
 {%- endif -%}
{%- endfor -%}
{#- ───── generation prompt ───── -#}
{%- if add_generation_prompt -%}
 {{- "<assistant>" -}}
 {#- ───── Include reasoning mode directive ───── -#}
 {%- if enable_thinking -%}
 {{- '<think>' -}}
 {%- else -%}
 {{- '</think>' -}}
 {%- endif -%}
{%- endif -%}
WARNING:gguf.gguf_writer:Duplicated key name 'tokenizer.chat_template', overwriting it with new value '{#- Iteration on laguna_glm_thinking_v8/chat_template.jinja -#}\n{#- No formatting instructions -#}\n{{- "γ€ˆ|EOS|〉" -}}\n{%- set enable_thinking = enable_thinking | default(false) -%}\n{%- set add_generation_prompt = add_generation_prompt | default(false) -%}\n\n{#- ───── header (system message) ───── -#}\n{#- A caller-supplied system message with empty content opts out of the default below, producing no <system> block β€” used to train without a system message. -#}\n{%- set system_message = "You are a helpful, conversationally-fluent assistant made by Poolside. You are here to be helpful to users through natural language conversations." -%}\n{%- if messages and messages[0].role == "system" -%}\n {%- set system_message = messages[0].content -%}\n {%- set messages = messages[1:] -%}\n{%- endif -%}\n\n{%- set has_sys = system_message and system_message.strip() -%}\n{%- if has_sys or tools or enable_thinking -%}\n {{- "<system>" -}}\n\n {%- if has_sys -%}\n {{- system_message.rstrip() -}}\n {%- if tools -%}{{- "\\n\\n" -}}{%- endif -%}\n {%- endif -%}\n\n {%- if tools -%}\n {{- "### Tools\\n\\n" -}}\n {{- "You may call functions to assist with the user query.\\n" -}}\n {{- "All available function signatures are listed below:\\n" -}}\n {{- "<available_tools>\\n" -}}\n {%- for tool in tools -%}\n {{- (tool | tojson) ~ "\\n" -}}\n {%- endfor -%}\n {{- "</available_tools>" -}}\n {%- endif -%}\n\n {{- "</system>\\n" -}}\n{%- endif -%}\n\n{#- ───── main loop ───── -#}\n{%- for message in messages -%}\n {%- set content = message.content if message.content is string else "" -%}\n {%- if message.role == "user" -%}\n {{- "<user>" + content + "</user>\\n" -}}\n {%- elif message.role == "assistant" -%}\n {%- generation -%}\n {{- "<assistant>" -}}\n {#- Extract reasoning content from message.reasoning (vLLM field name) or message.reasoning_content -#}\n {%- set reasoning_content = \'\' -%}\n {%- if message.reasoning is string -%}\n {%- set reasoning_content = message.reasoning -%}\n {%- elif message.reasoning_content is string -%}\n {%- set reasoning_content = message.reasoning_content -%}\n {%- endif -%}\n {#- Display reasoning content for all messages if enable_thinking -#}\n {%- if enable_thinking -%}\n {{- \'<think>\' + reasoning_content + \'</think>\' -}}\n {%- else -%}\n {{- \'</think>\' -}}\n {%- endif -%}\n {#- Display main content (trailing newline only when no tool_calls follow) -#}\n {%- if content -%}\n {{- content -}}\n {%- endif -%}\n {%- if message.tool_calls -%}\n {%- for tool_call in message.tool_calls -%}\n {%- set function_data = tool_call.function -%}\n {{- \'<tool_call>\' + function_data.name -}}\n {%- set _args = function_data.arguments -%}\n {%- for k, v in _args.items() -%}\n {{- "<arg_key>" ~ k ~ "</arg_key>" -}}\n {{- "<arg_value>" -}}{{- v | tojson(ensure_ascii=False) if v is not string else v -}}{{- "</arg_value>" -}}\n {%- endfor -%}\n {{- "</tool_call>" -}}\n {%- endfor -%}\n {%- endif -%}\n {{- "</assistant>\\n" -}}\n {%- endgeneration -%}\n {%- elif message.role == "tool" -%}\n {{- "<tool_response>" + content + "</tool_response>\\n" -}}\n {%- elif message.role == "system" -%}\n {#- Render additional system messages (the first one, if any, is handled separately in the header and was sliced off above) -#}\n {{- "<system>" + content + "</system>\\n" -}}\n {%- endif -%}\n{%- endfor -%}\n{#- ───── generation prompt ───── -#}\n{%- if add_generation_prompt -%}\n {{- "<assistant>" -}}\n {#- ───── Include reasoning mode directive ───── -#}\n {%- if enable_thinking -%}\n {{- \'<think>\' -}}\n {%- else -%}\n {{- \'</think>\' -}}\n {%- endif -%}\n{%- endif -%}' of type STRING
INFO:hf-to-gguf:gguf: embedded resolved chat_template.jinja (overriding include directive)
INFO:hf-to-gguf:gguf: registered eot_token_id=24 from eos list [2, 24]
INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:upload-Laguna_XS_2.1/Laguna-XS-2.1-BF16.gguf: n_tensors = 678, total_size = 66.9G
Writing: 0%| | 0.00/66.9G [00:00<?, ?byte/s]
 Writing: 1%| | 411M/66.9G [00:01<02:49, 392Mbyte/s]
 Writing: 1%| | 822M/66.9G [00:02<02:54, 379Mbyte/s]
 Writing: 2%|▏ | 1.23G/66.9G [00:03<02:48, 391Mbyte/s]
 Writing: 3%|β–Ž | 1.71G/66.9G [00:04<02:35, 419Mbyte/s]
 Writing: 3%|β–Ž | 2.19G/66.9G [00:05<02:28, 437Mbyte/s]
 Writing: 4%|▍ | 2.66G/66.9G [00:06<02:23, 448Mbyte/s]
 Writing: 5%|▍ | 3.15G/66.9G [00:07<02:19, 456Mbyte/s]
 Writing: 5%|β–Œ | 3.64G/66.9G [00:08<02:17, 461Mbyte/s]
 Writing: 6%|β–Œ | 4.11G/66.9G [00:09<02:15, 463Mbyte/s]
 Writing: 7%|β–‹ | 4.65G/66.9G [00:10<02:23, 435Mbyte/s]
 Writing: 8%|β–Š | 5.19G/66.9G [00:12<02:26, 422Mbyte/s]
 Writing: 9%|β–Š | 5.72G/66.9G [00:13<02:46, 368Mbyte/s]
 Writing: 9%|β–‰ | 6.26G/66.9G [00:15<02:43, 370Mbyte/s]
 Writing: 10%|β–ˆ | 6.80G/66.9G [00:16<02:39, 377Mbyte/s]
 Writing: 11%|β–ˆ | 7.33G/66.9G [00:18<02:49, 351Mbyte/s]
 Writing: 12%|β–ˆβ– | 7.87G/66.9G [00:19<02:45, 358Mbyte/s]
 Writing: 13%|β–ˆβ–Ž | 8.41G/66.9G [00:21<02:39, 368Mbyte/s]
 Writing: 13%|β–ˆβ–Ž | 8.94G/66.9G [00:23<02:48, 343Mbyte/s]
 Writing: 14%|β–ˆβ– | 9.48G/66.9G [00:24<02:41, 355Mbyte/s]
 Writing: 15%|β–ˆβ– | 10.0G/66.9G [00:25<02:33, 371Mbyte/s]
 Writing: 16%|β–ˆβ–Œ | 10.6G/66.9G [00:27<02:50, 331Mbyte/s]
 Writing: 17%|β–ˆβ–‹ | 11.1G/66.9G [00:29<02:39, 349Mbyte/s]
 Writing: 17%|β–ˆβ–‹ | 11.6G/66.9G [00:30<02:33, 360Mbyte/s]
 Writing: 18%|β–ˆβ–Š | 12.2G/66.9G [00:32<02:42, 338Mbyte/s]
 Writing: 19%|β–ˆβ–‰ | 12.7G/66.9G [00:33<02:31, 358Mbyte/s]
 Writing: 20%|β–ˆβ–‰ | 13.2G/66.9G [00:34<02:25, 369Mbyte/s]
 Writing: 21%|β–ˆβ–ˆ | 13.8G/66.9G [00:36<02:33, 346Mbyte/s]
 Writing: 21%|β–ˆβ–ˆβ– | 14.3G/66.9G [00:38<02:26, 360Mbyte/s]
 Writing: 22%|β–ˆβ–ˆβ– | 14.9G/66.9G [00:39<02:17, 378Mbyte/s]
 Writing: 23%|β–ˆβ–ˆβ–Ž | 15.4G/66.9G [00:41<02:28, 347Mbyte/s]
 Writing: 24%|β–ˆβ–ˆβ– | 15.9G/66.9G [00:42<02:23, 356Mbyte/s]
 Writing: 25%|β–ˆβ–ˆβ– | 16.5G/66.9G [00:44<02:18, 364Mbyte/s]
 Writing: 25%|β–ˆβ–ˆβ–Œ | 17.0G/66.9G [00:46<02:32, 327Mbyte/s]
 Writing: 26%|β–ˆβ–ˆβ–Œ | 17.5G/66.9G [00:47<02:23, 344Mbyte/s]
 Writing: 27%|β–ˆβ–ˆβ–‹ | 18.1G/66.9G [00:48<02:16, 358Mbyte/s]
 Writing: 28%|β–ˆβ–ˆβ–Š | 18.6G/66.9G [00:50<02:24, 334Mbyte/s]
 Writing: 29%|β–ˆβ–ˆβ–Š | 19.1G/66.9G [00:51<02:15, 354Mbyte/s]
 Writing: 29%|β–ˆβ–ˆβ–‰ | 19.7G/66.9G [00:53<02:09, 366Mbyte/s]
 Writing: 30%|β–ˆβ–ˆβ–ˆ | 20.2G/66.9G [00:55<02:20, 334Mbyte/s]
 Writing: 31%|β–ˆβ–ˆβ–ˆ | 20.8G/66.9G [00:56<02:12, 349Mbyte/s]
 Writing: 32%|β–ˆβ–ˆβ–ˆβ– | 21.3G/66.9G [00:57<02:05, 363Mbyte/s]
 Writing: 33%|β–ˆβ–ˆβ–ˆβ–Ž | 21.8G/66.9G [00:59<02:13, 337Mbyte/s]
 Writing: 33%|β–ˆβ–ˆβ–ˆβ–Ž | 22.4G/66.9G [01:01<02:04, 357Mbyte/s]
 Writing: 34%|β–ˆβ–ˆβ–ˆβ– | 22.9G/66.9G [01:02<01:59, 367Mbyte/s]
 Writing: 35%|β–ˆβ–ˆβ–ˆβ–Œ | 23.4G/66.9G [01:04<02:08, 338Mbyte/s]
 Writing: 36%|β–ˆβ–ˆβ–ˆβ–Œ | 24.0G/66.9G [01:05<01:59, 359Mbyte/s]
 Writing: 37%|β–ˆβ–ˆβ–ˆβ–‹ | 24.5G/66.9G [01:06<01:54, 371Mbyte/s]
 Writing: 37%|β–ˆβ–ˆβ–ˆβ–‹ | 25.1G/66.9G [01:08<02:00, 348Mbyte/s]
 Writing: 38%|β–ˆβ–ˆβ–ˆβ–Š | 25.6G/66.9G [01:10<01:53, 363Mbyte/s]
 Writing: 39%|β–ˆβ–ˆβ–ˆβ–‰ | 26.1G/66.9G [01:11<01:47, 380Mbyte/s]
 Writing: 40%|β–ˆβ–ˆβ–ˆβ–‰ | 26.7G/66.9G [01:13<01:55, 350Mbyte/s]
 Writing: 41%|β–ˆβ–ˆβ–ˆβ–ˆ | 27.2G/66.9G [01:14<01:47, 368Mbyte/s]
 Writing: 41%|β–ˆβ–ˆβ–ˆβ–ˆβ– | 27.7G/66.9G [01:15<01:44, 376Mbyte/s]
 Writing: 42%|β–ˆβ–ˆβ–ˆβ–ˆβ– | 28.3G/66.9G [01:17<01:52, 344Mbyte/s]
 Writing: 43%|β–ˆβ–ˆβ–ˆβ–ˆβ–Ž | 28.8G/66.9G [01:19<01:47, 354Mbyte/s]
 Writing: 44%|β–ˆβ–ˆβ–ˆβ–ˆβ– | 29.3G/66.9G [01:20<01:43, 363Mbyte/s]
 Writing: 45%|β–ˆβ–ˆβ–ˆβ–ˆβ– | 29.9G/66.9G [01:22<01:50, 336Mbyte/s]
 Writing: 45%|β–ˆβ–ˆβ–ˆβ–ˆβ–Œ | 30.4G/66.9G [01:23<01:43, 351Mbyte/s]
 Writing: 46%|β–ˆβ–ˆβ–ˆβ–ˆβ–‹ | 31.0G/66.9G [01:25<01:38, 364Mbyte/s]
 Writing: 47%|β–ˆβ–ˆβ–ˆβ–ˆβ–‹ | 31.5G/66.9G [01:26<01:45, 336Mbyte/s]
 Writing: 48%|β–ˆβ–ˆβ–ˆβ–ˆβ–Š | 32.0G/66.9G [01:28<01:38, 356Mbyte/s]
 Writing: 49%|β–ˆβ–ˆβ–ˆβ–ˆβ–Š | 32.6G/66.9G [01:29<01:34, 363Mbyte/s]
 Writing: 49%|β–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 33.1G/66.9G [01:31<01:39, 340Mbyte/s]
 Writing: 50%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 33.6G/66.9G [01:32<01:34, 353Mbyte/s]
 Writing: 51%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 34.2G/66.9G [01:34<01:32, 354Mbyte/s]
 Writing: 52%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 34.7G/66.9G [01:36<01:38, 327Mbyte/s]
 Writing: 53%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž | 35.3G/66.9G [01:37<01:34, 335Mbyte/s]
 Writing: 53%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž | 35.8G/66.9G [01:39<01:29, 349Mbyte/s]
 Writing: 54%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 36.3G/66.9G [01:41<01:36, 317Mbyte/s]
 Writing: 55%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Œ | 36.9G/66.9G [01:42<01:29, 337Mbyte/s]
 Writing: 56%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Œ | 37.4G/66.9G [01:43<01:24, 351Mbyte/s]
 Writing: 57%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‹ | 37.9G/66.9G [01:45<01:28, 327Mbyte/s]
 Writing: 57%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‹ | 38.5G/66.9G [01:47<01:21, 348Mbyte/s]
 Writing: 58%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Š | 39.0G/66.9G [01:48<01:18, 358Mbyte/s]
 Writing: 59%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 39.5G/66.9G [01:50<01:24, 323Mbyte/s]
 Writing: 60%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 40.1G/66.9G [01:51<01:18, 342Mbyte/s]
 Writing: 61%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 40.6G/66.9G [01:53<01:14, 355Mbyte/s]
 Writing: 61%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 41.2G/66.9G [01:55<01:17, 332Mbyte/s]
 Writing: 62%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 41.7G/66.9G [01:56<01:12, 348Mbyte/s]
 Writing: 63%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž | 42.2G/66.9G [01:57<01:08, 361Mbyte/s]
 Writing: 64%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 42.8G/66.9G [01:59<01:10, 341Mbyte/s]
 Writing: 65%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 43.3G/66.9G [02:01<01:06, 357Mbyte/s]
 Writing: 66%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Œ | 43.8G/66.9G [02:02<01:01, 376Mbyte/s]
 Writing: 66%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‹ | 44.4G/66.9G [02:04<01:05, 343Mbyte/s]
 Writing: 67%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‹ | 44.9G/66.9G [02:05<01:00, 363Mbyte/s]
 Writing: 68%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Š | 45.5G/66.9G [02:06<00:58, 368Mbyte/s]
 Writing: 69%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Š | 46.0G/66.9G [02:08<01:02, 337Mbyte/s]
 Writing: 70%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 46.5G/66.9G [02:10<00:57, 352Mbyte/s]
 Writing: 70%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 47.1G/66.9G [02:11<00:54, 364Mbyte/s]
 Writing: 71%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 47.6G/66.9G [02:13<00:56, 341Mbyte/s]
 Writing: 72%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 48.1G/66.9G [02:14<00:52, 360Mbyte/s]
 Writing: 73%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž | 48.7G/66.9G [02:16<00:49, 368Mbyte/s]
 Writing: 74%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž | 49.2G/66.9G [02:17<00:51, 345Mbyte/s]
 Writing: 74%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 49.7G/66.9G [02:19<00:47, 363Mbyte/s]
 Writing: 75%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Œ | 50.3G/66.9G [02:20<00:45, 367Mbyte/s]
 Writing: 76%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Œ | 50.8G/66.9G [02:22<00:47, 342Mbyte/s]
 Writing: 77%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‹ | 51.4G/66.9G [02:23<00:42, 362Mbyte/s]
 Writing: 78%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Š | 51.9G/66.9G [02:24<00:40, 372Mbyte/s]
 Writing: 78%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Š | 52.4G/66.9G [02:26<00:41, 347Mbyte/s]
 Writing: 79%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 53.0G/66.9G [02:28<00:37, 368Mbyte/s]
 Writing: 80%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 53.5G/66.9G [02:29<00:35, 377Mbyte/s]
 Writing: 81%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 54.0G/66.9G [02:31<00:38, 335Mbyte/s]
 Writing: 82%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 54.6G/66.9G [02:32<00:35, 348Mbyte/s]
 Writing: 82%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 55.1G/66.9G [02:34<00:32, 360Mbyte/s]
 Writing: 83%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž | 55.7G/66.9G [02:35<00:33, 338Mbyte/s]
 Writing: 84%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 56.2G/66.9G [02:37<00:30, 354Mbyte/s]
 Writing: 85%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– | 56.7G/66.9G [02:38<00:27, 374Mbyte/s]
 Writing: 86%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Œ | 57.3G/66.9G [02:40<00:28, 340Mbyte/s]
 Writing: 86%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‹ | 57.8G/66.9G [02:41<00:25, 354Mbyte/s]
 Writing: 87%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‹ | 58.3G/66.9G [02:43<00:23, 366Mbyte/s]
 Writing: 88%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Š | 58.9G/66.9G [02:45<00:23, 338Mbyte/s]
 Writing: 89%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 59.4G/66.9G [02:46<00:21, 354Mbyte/s]
 Writing: 90%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ | 59.9G/66.9G [02:47<00:18, 372Mbyte/s]
 Writing: 90%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 60.5G/66.9G [02:49<00:18, 343Mbyte/s]
 Writing: 91%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ | 61.0G/66.9G [02:50<00:16, 359Mbyte/s]
 Writing: 92%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–| 61.6G/66.9G [02:52<00:14, 376Mbyte/s]
 Writing: 93%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž| 62.1G/66.9G [02:53<00:13, 346Mbyte/s]
 Writing: 94%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž| 62.6G/66.9G [02:55<00:11, 367Mbyte/s]
 Writing: 94%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–| 63.2G/66.9G [02:56<00:10, 373Mbyte/s]
 Writing: 95%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Œ| 63.7G/66.9G [02:58<00:09, 344Mbyte/s]
 Writing: 96%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Œ| 64.2G/66.9G [02:59<00:07, 360Mbyte/s]
 Writing: 97%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‹| 64.8G/66.9G [03:01<00:05, 368Mbyte/s]
 Writing: 98%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Š| 65.3G/66.9G [03:03<00:04, 337Mbyte/s]
 Writing: 98%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Š| 65.9G/66.9G [03:04<00:02, 358Mbyte/s]
 Writing: 99%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰| 66.4G/66.9G [03:05<00:01, 369Mbyte/s]
 Writing: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 66.9G/66.9G [03:07<00:00, 340Mbyte/s] Writing: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 66.9G/66.9G [03:07<00:00, 357Mbyte/s]
INFO:hf-to-gguf:Model successfully exported to upload-Laguna_XS_2.1/Laguna-XS-2.1-BF16.gguf
+ FLAGS_Q4_K_M='--pure --tensor-type output.weight=q6_k --tensor-type shexp=q8_0 --tensor-type attn_=q8_0'
+ ./llama.cpp/build/bin/llama-quantize ./upload-Laguna_XS_2.1/Laguna-XS-2.1-BF16.gguf ./upload-Laguna_XS_2.1/Laguna-XS-2.1-Q8_0.gguf Q8_0
version: 0.2.0-dev (build 1, commit 70adb1b)
built with GNU 14.2.0 for Linux x86_64
llama_quantize: quantizing './upload-Laguna_XS_2.1/Laguna-XS-2.1-BF16.gguf' to './upload-Laguna_XS_2.1/Laguna-XS-2.1-Q8_0.gguf' as Q8_0
llama_model_loader: loaded meta data with 56 key-value pairs and 678 tensors from ./upload-Laguna_XS_2.1/Laguna-XS-2.1-BF16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = laguna
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.sampling.top_k i32 = 20
llama_model_loader: - kv 3: general.sampling.top_p f32 = 1.000000
llama_model_loader: - kv 4: general.sampling.min_p f32 = 0.000000
llama_model_loader: - kv 5: general.sampling.temp f32 = 1.000000
llama_model_loader: - kv 6: general.name str = Laguna-XS-2.1
llama_model_loader: - kv 7: general.size_label str = 256x2.2B
llama_model_loader: - kv 8: general.license str = openmdw-1.1
llama_model_loader: - kv 9: general.tags arr[str,3] = ["laguna-xs-2.1", "vllm", "text-gener...
llama_model_loader: - kv 10: laguna.block_count u32 = 40
llama_model_loader: - kv 11: laguna.context_length u32 = 262144
llama_model_loader: - kv 12: laguna.embedding_length u32 = 2048
llama_model_loader: - kv 13: laguna.feed_forward_length u32 = 8192
llama_model_loader: - kv 14: laguna.attention.head_count arr[i32,40] = [48, 64, 64, 64, 48, 64, 64, 64, 48, ...
llama_model_loader: - kv 15: laguna.attention.head_count_kv u32 = 8
llama_model_loader: - kv 16: laguna.rope.scaling.type str = yarn
llama_model_loader: - kv 17: laguna.rope.scaling.factor f32 = 32.000000
llama_model_loader: - kv 18: laguna.rope.scaling.original_context_length u32 = 8192
llama_model_loader: - kv 19: laguna.rope.scaling.yarn_attn_factor f32 = 1.346574
llama_model_loader: - kv 20: laguna.rope.scaling.yarn_beta_fast f32 = 64.000000
llama_model_loader: - kv 21: laguna.rope.scaling.yarn_beta_slow f32 = 1.000000
llama_model_loader: - kv 22: laguna.rope.freq_base f32 = 500000.000000
llama_model_loader: - kv 23: laguna.rope.freq_base_swa f32 = 10000.000000
llama_model_loader: - kv 24: laguna.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 25: laguna.expert_count u32 = 256
llama_model_loader: - kv 26: laguna.expert_used_count u32 = 8
llama_model_loader: - kv 27: laguna.attention.key_length u32 = 128
llama_model_loader: - kv 28: laguna.attention.value_length u32 = 128
llama_model_loader: - kv 29: general.file_type u32 = 32
llama_model_loader: - kv 30: laguna.vocab_size u32 = 100352
llama_model_loader: - kv 31: laguna.attention.sliding_window u32 = 512
llama_model_loader: - kv 32: laguna.expert_feed_forward_length u32 = 512
llama_model_loader: - kv 33: laguna.expert_shared_feed_forward_length u32 = 512
llama_model_loader: - kv 34: laguna.expert_weights_norm bool = true
llama_model_loader: - kv 35: laguna.expert_weights_scale f32 = 2.500000
llama_model_loader: - kv 36: laguna.expert_gating_func u32 = 2
llama_model_loader: - kv 37: laguna.leading_dense_block_count u32 = 1
llama_model_loader: - kv 38: laguna.rope.dimension_count u32 = 64
llama_model_loader: - kv 39: laguna.rope.dimension_count_swa u32 = 128
llama_model_loader: - kv 40: general.quantization_version u32 = 2
llama_model_loader: - kv 41: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 42: tokenizer.ggml.pre str = laguna
llama_model_loader: - kv 43: tokenizer.ggml.tokens arr[str,100352] = ["γ€ˆ|UNK|〉", "γ€ˆ|CODE_START|〉",...
llama_model_loader: - kv 44: tokenizer.ggml.token_type arr[i32,100352] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ...
llama_model_loader: - kv 45: tokenizer.ggml.merges arr[str,100026] = ["i n", "Δ  t", "Δ  Δ ", "e r", "Δ  a...
llama_model_loader: - kv 46: tokenizer.ggml.bos_token_id u32 = 2
llama_model_loader: - kv 47: tokenizer.ggml.eos_token_id u32 = 2
llama_model_loader: - kv 48: tokenizer.ggml.unknown_token_id u32 = 0
llama_model_loader: - kv 49: tokenizer.ggml.seperator_token_id u32 = 8
llama_model_loader: - kv 50: tokenizer.ggml.padding_token_id u32 = 9
llama_model_loader: - kv 51: tokenizer.ggml.mask_token_id u32 = 12
llama_model_loader: - kv 52: tokenizer.ggml.add_bos_token bool = true
llama_model_loader: - kv 53: tokenizer.ggml.add_sep_token bool = true
llama_model_loader: - kv 54: tokenizer.chat_template str = {#- Iteration on laguna_glm_thinking_...
llama_model_loader: - kv 55: tokenizer.ggml.eot_token_id u32 = 24
llama_model_loader: - type f32: 239 tensors
llama_model_loader: - type bf16: 439 tensors
[ 1/ 678] output.weight - [ 2048, 100352, 1, 1], type = bf16, converting to q8_0 .. size = 392.00 MiB -> 208.25 MiB
[ 2/ 678] output_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 3/ 678] token_embd.weight - [ 2048, 100352, 1, 1], type = bf16, converting to q8_0 .. size = 392.00 MiB -> 208.25 MiB
[ 4/ 678] blk.0.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 5/ 678] blk.0.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 6/ 678] blk.0.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 7/ 678] blk.0.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 8/ 678] blk.0.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 9/ 678] blk.0.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 10/ 678] blk.0.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 11/ 678] blk.0.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 12/ 678] blk.0.ffn_down.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 13/ 678] blk.0.ffn_gate.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 14/ 678] blk.0.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 15/ 678] blk.0.ffn_up.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 16/ 678] blk.1.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 17/ 678] blk.1.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 18/ 678] blk.1.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 19/ 678] blk.1.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 20/ 678] blk.1.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 21/ 678] blk.1.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 22/ 678] blk.1.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 23/ 678] blk.1.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 24/ 678] blk.1.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 25/ 678] blk.1.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 26/ 678] blk.1.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 27/ 678] blk.1.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 28/ 678] blk.1.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 29/ 678] blk.1.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 30/ 678] blk.1.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 31/ 678] blk.1.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 32/ 678] blk.1.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 33/ 678] blk.2.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 34/ 678] blk.2.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 35/ 678] blk.2.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 36/ 678] blk.2.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 37/ 678] blk.2.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 38/ 678] blk.2.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 39/ 678] blk.2.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 40/ 678] blk.2.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 41/ 678] blk.2.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 42/ 678] blk.2.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 43/ 678] blk.2.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 44/ 678] blk.2.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 45/ 678] blk.2.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 46/ 678] blk.2.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 47/ 678] blk.2.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 48/ 678] blk.2.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 49/ 678] blk.2.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 50/ 678] blk.3.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 51/ 678] blk.3.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 52/ 678] blk.3.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 53/ 678] blk.3.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 54/ 678] blk.3.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 55/ 678] blk.3.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 56/ 678] blk.3.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 57/ 678] blk.3.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 58/ 678] blk.3.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 59/ 678] blk.3.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 60/ 678] blk.3.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 61/ 678] blk.3.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 62/ 678] blk.3.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 63/ 678] blk.3.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 64/ 678] blk.3.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 65/ 678] blk.3.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 66/ 678] blk.3.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 67/ 678] blk.4.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 68/ 678] blk.4.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 69/ 678] blk.4.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 70/ 678] blk.4.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 71/ 678] blk.4.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 72/ 678] blk.4.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 73/ 678] blk.4.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 74/ 678] blk.4.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 75/ 678] blk.4.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 76/ 678] blk.4.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 77/ 678] blk.4.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 78/ 678] blk.4.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 79/ 678] blk.4.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 80/ 678] blk.4.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 81/ 678] blk.4.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 82/ 678] blk.4.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 83/ 678] blk.4.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 84/ 678] blk.5.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 85/ 678] blk.5.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 86/ 678] blk.5.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 87/ 678] blk.5.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 88/ 678] blk.5.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 89/ 678] blk.5.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 90/ 678] blk.5.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 91/ 678] blk.5.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 92/ 678] blk.5.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 93/ 678] blk.5.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 94/ 678] blk.5.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 95/ 678] blk.5.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 96/ 678] blk.5.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 97/ 678] blk.5.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 98/ 678] blk.5.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 99/ 678] blk.5.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 100/ 678] blk.5.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 101/ 678] blk.6.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 102/ 678] blk.6.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 103/ 678] blk.6.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 104/ 678] blk.6.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 105/ 678] blk.6.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 106/ 678] blk.6.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 107/ 678] blk.6.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 108/ 678] blk.6.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 109/ 678] blk.6.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 110/ 678] blk.6.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 111/ 678] blk.6.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 112/ 678] blk.6.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 113/ 678] blk.6.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 114/ 678] blk.6.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 115/ 678] blk.6.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 116/ 678] blk.6.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 117/ 678] blk.6.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 118/ 678] blk.7.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 119/ 678] blk.7.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 120/ 678] blk.7.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 121/ 678] blk.7.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 122/ 678] blk.7.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 123/ 678] blk.7.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 124/ 678] blk.7.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 125/ 678] blk.7.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 126/ 678] blk.7.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 127/ 678] blk.7.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 128/ 678] blk.7.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 129/ 678] blk.7.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 130/ 678] blk.7.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 131/ 678] blk.7.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 132/ 678] blk.7.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 133/ 678] blk.7.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 134/ 678] blk.7.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 135/ 678] blk.8.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 136/ 678] blk.8.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 137/ 678] blk.8.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 138/ 678] blk.8.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 139/ 678] blk.8.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 140/ 678] blk.8.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 141/ 678] blk.8.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 142/ 678] blk.8.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 143/ 678] blk.8.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 144/ 678] blk.8.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 145/ 678] blk.8.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 146/ 678] blk.8.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 147/ 678] blk.8.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 148/ 678] blk.8.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 149/ 678] blk.8.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 150/ 678] blk.8.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 151/ 678] blk.8.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 152/ 678] blk.9.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 153/ 678] blk.9.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 154/ 678] blk.9.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 155/ 678] blk.9.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 156/ 678] blk.9.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 157/ 678] blk.9.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 158/ 678] blk.9.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 159/ 678] blk.9.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 160/ 678] blk.9.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 161/ 678] blk.9.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 162/ 678] blk.9.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 163/ 678] blk.9.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 164/ 678] blk.9.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 165/ 678] blk.9.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 166/ 678] blk.9.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 167/ 678] blk.9.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 168/ 678] blk.9.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 169/ 678] blk.10.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 170/ 678] blk.10.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 171/ 678] blk.10.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 172/ 678] blk.10.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 173/ 678] blk.10.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 174/ 678] blk.10.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 175/ 678] blk.10.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 176/ 678] blk.10.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 177/ 678] blk.10.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 178/ 678] blk.10.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 179/ 678] blk.10.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 180/ 678] blk.10.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 181/ 678] blk.10.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 182/ 678] blk.10.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 183/ 678] blk.10.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 184/ 678] blk.10.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 185/ 678] blk.10.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 186/ 678] blk.11.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 187/ 678] blk.11.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 188/ 678] blk.11.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 189/ 678] blk.11.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 190/ 678] blk.11.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 191/ 678] blk.11.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 192/ 678] blk.11.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 193/ 678] blk.11.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 194/ 678] blk.11.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 195/ 678] blk.11.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 196/ 678] blk.11.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 197/ 678] blk.11.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 198/ 678] blk.11.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 199/ 678] blk.11.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 200/ 678] blk.11.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 201/ 678] blk.11.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 202/ 678] blk.11.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 203/ 678] blk.12.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 204/ 678] blk.12.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 205/ 678] blk.12.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 206/ 678] blk.12.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 207/ 678] blk.12.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 208/ 678] blk.12.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 209/ 678] blk.12.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 210/ 678] blk.12.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 211/ 678] blk.12.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 212/ 678] blk.12.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 213/ 678] blk.12.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 214/ 678] blk.12.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 215/ 678] blk.12.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 216/ 678] blk.12.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 217/ 678] blk.12.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 218/ 678] blk.12.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 219/ 678] blk.12.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 220/ 678] blk.13.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 221/ 678] blk.13.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 222/ 678] blk.13.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 223/ 678] blk.13.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 224/ 678] blk.13.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 225/ 678] blk.13.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 226/ 678] blk.13.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 227/ 678] blk.13.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 228/ 678] blk.13.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 229/ 678] blk.13.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 230/ 678] blk.13.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 231/ 678] blk.13.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 232/ 678] blk.13.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 233/ 678] blk.13.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 234/ 678] blk.13.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 235/ 678] blk.13.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 236/ 678] blk.13.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 237/ 678] blk.14.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 238/ 678] blk.14.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 239/ 678] blk.14.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 240/ 678] blk.14.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 241/ 678] blk.14.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 242/ 678] blk.14.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 243/ 678] blk.14.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 244/ 678] blk.14.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 245/ 678] blk.14.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 246/ 678] blk.14.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 247/ 678] blk.14.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 248/ 678] blk.14.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 249/ 678] blk.14.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 250/ 678] blk.14.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 251/ 678] blk.14.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 252/ 678] blk.14.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 253/ 678] blk.14.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 254/ 678] blk.15.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 255/ 678] blk.15.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 256/ 678] blk.15.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 257/ 678] blk.15.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 258/ 678] blk.15.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 259/ 678] blk.15.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 260/ 678] blk.15.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 261/ 678] blk.15.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 262/ 678] blk.15.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 263/ 678] blk.15.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 264/ 678] blk.15.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 265/ 678] blk.15.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 266/ 678] blk.15.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 267/ 678] blk.15.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 268/ 678] blk.15.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 269/ 678] blk.15.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 270/ 678] blk.15.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 271/ 678] blk.16.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 272/ 678] blk.16.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 273/ 678] blk.16.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 274/ 678] blk.16.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 275/ 678] blk.16.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 276/ 678] blk.16.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 277/ 678] blk.16.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 278/ 678] blk.16.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 279/ 678] blk.16.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 280/ 678] blk.16.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 281/ 678] blk.16.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 282/ 678] blk.16.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 283/ 678] blk.16.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 284/ 678] blk.16.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 285/ 678] blk.16.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 286/ 678] blk.16.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 287/ 678] blk.16.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 288/ 678] blk.17.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 289/ 678] blk.17.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 290/ 678] blk.17.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 291/ 678] blk.17.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 292/ 678] blk.17.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 293/ 678] blk.17.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 294/ 678] blk.17.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 295/ 678] blk.17.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 296/ 678] blk.17.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 297/ 678] blk.17.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 298/ 678] blk.17.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 299/ 678] blk.17.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 300/ 678] blk.17.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 301/ 678] blk.17.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 302/ 678] blk.17.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 303/ 678] blk.17.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 304/ 678] blk.17.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 305/ 678] blk.18.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 306/ 678] blk.18.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 307/ 678] blk.18.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 308/ 678] blk.18.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 309/ 678] blk.18.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 310/ 678] blk.18.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 311/ 678] blk.18.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 312/ 678] blk.18.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 313/ 678] blk.18.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 314/ 678] blk.18.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 315/ 678] blk.18.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 316/ 678] blk.18.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 317/ 678] blk.18.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 318/ 678] blk.18.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 319/ 678] blk.18.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 320/ 678] blk.18.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 321/ 678] blk.18.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 322/ 678] blk.19.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 323/ 678] blk.19.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 324/ 678] blk.19.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 325/ 678] blk.19.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 326/ 678] blk.19.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 327/ 678] blk.19.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 328/ 678] blk.19.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 329/ 678] blk.19.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 330/ 678] blk.19.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 331/ 678] blk.19.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 332/ 678] blk.19.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 333/ 678] blk.19.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 334/ 678] blk.19.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 335/ 678] blk.19.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 336/ 678] blk.19.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 337/ 678] blk.19.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 338/ 678] blk.19.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 339/ 678] blk.20.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 340/ 678] blk.20.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 341/ 678] blk.20.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 342/ 678] blk.20.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 343/ 678] blk.20.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 344/ 678] blk.20.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 345/ 678] blk.20.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 346/ 678] blk.20.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 347/ 678] blk.20.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 348/ 678] blk.20.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 349/ 678] blk.20.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 350/ 678] blk.20.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 351/ 678] blk.20.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 352/ 678] blk.20.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 353/ 678] blk.20.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 354/ 678] blk.20.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 355/ 678] blk.20.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 356/ 678] blk.21.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 357/ 678] blk.21.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 358/ 678] blk.21.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 359/ 678] blk.21.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 360/ 678] blk.21.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 361/ 678] blk.21.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 362/ 678] blk.21.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 363/ 678] blk.21.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 364/ 678] blk.21.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 365/ 678] blk.21.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 366/ 678] blk.21.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 367/ 678] blk.21.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 368/ 678] blk.21.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 369/ 678] blk.21.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 370/ 678] blk.21.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 371/ 678] blk.21.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 372/ 678] blk.21.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 373/ 678] blk.22.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 374/ 678] blk.22.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 375/ 678] blk.22.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 376/ 678] blk.22.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 377/ 678] blk.22.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 378/ 678] blk.22.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 379/ 678] blk.22.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 380/ 678] blk.22.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 381/ 678] blk.22.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 382/ 678] blk.22.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 383/ 678] blk.22.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 384/ 678] blk.22.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 385/ 678] blk.22.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 386/ 678] blk.22.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 387/ 678] blk.22.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 388/ 678] blk.22.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 389/ 678] blk.22.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 390/ 678] blk.23.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 391/ 678] blk.23.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 392/ 678] blk.23.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 393/ 678] blk.23.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 394/ 678] blk.23.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 395/ 678] blk.23.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 396/ 678] blk.23.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 397/ 678] blk.23.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 398/ 678] blk.23.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 399/ 678] blk.23.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 400/ 678] blk.23.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 401/ 678] blk.23.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 402/ 678] blk.23.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 403/ 678] blk.23.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 404/ 678] blk.23.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 405/ 678] blk.23.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 406/ 678] blk.23.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 407/ 678] blk.24.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 408/ 678] blk.24.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 409/ 678] blk.24.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 410/ 678] blk.24.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 411/ 678] blk.24.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 412/ 678] blk.24.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 413/ 678] blk.24.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 414/ 678] blk.24.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 415/ 678] blk.24.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 416/ 678] blk.24.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 417/ 678] blk.24.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 418/ 678] blk.24.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 419/ 678] blk.24.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 420/ 678] blk.24.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 421/ 678] blk.24.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 422/ 678] blk.24.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 423/ 678] blk.24.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 424/ 678] blk.25.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 425/ 678] blk.25.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 426/ 678] blk.25.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 427/ 678] blk.25.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 428/ 678] blk.25.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 429/ 678] blk.25.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 430/ 678] blk.25.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 431/ 678] blk.25.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 432/ 678] blk.25.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 433/ 678] blk.25.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 434/ 678] blk.25.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 435/ 678] blk.25.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 436/ 678] blk.25.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 437/ 678] blk.25.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 438/ 678] blk.25.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 439/ 678] blk.25.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 440/ 678] blk.25.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 441/ 678] blk.26.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 442/ 678] blk.26.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 443/ 678] blk.26.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 444/ 678] blk.26.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 445/ 678] blk.26.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 446/ 678] blk.26.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 447/ 678] blk.26.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 448/ 678] blk.26.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 449/ 678] blk.26.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 450/ 678] blk.26.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 451/ 678] blk.26.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 452/ 678] blk.26.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 453/ 678] blk.26.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 454/ 678] blk.26.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 455/ 678] blk.26.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 456/ 678] blk.26.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 457/ 678] blk.26.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 458/ 678] blk.27.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 459/ 678] blk.27.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 460/ 678] blk.27.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 461/ 678] blk.27.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 462/ 678] blk.27.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 463/ 678] blk.27.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 464/ 678] blk.27.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 465/ 678] blk.27.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 466/ 678] blk.27.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 467/ 678] blk.27.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 468/ 678] blk.27.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 469/ 678] blk.27.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 470/ 678] blk.27.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 471/ 678] blk.27.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 472/ 678] blk.27.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 473/ 678] blk.27.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 474/ 678] blk.27.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 475/ 678] blk.28.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 476/ 678] blk.28.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 477/ 678] blk.28.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 478/ 678] blk.28.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 479/ 678] blk.28.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 480/ 678] blk.28.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 481/ 678] blk.28.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 482/ 678] blk.28.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 483/ 678] blk.28.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 484/ 678] blk.28.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 485/ 678] blk.28.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 486/ 678] blk.28.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 487/ 678] blk.28.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 488/ 678] blk.28.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 489/ 678] blk.28.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 490/ 678] blk.28.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 491/ 678] blk.28.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 492/ 678] blk.29.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 493/ 678] blk.29.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 494/ 678] blk.29.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 495/ 678] blk.29.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 496/ 678] blk.29.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 497/ 678] blk.29.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 498/ 678] blk.29.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 499/ 678] blk.29.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 500/ 678] blk.29.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 501/ 678] blk.29.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 502/ 678] blk.29.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 503/ 678] blk.29.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 504/ 678] blk.29.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 505/ 678] blk.29.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 506/ 678] blk.29.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 507/ 678] blk.29.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 508/ 678] blk.29.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 509/ 678] blk.30.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 510/ 678] blk.30.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 511/ 678] blk.30.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 512/ 678] blk.30.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 513/ 678] blk.30.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 514/ 678] blk.30.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 515/ 678] blk.30.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 516/ 678] blk.30.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 517/ 678] blk.30.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 518/ 678] blk.30.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 519/ 678] blk.30.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 520/ 678] blk.30.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 521/ 678] blk.30.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 522/ 678] blk.30.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 523/ 678] blk.30.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 524/ 678] blk.30.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 525/ 678] blk.30.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 526/ 678] blk.31.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 527/ 678] blk.31.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 528/ 678] blk.31.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 529/ 678] blk.31.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 530/ 678] blk.31.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 531/ 678] blk.31.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 532/ 678] blk.31.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 533/ 678] blk.31.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 534/ 678] blk.31.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 535/ 678] blk.31.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 536/ 678] blk.31.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 537/ 678] blk.31.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 538/ 678] blk.31.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 539/ 678] blk.31.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 540/ 678] blk.31.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 541/ 678] blk.31.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 542/ 678] blk.31.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 543/ 678] blk.32.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 544/ 678] blk.32.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 545/ 678] blk.32.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 546/ 678] blk.32.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 547/ 678] blk.32.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 548/ 678] blk.32.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 549/ 678] blk.32.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 550/ 678] blk.32.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 551/ 678] blk.32.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 552/ 678] blk.32.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 553/ 678] blk.32.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 554/ 678] blk.32.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 555/ 678] blk.32.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 556/ 678] blk.32.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 557/ 678] blk.32.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 558/ 678] blk.32.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 559/ 678] blk.32.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 560/ 678] blk.33.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 561/ 678] blk.33.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 562/ 678] blk.33.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 563/ 678] blk.33.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 564/ 678] blk.33.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 565/ 678] blk.33.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 566/ 678] blk.33.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 567/ 678] blk.33.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 568/ 678] blk.33.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 569/ 678] blk.33.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 570/ 678] blk.33.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 571/ 678] blk.33.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 572/ 678] blk.33.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 573/ 678] blk.33.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 574/ 678] blk.33.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 575/ 678] blk.33.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 576/ 678] blk.33.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 577/ 678] blk.34.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 578/ 678] blk.34.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 579/ 678] blk.34.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 580/ 678] blk.34.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 581/ 678] blk.34.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 582/ 678] blk.34.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 583/ 678] blk.34.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 584/ 678] blk.34.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 585/ 678] blk.34.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 586/ 678] blk.34.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 587/ 678] blk.34.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 588/ 678] blk.34.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 589/ 678] blk.34.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 590/ 678] blk.34.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 591/ 678] blk.34.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 592/ 678] blk.34.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 593/ 678] blk.34.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 594/ 678] blk.35.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 595/ 678] blk.35.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 596/ 678] blk.35.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 597/ 678] blk.35.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 598/ 678] blk.35.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 599/ 678] blk.35.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 600/ 678] blk.35.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 601/ 678] blk.35.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 602/ 678] blk.35.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 603/ 678] blk.35.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 604/ 678] blk.35.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 605/ 678] blk.35.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 606/ 678] blk.35.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 607/ 678] blk.35.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 608/ 678] blk.35.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 609/ 678] blk.35.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 610/ 678] blk.35.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 611/ 678] blk.36.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 612/ 678] blk.36.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 613/ 678] blk.36.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 614/ 678] blk.36.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 615/ 678] blk.36.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 616/ 678] blk.36.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 617/ 678] blk.36.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 618/ 678] blk.36.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 619/ 678] blk.36.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 620/ 678] blk.36.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 621/ 678] blk.36.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 622/ 678] blk.36.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 623/ 678] blk.36.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 624/ 678] blk.36.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 625/ 678] blk.36.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 626/ 678] blk.36.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 627/ 678] blk.36.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 628/ 678] blk.37.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 629/ 678] blk.37.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 630/ 678] blk.37.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 631/ 678] blk.37.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 632/ 678] blk.37.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 633/ 678] blk.37.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 634/ 678] blk.37.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 635/ 678] blk.37.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 636/ 678] blk.37.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 637/ 678] blk.37.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 638/ 678] blk.37.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 639/ 678] blk.37.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 640/ 678] blk.37.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 641/ 678] blk.37.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 642/ 678] blk.37.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 643/ 678] blk.37.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 644/ 678] blk.37.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 645/ 678] blk.38.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 646/ 678] blk.38.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 647/ 678] blk.38.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 648/ 678] blk.38.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 649/ 678] blk.38.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 650/ 678] blk.38.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 651/ 678] blk.38.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 652/ 678] blk.38.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 653/ 678] blk.38.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 654/ 678] blk.38.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 655/ 678] blk.38.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 656/ 678] blk.38.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 657/ 678] blk.38.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 658/ 678] blk.38.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 659/ 678] blk.38.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 660/ 678] blk.38.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 661/ 678] blk.38.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 662/ 678] blk.39.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 663/ 678] blk.39.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 664/ 678] blk.39.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 665/ 678] blk.39.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 666/ 678] blk.39.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 667/ 678] blk.39.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 668/ 678] blk.39.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 669/ 678] blk.39.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 670/ 678] blk.39.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 671/ 678] blk.39.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 672/ 678] blk.39.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 673/ 678] blk.39.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 674/ 678] blk.39.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 675/ 678] blk.39.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 676/ 678] blk.39.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 677/ 678] blk.39.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q8_0 .. size = 512.00 MiB -> 272.00 MiB
[ 678/ 678] blk.39.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
llama_model_quantize_impl: model size = 63826.08 MiB (16.01 BPW)
llama_model_quantize_impl: quant size = 33944.50 MiB (8.51 BPW)
llama_quantize: quantize time = 73160.52 ms
llama_quantize: total time = 73160.52 ms
+ ./llama.cpp/build/bin/llama-quantize --pure --tensor-type output.weight=q6_k --tensor-type shexp=q8_0 --tensor-type attn_=q8_0 ./upload-Laguna_XS_2.1/Laguna-XS-2.1-BF16.gguf ./upload-Laguna_XS_2.1/Laguna-XS-2.1-Q4_K_M.gguf Q4_K_M
version: 0.2.0-dev (build 1, commit 70adb1b)
built with GNU 14.2.0 for Linux x86_64
llama_quantize: quantizing './upload-Laguna_XS_2.1/Laguna-XS-2.1-BF16.gguf' to './upload-Laguna_XS_2.1/Laguna-XS-2.1-Q4_K_M.gguf' as Q4_K_M
llama_model_loader: loaded meta data with 56 key-value pairs and 678 tensors from ./upload-Laguna_XS_2.1/Laguna-XS-2.1-BF16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = laguna
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.sampling.top_k i32 = 20
llama_model_loader: - kv 3: general.sampling.top_p f32 = 1.000000
llama_model_loader: - kv 4: general.sampling.min_p f32 = 0.000000
llama_model_loader: - kv 5: general.sampling.temp f32 = 1.000000
llama_model_loader: - kv 6: general.name str = Laguna-XS-2.1
llama_model_loader: - kv 7: general.size_label str = 256x2.2B
llama_model_loader: - kv 8: general.license str = openmdw-1.1
llama_model_loader: - kv 9: general.tags arr[str,3] = ["laguna-xs-2.1", "vllm", "text-gener...
llama_model_loader: - kv 10: laguna.block_count u32 = 40
llama_model_loader: - kv 11: laguna.context_length u32 = 262144
llama_model_loader: - kv 12: laguna.embedding_length u32 = 2048
llama_model_loader: - kv 13: laguna.feed_forward_length u32 = 8192
llama_model_loader: - kv 14: laguna.attention.head_count arr[i32,40] = [48, 64, 64, 64, 48, 64, 64, 64, 48, ...
llama_model_loader: - kv 15: laguna.attention.head_count_kv u32 = 8
llama_model_loader: - kv 16: laguna.rope.scaling.type str = yarn
llama_model_loader: - kv 17: laguna.rope.scaling.factor f32 = 32.000000
llama_model_loader: - kv 18: laguna.rope.scaling.original_context_length u32 = 8192
llama_model_loader: - kv 19: laguna.rope.scaling.yarn_attn_factor f32 = 1.346574
llama_model_loader: - kv 20: laguna.rope.scaling.yarn_beta_fast f32 = 64.000000
llama_model_loader: - kv 21: laguna.rope.scaling.yarn_beta_slow f32 = 1.000000
llama_model_loader: - kv 22: laguna.rope.freq_base f32 = 500000.000000
llama_model_loader: - kv 23: laguna.rope.freq_base_swa f32 = 10000.000000
llama_model_loader: - kv 24: laguna.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 25: laguna.expert_count u32 = 256
llama_model_loader: - kv 26: laguna.expert_used_count u32 = 8
llama_model_loader: - kv 27: laguna.attention.key_length u32 = 128
llama_model_loader: - kv 28: laguna.attention.value_length u32 = 128
llama_model_loader: - kv 29: general.file_type u32 = 32
llama_model_loader: - kv 30: laguna.vocab_size u32 = 100352
llama_model_loader: - kv 31: laguna.attention.sliding_window u32 = 512
llama_model_loader: - kv 32: laguna.expert_feed_forward_length u32 = 512
llama_model_loader: - kv 33: laguna.expert_shared_feed_forward_length u32 = 512
llama_model_loader: - kv 34: laguna.expert_weights_norm bool = true
llama_model_loader: - kv 35: laguna.expert_weights_scale f32 = 2.500000
llama_model_loader: - kv 36: laguna.expert_gating_func u32 = 2
llama_model_loader: - kv 37: laguna.leading_dense_block_count u32 = 1
llama_model_loader: - kv 38: laguna.rope.dimension_count u32 = 64
llama_model_loader: - kv 39: laguna.rope.dimension_count_swa u32 = 128
llama_model_loader: - kv 40: general.quantization_version u32 = 2
llama_model_loader: - kv 41: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 42: tokenizer.ggml.pre str = laguna
llama_model_loader: - kv 43: tokenizer.ggml.tokens arr[str,100352] = ["γ€ˆ|UNK|〉", "γ€ˆ|CODE_START|〉",...
llama_model_loader: - kv 44: tokenizer.ggml.token_type arr[i32,100352] = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ...
llama_model_loader: - kv 45: tokenizer.ggml.merges arr[str,100026] = ["i n", "Δ  t", "Δ  Δ ", "e r", "Δ  a...
llama_model_loader: - kv 46: tokenizer.ggml.bos_token_id u32 = 2
llama_model_loader: - kv 47: tokenizer.ggml.eos_token_id u32 = 2
llama_model_loader: - kv 48: tokenizer.ggml.unknown_token_id u32 = 0
llama_model_loader: - kv 49: tokenizer.ggml.seperator_token_id u32 = 8
llama_model_loader: - kv 50: tokenizer.ggml.padding_token_id u32 = 9
llama_model_loader: - kv 51: tokenizer.ggml.mask_token_id u32 = 12
llama_model_loader: - kv 52: tokenizer.ggml.add_bos_token bool = true
llama_model_loader: - kv 53: tokenizer.ggml.add_sep_token bool = true
llama_model_loader: - kv 54: tokenizer.chat_template str = {#- Iteration on laguna_glm_thinking_...
llama_model_loader: - kv 55: tokenizer.ggml.eot_token_id u32 = 24
llama_model_loader: - type f32: 239 tensors
llama_model_loader: - type bf16: 439 tensors
llama_tensor_get_type: output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.0.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.0.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.0.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.0.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.0.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.1.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.1.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.1.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.1.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.1.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.1.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.1.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.1.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.2.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.2.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.2.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.2.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.2.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.2.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.2.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.2.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.3.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.3.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.3.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.3.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.3.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.3.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.3.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.3.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.4.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.4.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.4.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.4.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.4.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.4.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.4.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.4.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.5.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.5.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.5.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.5.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.5.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.5.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.5.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.5.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.6.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.6.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.6.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.6.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.6.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.6.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.6.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.6.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.7.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.7.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.7.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.7.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.7.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.7.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.7.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.7.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.8.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.8.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.8.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.8.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.8.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.8.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.8.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.8.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.9.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.9.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.9.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.9.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.9.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.9.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.9.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.9.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.10.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.10.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.10.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.10.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.10.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.10.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.10.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.10.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.11.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.11.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.11.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.11.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.11.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.11.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.11.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.11.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.12.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.12.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.12.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.12.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.12.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.12.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.12.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.12.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.13.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.13.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.13.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.13.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.13.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.13.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.13.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.13.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.14.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.14.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.14.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.14.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.14.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.14.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.14.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.14.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.15.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.15.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.15.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.15.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.15.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.15.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.15.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.15.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.16.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.16.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.16.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.16.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.16.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.16.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.16.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.16.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.17.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.17.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.17.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.17.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.17.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.17.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.17.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.17.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.18.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.18.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.18.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.18.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.18.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.18.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.18.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.18.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.19.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.19.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.19.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.19.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.19.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.19.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.19.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.19.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.20.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.20.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.20.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.20.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.20.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.20.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.20.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.20.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.21.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.21.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.21.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.21.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.21.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.21.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.21.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.21.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.22.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.22.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.22.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.22.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.22.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.22.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.22.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.22.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.23.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.23.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.23.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.23.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.23.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.23.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.23.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.23.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.24.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.24.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.24.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.24.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.24.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.24.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.24.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.24.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.25.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.25.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.25.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.25.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.25.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.25.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.25.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.25.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.26.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.26.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.26.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.26.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.26.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.26.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.26.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.26.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.27.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.27.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.27.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.27.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.27.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.27.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.27.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.27.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.28.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.28.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.28.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.28.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.28.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.28.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.28.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.28.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.29.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.29.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.29.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.29.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.29.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.29.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.29.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.29.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.30.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.30.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.30.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.30.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.30.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.30.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.30.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.30.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.31.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.31.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.31.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.31.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.31.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.31.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.31.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.31.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.32.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.32.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.32.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.32.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.32.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.32.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.32.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.32.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.33.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.33.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.33.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.33.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.33.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.33.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.33.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.33.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.34.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.34.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.34.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.34.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.34.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.34.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.34.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.34.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.35.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.35.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.35.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.35.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.35.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.35.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.35.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.35.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.36.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.36.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.36.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.36.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.36.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.36.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.36.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.36.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.37.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.37.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.37.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.37.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.37.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.37.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.37.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.37.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.38.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.38.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.38.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.38.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.38.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.38.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.38.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.38.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.39.attn_gate.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.39.attn_k.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.39.attn_output.weight - applying manual override: q4_K -> q6_K
llama_tensor_get_type: blk.39.attn_q.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.39.attn_v.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.39.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.39.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0
llama_tensor_get_type: blk.39.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0
[ 1/ 678] output.weight - [ 2048, 100352, 1, 1], type = bf16, converting to q6_K .. size = 392.00 MiB -> 160.78 MiB
[ 2/ 678] output_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 3/ 678] token_embd.weight - [ 2048, 100352, 1, 1], type = bf16, converting to q4_K .. size = 392.00 MiB -> 110.25 MiB
[ 4/ 678] blk.0.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 5/ 678] blk.0.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 6/ 678] blk.0.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 7/ 678] blk.0.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 8/ 678] blk.0.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q6_K .. size = 24.00 MiB -> 9.84 MiB
[ 9/ 678] blk.0.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 10/ 678] blk.0.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 11/ 678] blk.0.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 12/ 678] blk.0.ffn_down.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q4_K .. size = 32.00 MiB -> 9.00 MiB
[ 13/ 678] blk.0.ffn_gate.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q4_K .. size = 32.00 MiB -> 9.00 MiB
[ 14/ 678] blk.0.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 15/ 678] blk.0.ffn_up.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q4_K .. size = 32.00 MiB -> 9.00 MiB
[ 16/ 678] blk.1.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 17/ 678] blk.1.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 18/ 678] blk.1.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 19/ 678] blk.1.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 20/ 678] blk.1.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 21/ 678] blk.1.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 22/ 678] blk.1.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 23/ 678] blk.1.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 24/ 678] blk.1.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 25/ 678] blk.1.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 26/ 678] blk.1.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 27/ 678] blk.1.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 28/ 678] blk.1.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 29/ 678] blk.1.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 30/ 678] blk.1.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 31/ 678] blk.1.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 32/ 678] blk.1.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 33/ 678] blk.2.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 34/ 678] blk.2.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 35/ 678] blk.2.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 36/ 678] blk.2.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 37/ 678] blk.2.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 38/ 678] blk.2.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 39/ 678] blk.2.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 40/ 678] blk.2.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 41/ 678] blk.2.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 42/ 678] blk.2.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 43/ 678] blk.2.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 44/ 678] blk.2.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 45/ 678] blk.2.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 46/ 678] blk.2.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 47/ 678] blk.2.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 48/ 678] blk.2.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 49/ 678] blk.2.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 50/ 678] blk.3.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 51/ 678] blk.3.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 52/ 678] blk.3.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 53/ 678] blk.3.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 54/ 678] blk.3.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 55/ 678] blk.3.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 56/ 678] blk.3.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 57/ 678] blk.3.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 58/ 678] blk.3.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 59/ 678] blk.3.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 60/ 678] blk.3.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 61/ 678] blk.3.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 62/ 678] blk.3.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 63/ 678] blk.3.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 64/ 678] blk.3.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 65/ 678] blk.3.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 66/ 678] blk.3.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 67/ 678] blk.4.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 68/ 678] blk.4.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 69/ 678] blk.4.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 70/ 678] blk.4.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 71/ 678] blk.4.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q6_K .. size = 24.00 MiB -> 9.84 MiB
[ 72/ 678] blk.4.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 73/ 678] blk.4.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 74/ 678] blk.4.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 75/ 678] blk.4.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 76/ 678] blk.4.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 77/ 678] blk.4.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 78/ 678] blk.4.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 79/ 678] blk.4.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 80/ 678] blk.4.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 81/ 678] blk.4.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 82/ 678] blk.4.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 83/ 678] blk.4.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 84/ 678] blk.5.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 85/ 678] blk.5.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 86/ 678] blk.5.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 87/ 678] blk.5.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 88/ 678] blk.5.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 89/ 678] blk.5.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 90/ 678] blk.5.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 91/ 678] blk.5.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 92/ 678] blk.5.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 93/ 678] blk.5.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 94/ 678] blk.5.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 95/ 678] blk.5.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 96/ 678] blk.5.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 97/ 678] blk.5.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 98/ 678] blk.5.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 99/ 678] blk.5.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 100/ 678] blk.5.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 101/ 678] blk.6.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 102/ 678] blk.6.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 103/ 678] blk.6.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 104/ 678] blk.6.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 105/ 678] blk.6.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 106/ 678] blk.6.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 107/ 678] blk.6.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 108/ 678] blk.6.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 109/ 678] blk.6.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 110/ 678] blk.6.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 111/ 678] blk.6.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 112/ 678] blk.6.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 113/ 678] blk.6.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 114/ 678] blk.6.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 115/ 678] blk.6.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 116/ 678] blk.6.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 117/ 678] blk.6.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 118/ 678] blk.7.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 119/ 678] blk.7.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 120/ 678] blk.7.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 121/ 678] blk.7.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 122/ 678] blk.7.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 123/ 678] blk.7.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 124/ 678] blk.7.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 125/ 678] blk.7.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 126/ 678] blk.7.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 127/ 678] blk.7.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 128/ 678] blk.7.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 129/ 678] blk.7.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 130/ 678] blk.7.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 131/ 678] blk.7.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 132/ 678] blk.7.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 133/ 678] blk.7.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 134/ 678] blk.7.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 135/ 678] blk.8.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 136/ 678] blk.8.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 137/ 678] blk.8.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 138/ 678] blk.8.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 139/ 678] blk.8.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q6_K .. size = 24.00 MiB -> 9.84 MiB
[ 140/ 678] blk.8.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 141/ 678] blk.8.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 142/ 678] blk.8.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 143/ 678] blk.8.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 144/ 678] blk.8.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 145/ 678] blk.8.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 146/ 678] blk.8.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 147/ 678] blk.8.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 148/ 678] blk.8.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 149/ 678] blk.8.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 150/ 678] blk.8.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 151/ 678] blk.8.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 152/ 678] blk.9.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 153/ 678] blk.9.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 154/ 678] blk.9.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 155/ 678] blk.9.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 156/ 678] blk.9.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 157/ 678] blk.9.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 158/ 678] blk.9.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 159/ 678] blk.9.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 160/ 678] blk.9.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 161/ 678] blk.9.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 162/ 678] blk.9.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 163/ 678] blk.9.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 164/ 678] blk.9.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 165/ 678] blk.9.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 166/ 678] blk.9.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 167/ 678] blk.9.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 168/ 678] blk.9.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 169/ 678] blk.10.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 170/ 678] blk.10.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 171/ 678] blk.10.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 172/ 678] blk.10.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 173/ 678] blk.10.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 174/ 678] blk.10.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 175/ 678] blk.10.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 176/ 678] blk.10.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 177/ 678] blk.10.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 178/ 678] blk.10.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 179/ 678] blk.10.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 180/ 678] blk.10.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 181/ 678] blk.10.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 182/ 678] blk.10.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 183/ 678] blk.10.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 184/ 678] blk.10.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 185/ 678] blk.10.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 186/ 678] blk.11.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 187/ 678] blk.11.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 188/ 678] blk.11.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 189/ 678] blk.11.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 190/ 678] blk.11.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 191/ 678] blk.11.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 192/ 678] blk.11.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 193/ 678] blk.11.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 194/ 678] blk.11.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 195/ 678] blk.11.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 196/ 678] blk.11.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 197/ 678] blk.11.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 198/ 678] blk.11.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 199/ 678] blk.11.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 200/ 678] blk.11.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 201/ 678] blk.11.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 202/ 678] blk.11.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 203/ 678] blk.12.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 204/ 678] blk.12.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 205/ 678] blk.12.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 206/ 678] blk.12.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 207/ 678] blk.12.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q6_K .. size = 24.00 MiB -> 9.84 MiB
[ 208/ 678] blk.12.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 209/ 678] blk.12.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 210/ 678] blk.12.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 211/ 678] blk.12.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 212/ 678] blk.12.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 213/ 678] blk.12.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 214/ 678] blk.12.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 215/ 678] blk.12.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 216/ 678] blk.12.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 217/ 678] blk.12.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 218/ 678] blk.12.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 219/ 678] blk.12.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 220/ 678] blk.13.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 221/ 678] blk.13.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 222/ 678] blk.13.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 223/ 678] blk.13.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 224/ 678] blk.13.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 225/ 678] blk.13.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 226/ 678] blk.13.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 227/ 678] blk.13.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 228/ 678] blk.13.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 229/ 678] blk.13.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 230/ 678] blk.13.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 231/ 678] blk.13.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 232/ 678] blk.13.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 233/ 678] blk.13.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 234/ 678] blk.13.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 235/ 678] blk.13.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 236/ 678] blk.13.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 237/ 678] blk.14.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 238/ 678] blk.14.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 239/ 678] blk.14.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 240/ 678] blk.14.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 241/ 678] blk.14.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 242/ 678] blk.14.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 243/ 678] blk.14.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 244/ 678] blk.14.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 245/ 678] blk.14.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 246/ 678] blk.14.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 247/ 678] blk.14.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 248/ 678] blk.14.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 249/ 678] blk.14.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 250/ 678] blk.14.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 251/ 678] blk.14.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 252/ 678] blk.14.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 253/ 678] blk.14.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 254/ 678] blk.15.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 255/ 678] blk.15.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 256/ 678] blk.15.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 257/ 678] blk.15.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 258/ 678] blk.15.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 259/ 678] blk.15.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 260/ 678] blk.15.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 261/ 678] blk.15.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 262/ 678] blk.15.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 263/ 678] blk.15.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 264/ 678] blk.15.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 265/ 678] blk.15.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 266/ 678] blk.15.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 267/ 678] blk.15.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 268/ 678] blk.15.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 269/ 678] blk.15.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 270/ 678] blk.15.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 271/ 678] blk.16.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 272/ 678] blk.16.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 273/ 678] blk.16.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 274/ 678] blk.16.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 275/ 678] blk.16.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q6_K .. size = 24.00 MiB -> 9.84 MiB
[ 276/ 678] blk.16.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 277/ 678] blk.16.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 278/ 678] blk.16.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 279/ 678] blk.16.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 280/ 678] blk.16.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 281/ 678] blk.16.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 282/ 678] blk.16.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 283/ 678] blk.16.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 284/ 678] blk.16.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 285/ 678] blk.16.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 286/ 678] blk.16.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 287/ 678] blk.16.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 288/ 678] blk.17.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 289/ 678] blk.17.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 290/ 678] blk.17.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 291/ 678] blk.17.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 292/ 678] blk.17.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 293/ 678] blk.17.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 294/ 678] blk.17.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 295/ 678] blk.17.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 296/ 678] blk.17.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 297/ 678] blk.17.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 298/ 678] blk.17.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 299/ 678] blk.17.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 300/ 678] blk.17.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 301/ 678] blk.17.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 302/ 678] blk.17.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 303/ 678] blk.17.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 304/ 678] blk.17.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 305/ 678] blk.18.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 306/ 678] blk.18.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 307/ 678] blk.18.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 308/ 678] blk.18.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 309/ 678] blk.18.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 310/ 678] blk.18.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 311/ 678] blk.18.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 312/ 678] blk.18.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 313/ 678] blk.18.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 314/ 678] blk.18.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 315/ 678] blk.18.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 316/ 678] blk.18.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 317/ 678] blk.18.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 318/ 678] blk.18.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 319/ 678] blk.18.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 320/ 678] blk.18.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 321/ 678] blk.18.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 322/ 678] blk.19.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 323/ 678] blk.19.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 324/ 678] blk.19.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 325/ 678] blk.19.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 326/ 678] blk.19.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 327/ 678] blk.19.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 328/ 678] blk.19.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 329/ 678] blk.19.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 330/ 678] blk.19.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 331/ 678] blk.19.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 332/ 678] blk.19.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 333/ 678] blk.19.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 334/ 678] blk.19.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 335/ 678] blk.19.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 336/ 678] blk.19.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 337/ 678] blk.19.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 338/ 678] blk.19.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 339/ 678] blk.20.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 340/ 678] blk.20.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 341/ 678] blk.20.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 342/ 678] blk.20.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 343/ 678] blk.20.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q6_K .. size = 24.00 MiB -> 9.84 MiB
[ 344/ 678] blk.20.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 345/ 678] blk.20.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 346/ 678] blk.20.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 347/ 678] blk.20.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 348/ 678] blk.20.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 349/ 678] blk.20.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 350/ 678] blk.20.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 351/ 678] blk.20.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 352/ 678] blk.20.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 353/ 678] blk.20.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 354/ 678] blk.20.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 355/ 678] blk.20.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 356/ 678] blk.21.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 357/ 678] blk.21.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 358/ 678] blk.21.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 359/ 678] blk.21.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 360/ 678] blk.21.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 361/ 678] blk.21.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 362/ 678] blk.21.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 363/ 678] blk.21.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 364/ 678] blk.21.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 365/ 678] blk.21.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 366/ 678] blk.21.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 367/ 678] blk.21.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 368/ 678] blk.21.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 369/ 678] blk.21.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 370/ 678] blk.21.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 371/ 678] blk.21.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 372/ 678] blk.21.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 373/ 678] blk.22.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 374/ 678] blk.22.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 375/ 678] blk.22.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 376/ 678] blk.22.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 377/ 678] blk.22.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 378/ 678] blk.22.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 379/ 678] blk.22.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 380/ 678] blk.22.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 381/ 678] blk.22.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 382/ 678] blk.22.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 383/ 678] blk.22.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 384/ 678] blk.22.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 385/ 678] blk.22.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 386/ 678] blk.22.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 387/ 678] blk.22.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 388/ 678] blk.22.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 389/ 678] blk.22.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 390/ 678] blk.23.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 391/ 678] blk.23.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 392/ 678] blk.23.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 393/ 678] blk.23.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 394/ 678] blk.23.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 395/ 678] blk.23.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 396/ 678] blk.23.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 397/ 678] blk.23.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 398/ 678] blk.23.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 399/ 678] blk.23.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 400/ 678] blk.23.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 401/ 678] blk.23.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 402/ 678] blk.23.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 403/ 678] blk.23.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 404/ 678] blk.23.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 405/ 678] blk.23.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 406/ 678] blk.23.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 407/ 678] blk.24.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 408/ 678] blk.24.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 409/ 678] blk.24.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 410/ 678] blk.24.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 411/ 678] blk.24.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q6_K .. size = 24.00 MiB -> 9.84 MiB
[ 412/ 678] blk.24.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 413/ 678] blk.24.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 414/ 678] blk.24.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 415/ 678] blk.24.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 416/ 678] blk.24.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 417/ 678] blk.24.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 418/ 678] blk.24.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 419/ 678] blk.24.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 420/ 678] blk.24.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 421/ 678] blk.24.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 422/ 678] blk.24.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 423/ 678] blk.24.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 424/ 678] blk.25.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 425/ 678] blk.25.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 426/ 678] blk.25.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 427/ 678] blk.25.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 428/ 678] blk.25.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 429/ 678] blk.25.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 430/ 678] blk.25.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 431/ 678] blk.25.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 432/ 678] blk.25.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 433/ 678] blk.25.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 434/ 678] blk.25.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 435/ 678] blk.25.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 436/ 678] blk.25.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 437/ 678] blk.25.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 438/ 678] blk.25.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 439/ 678] blk.25.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 440/ 678] blk.25.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 441/ 678] blk.26.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 442/ 678] blk.26.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 443/ 678] blk.26.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 444/ 678] blk.26.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 445/ 678] blk.26.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 446/ 678] blk.26.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 447/ 678] blk.26.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 448/ 678] blk.26.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 449/ 678] blk.26.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 450/ 678] blk.26.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 451/ 678] blk.26.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 452/ 678] blk.26.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 453/ 678] blk.26.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 454/ 678] blk.26.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 455/ 678] blk.26.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 456/ 678] blk.26.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 457/ 678] blk.26.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 458/ 678] blk.27.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 459/ 678] blk.27.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 460/ 678] blk.27.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 461/ 678] blk.27.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 462/ 678] blk.27.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 463/ 678] blk.27.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 464/ 678] blk.27.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 465/ 678] blk.27.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 466/ 678] blk.27.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 467/ 678] blk.27.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 468/ 678] blk.27.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 469/ 678] blk.27.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 470/ 678] blk.27.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 471/ 678] blk.27.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 472/ 678] blk.27.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 473/ 678] blk.27.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 474/ 678] blk.27.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 475/ 678] blk.28.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 476/ 678] blk.28.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 477/ 678] blk.28.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 478/ 678] blk.28.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 479/ 678] blk.28.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q6_K .. size = 24.00 MiB -> 9.84 MiB
[ 480/ 678] blk.28.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 481/ 678] blk.28.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 482/ 678] blk.28.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 483/ 678] blk.28.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 484/ 678] blk.28.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 485/ 678] blk.28.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 486/ 678] blk.28.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 487/ 678] blk.28.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 488/ 678] blk.28.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 489/ 678] blk.28.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 490/ 678] blk.28.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 491/ 678] blk.28.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 492/ 678] blk.29.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 493/ 678] blk.29.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 494/ 678] blk.29.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 495/ 678] blk.29.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 496/ 678] blk.29.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 497/ 678] blk.29.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 498/ 678] blk.29.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 499/ 678] blk.29.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 500/ 678] blk.29.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 501/ 678] blk.29.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 502/ 678] blk.29.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 503/ 678] blk.29.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 504/ 678] blk.29.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 505/ 678] blk.29.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 506/ 678] blk.29.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 507/ 678] blk.29.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 508/ 678] blk.29.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 509/ 678] blk.30.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 510/ 678] blk.30.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 511/ 678] blk.30.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 512/ 678] blk.30.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 513/ 678] blk.30.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 514/ 678] blk.30.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 515/ 678] blk.30.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 516/ 678] blk.30.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 517/ 678] blk.30.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 518/ 678] blk.30.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 519/ 678] blk.30.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 520/ 678] blk.30.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 521/ 678] blk.30.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 522/ 678] blk.30.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 523/ 678] blk.30.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 524/ 678] blk.30.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 525/ 678] blk.30.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 526/ 678] blk.31.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 527/ 678] blk.31.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 528/ 678] blk.31.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 529/ 678] blk.31.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 530/ 678] blk.31.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 531/ 678] blk.31.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 532/ 678] blk.31.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 533/ 678] blk.31.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 534/ 678] blk.31.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 535/ 678] blk.31.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 536/ 678] blk.31.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 537/ 678] blk.31.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 538/ 678] blk.31.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 539/ 678] blk.31.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 540/ 678] blk.31.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 541/ 678] blk.31.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 542/ 678] blk.31.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 543/ 678] blk.32.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 544/ 678] blk.32.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 545/ 678] blk.32.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 546/ 678] blk.32.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 547/ 678] blk.32.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q6_K .. size = 24.00 MiB -> 9.84 MiB
[ 548/ 678] blk.32.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 549/ 678] blk.32.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 550/ 678] blk.32.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 551/ 678] blk.32.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 552/ 678] blk.32.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 553/ 678] blk.32.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 554/ 678] blk.32.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 555/ 678] blk.32.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 556/ 678] blk.32.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 557/ 678] blk.32.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 558/ 678] blk.32.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 559/ 678] blk.32.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 560/ 678] blk.33.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 561/ 678] blk.33.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 562/ 678] blk.33.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 563/ 678] blk.33.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 564/ 678] blk.33.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 565/ 678] blk.33.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 566/ 678] blk.33.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 567/ 678] blk.33.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 568/ 678] blk.33.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 569/ 678] blk.33.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 570/ 678] blk.33.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 571/ 678] blk.33.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 572/ 678] blk.33.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 573/ 678] blk.33.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 574/ 678] blk.33.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 575/ 678] blk.33.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 576/ 678] blk.33.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 577/ 678] blk.34.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 578/ 678] blk.34.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 579/ 678] blk.34.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 580/ 678] blk.34.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 581/ 678] blk.34.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 582/ 678] blk.34.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 583/ 678] blk.34.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 584/ 678] blk.34.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 585/ 678] blk.34.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 586/ 678] blk.34.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 587/ 678] blk.34.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 588/ 678] blk.34.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 589/ 678] blk.34.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 590/ 678] blk.34.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 591/ 678] blk.34.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 592/ 678] blk.34.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 593/ 678] blk.34.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 594/ 678] blk.35.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 595/ 678] blk.35.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 596/ 678] blk.35.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 597/ 678] blk.35.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 598/ 678] blk.35.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 599/ 678] blk.35.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 600/ 678] blk.35.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 601/ 678] blk.35.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 602/ 678] blk.35.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 603/ 678] blk.35.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 604/ 678] blk.35.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 605/ 678] blk.35.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 606/ 678] blk.35.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 607/ 678] blk.35.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 608/ 678] blk.35.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 609/ 678] blk.35.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 610/ 678] blk.35.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 611/ 678] blk.36.attn_gate.weight - [ 2048, 48, 1, 1], type = bf16, converting to q8_0 .. size = 0.19 MiB -> 0.10 MiB
[ 612/ 678] blk.36.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 613/ 678] blk.36.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 614/ 678] blk.36.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 615/ 678] blk.36.attn_output.weight - [ 6144, 2048, 1, 1], type = bf16, converting to q6_K .. size = 24.00 MiB -> 9.84 MiB
[ 616/ 678] blk.36.attn_q.weight - [ 2048, 6144, 1, 1], type = bf16, converting to q8_0 .. size = 24.00 MiB -> 12.75 MiB
[ 617/ 678] blk.36.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 618/ 678] blk.36.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 619/ 678] blk.36.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 620/ 678] blk.36.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 621/ 678] blk.36.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 622/ 678] blk.36.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 623/ 678] blk.36.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 624/ 678] blk.36.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 625/ 678] blk.36.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 626/ 678] blk.36.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 627/ 678] blk.36.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 628/ 678] blk.37.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 629/ 678] blk.37.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 630/ 678] blk.37.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 631/ 678] blk.37.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 632/ 678] blk.37.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 633/ 678] blk.37.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 634/ 678] blk.37.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 635/ 678] blk.37.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 636/ 678] blk.37.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 637/ 678] blk.37.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 638/ 678] blk.37.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 639/ 678] blk.37.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 640/ 678] blk.37.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 641/ 678] blk.37.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 642/ 678] blk.37.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 643/ 678] blk.37.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 644/ 678] blk.37.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 645/ 678] blk.38.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 646/ 678] blk.38.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 647/ 678] blk.38.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 648/ 678] blk.38.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 649/ 678] blk.38.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 650/ 678] blk.38.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 651/ 678] blk.38.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 652/ 678] blk.38.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 653/ 678] blk.38.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 654/ 678] blk.38.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 655/ 678] blk.38.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 656/ 678] blk.38.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 657/ 678] blk.38.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 658/ 678] blk.38.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 659/ 678] blk.38.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 660/ 678] blk.38.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 661/ 678] blk.38.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 662/ 678] blk.39.attn_gate.weight - [ 2048, 64, 1, 1], type = bf16, converting to q8_0 .. size = 0.25 MiB -> 0.13 MiB
[ 663/ 678] blk.39.attn_k.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 664/ 678] blk.39.attn_k_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 665/ 678] blk.39.attn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 666/ 678] blk.39.attn_output.weight - [ 8192, 2048, 1, 1], type = bf16, converting to q6_K .. size = 32.00 MiB -> 13.12 MiB
[ 667/ 678] blk.39.attn_q.weight - [ 2048, 8192, 1, 1], type = bf16, converting to q8_0 .. size = 32.00 MiB -> 17.00 MiB
[ 668/ 678] blk.39.attn_q_norm.weight - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB
[ 669/ 678] blk.39.attn_v.weight - [ 2048, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 4.00 MiB -> 2.12 MiB
[ 670/ 678] blk.39.exp_probs_b.bias - [ 256, 1, 1, 1], type = f32, size = 0.001 MiB
[ 671/ 678] blk.39.ffn_down_exps.weight - [ 512, 2048, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 672/ 678] blk.39.ffn_down_shexp.weight - [ 512, 2048, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 673/ 678] blk.39.ffn_gate_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 674/ 678] blk.39.ffn_gate_inp.weight - [ 2048, 256, 1, 1], type = f32, size = 2.000 MiB
[ 675/ 678] blk.39.ffn_gate_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
[ 676/ 678] blk.39.ffn_norm.weight - [ 2048, 1, 1, 1], type = f32, size = 0.008 MiB
[ 677/ 678] blk.39.ffn_up_exps.weight - [ 2048, 512, 256, 1], type = bf16, converting to q4_K .. size = 512.00 MiB -> 144.00 MiB
[ 678/ 678] blk.39.ffn_up_shexp.weight - [ 2048, 512, 1, 1], type = bf16, converting to q8_0 .. size = 2.00 MiB -> 1.06 MiB
llama_model_quantize_impl: model size = 63826.08 MiB (16.01 BPW)
llama_model_quantize_impl: quant size = 18653.72 MiB (4.68 BPW)
llama_quantize: quantize time = 188782.91 ms
llama_quantize: total time = 188782.91 ms
+ echo Laguna-XS-2.1-BF16.gguf
+ echo Laguna-XS-2.1-Q8_0.gguf
+ echo Laguna-XS-2.1-Q4_K_M.gguf