+ OUTPUT_DIR=./upload-GLM_4.5_Air + LLAMA_CPP=./llama.cpp + DISPLAY_NAME=GLM-4.5-Air + QUANTIZE=./llama.cpp/build/bin/llama-quantize + python3 ./llama.cpp/convert_hf_to_gguf.py ./model-temp-GLM_4.5_Air-PRIMARY --no-tensor-first-split --outtype bf16 --outfile ./upload-GLM_4.5_Air/GLM-4.5-Air-BF16.gguf --no-mtp --model-name GLM-4.5-Air INFO:hf-to-gguf:Loading model: model-temp-GLM_4.5_Air-PRIMARY INFO:hf-to-gguf:Model architecture: Glm4MoeForCausalLM INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json' INFO:hf-to-gguf:gguf: indexing model part 'model-00001-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00002-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00003-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00004-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00005-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00006-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00007-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00008-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00009-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00010-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00011-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00012-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00013-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00014-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00015-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00016-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00017-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00018-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00019-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00020-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00021-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00022-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00023-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00024-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00025-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00026-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00027-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00028-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00029-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00030-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00031-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00032-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00033-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00034-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00035-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00036-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00037-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00038-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00039-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00040-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00041-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00042-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00043-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00044-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00045-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00046-of-00047.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model-00047-of-00047.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> BF16, shape = {4096, 151552} INFO:hf-to-gguf:blk.0.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.0.ffn_down.weight, torch.bfloat16 --> BF16, shape = {10944, 4096} INFO:hf-to-gguf:blk.0.ffn_gate.weight, torch.bfloat16 --> BF16, shape = {4096, 10944} INFO:hf-to-gguf:blk.0.ffn_up.weight, torch.bfloat16 --> BF16, shape = {4096, 10944} INFO:hf-to-gguf:blk.0.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.0.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.0.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.0.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.0.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.0.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.0.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.0.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.1.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.1.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.1.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.1.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.1.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.1.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.1.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.1.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.1.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.1.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.1.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.1.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.1.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.1.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.1.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.1.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.1.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.2.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.2.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.2.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.2.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.2.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.2.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.2.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.2.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.2.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.2.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.2.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.2.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.2.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.2.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.2.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.2.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.2.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.3.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.3.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.3.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.3.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.3.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.3.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.3.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.3.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.3.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.3.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.3.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.3.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.3.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.3.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.3.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.3.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.3.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.4.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.4.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.4.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.4.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.4.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.4.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.4.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.4.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.4.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.4.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.4.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.4.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.4.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.4.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.4.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.4.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.4.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.5.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.5.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.5.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.5.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.5.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.5.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.5.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.5.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.5.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.5.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.5.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.5.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.5.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.5.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.5.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.5.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.5.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.6.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.6.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.6.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.6.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.6.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.6.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.6.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.6.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.6.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.6.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.6.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.6.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.6.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.6.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.6.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.6.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.6.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.7.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.7.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.7.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.7.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.7.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.7.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.7.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.7.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.7.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.7.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.7.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.7.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.7.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.7.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.7.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.7.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.7.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.8.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.8.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.8.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.8.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.8.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.8.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.8.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.8.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.8.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.8.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.8.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.8.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.8.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.8.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.8.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.8.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.8.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.9.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.9.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.9.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.9.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.9.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.9.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.9.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.9.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.9.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.9.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.9.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.9.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.9.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.9.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.9.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.9.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.9.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.10.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.10.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.10.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.10.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.10.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.10.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.10.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.10.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.10.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.10.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.10.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.10.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.10.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.10.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.10.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.10.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.10.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.11.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.11.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.11.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.11.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.11.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.11.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.11.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.11.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.11.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.11.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.11.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.11.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.11.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.11.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.11.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.11.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.11.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.12.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.12.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.12.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.12.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.12.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.12.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.12.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.12.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.12.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.12.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.12.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.12.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.12.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.12.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.12.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.12.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.12.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.13.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.13.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.13.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.13.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.13.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.13.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.13.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.13.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.13.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.13.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.13.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.13.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.13.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.13.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.13.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.13.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.13.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.14.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.14.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.14.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.14.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.14.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.14.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.14.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.14.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.14.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.14.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.14.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.14.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.14.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.14.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.14.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.14.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.14.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.15.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.15.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.15.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.15.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.15.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.15.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.15.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.15.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.15.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.15.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.15.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.15.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.15.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.15.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.15.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.15.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.15.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.16.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.16.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.16.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.16.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.16.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.16.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.16.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.16.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.16.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.16.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.16.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.16.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.16.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.16.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.16.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.16.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.16.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.17.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.17.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.17.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.17.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.17.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.17.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.17.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.17.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.17.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.17.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.17.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.17.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.17.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.17.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.17.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.17.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.17.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.18.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.18.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.18.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.18.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.18.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.18.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.18.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.18.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.18.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.18.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.18.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.18.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.18.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.18.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.18.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.18.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.18.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.19.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.19.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.19.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.19.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.19.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.19.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.19.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.19.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.19.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.19.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.19.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.19.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.19.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.19.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.19.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.19.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.19.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.20.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.20.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.20.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.20.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.20.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.20.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.20.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.20.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.20.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.20.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.20.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.20.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.20.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.20.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.20.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.20.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.20.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.21.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.21.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.21.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.21.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.21.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.21.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.21.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.21.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.21.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.21.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.21.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.21.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.21.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.21.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.21.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.21.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.21.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.22.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.22.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.22.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.22.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.22.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.22.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.22.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.22.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.22.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.22.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.22.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.22.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.22.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.22.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.22.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.22.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.22.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.23.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.23.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.23.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.23.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.23.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.23.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.23.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.23.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.23.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.23.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.23.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.23.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.23.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.23.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.23.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.23.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.23.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.24.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.24.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.24.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.24.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.24.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.24.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.24.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.24.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.24.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.24.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.24.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.24.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.24.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.24.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.24.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.24.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.24.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.25.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.25.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.25.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.25.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.25.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.25.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.25.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.25.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.25.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.25.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.25.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.25.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.25.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.25.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.25.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.25.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.25.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.26.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.26.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.26.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.26.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.26.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.26.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.26.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.26.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.26.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.26.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.26.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.26.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.26.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.26.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.26.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.26.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.26.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.27.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.27.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.27.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.27.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.27.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.27.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.27.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.27.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.27.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.27.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.27.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.27.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.27.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.27.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.27.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.27.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.27.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.28.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.28.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.28.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.28.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.28.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.28.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.28.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.28.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.28.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.28.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.28.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.28.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.28.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.28.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.28.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.28.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.28.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.29.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.29.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.29.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.29.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.29.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.29.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.29.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.29.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.29.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.29.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.29.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.29.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.29.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.29.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.29.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.29.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.29.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.30.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.30.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.30.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.30.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.30.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.30.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.30.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.30.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.30.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.30.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.30.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.30.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.30.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.30.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.30.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.30.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.30.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.31.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.31.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.31.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.31.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.31.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.31.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.31.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.31.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.31.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.31.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.31.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.31.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.31.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.31.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.31.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.31.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.31.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.32.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.32.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.32.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.32.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.32.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.32.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.32.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.32.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.32.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.32.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.32.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.32.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.32.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.32.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.32.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.32.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.32.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.33.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.33.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.33.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.33.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.33.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.33.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.33.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.33.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.33.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.33.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.33.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.33.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.33.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.33.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.33.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.33.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.33.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.34.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.34.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.34.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.34.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.34.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.34.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.34.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.34.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.34.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.34.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.34.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.34.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.34.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.34.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.34.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.34.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.34.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.35.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.35.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.35.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.35.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.35.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.35.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.35.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.35.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.35.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.35.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.35.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.35.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.35.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.35.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.35.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.35.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.35.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.36.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.36.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.36.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.36.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.36.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.36.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.36.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.36.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.36.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.36.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.36.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.36.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.36.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.36.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.36.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.36.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.36.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.37.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.37.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.37.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.37.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.37.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.37.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.37.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.37.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.37.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.37.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.37.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.37.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.37.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.37.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.37.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.37.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.37.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.38.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.38.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.38.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.38.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.38.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.38.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.38.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.38.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.38.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.38.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.38.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.38.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.38.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.38.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.38.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.38.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.38.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.39.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.39.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.39.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.39.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.39.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.39.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.39.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.39.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.39.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.39.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.39.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.39.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.39.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.39.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.39.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.39.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.39.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.40.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.40.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.40.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.40.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.40.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.40.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.40.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.40.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.40.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.40.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.40.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.40.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.40.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.40.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.40.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.40.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.40.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.41.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.41.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.41.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.41.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.41.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.41.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.41.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.41.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.41.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.41.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.41.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.41.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.41.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.41.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.41.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.41.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.41.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.42.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.42.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.42.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.42.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.42.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.42.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.42.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.42.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.42.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.42.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.42.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.42.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.42.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.42.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.42.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.42.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.42.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.43.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.43.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.43.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.43.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.43.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.43.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.43.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.43.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.43.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.43.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.43.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.43.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.43.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.43.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.43.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.43.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.43.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.44.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.44.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.44.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.44.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.44.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.44.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.44.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.44.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.44.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.44.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.44.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.44.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.44.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.44.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.44.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.44.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.44.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:output.weight, torch.bfloat16 --> BF16, shape = {4096, 151552} INFO:hf-to-gguf:blk.45.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.45.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.45.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.45.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.45.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.45.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.45.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.45.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.45.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.45.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.45.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.45.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.45.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.45.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.45.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.45.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.45.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 131072 INFO:hf-to-gguf:gguf: embedding length = 4096 INFO:hf-to-gguf:gguf: feed forward length = 10944 INFO:hf-to-gguf:gguf: head count = 96 INFO:hf-to-gguf:gguf: key-value head count = 8 WARNING:hf-to-gguf:Unknown RoPE type: default INFO:hf-to-gguf:gguf: rope scaling type = NONE INFO:hf-to-gguf:gguf: rope theta = 1000000 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: expert count = 128 INFO:hf-to-gguf:gguf: experts used count = 8 INFO:hf-to-gguf:gguf: expert groups count = 1 INFO:hf-to-gguf:gguf: expert groups used count = 1 INFO:hf-to-gguf:gguf: file type = 32 WARNING:gguf.gguf_writer:Duplicated key name 'glm4moe.expert_count', overwriting it with new value 128 of type UINT32 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer INFO:gguf.vocab:Adding 318088 merge(s). INFO:gguf.vocab:Setting special token type eos to 151329 INFO:gguf.vocab:Setting special token type pad to 151329 INFO:gguf.vocab:Setting special token type bos to 151331 INFO:gguf.vocab:Setting special token type eot to 151336 INFO:gguf.vocab:Setting special token type unk to 151329 INFO:gguf.vocab:Setting special token type eom to 151338 INFO:gguf.vocab:Setting chat_template to [gMASK] {%- if tools -%} <|system|> # Tools You may call one or more functions to assist with the user query. You are provided with function signatures within XML tags: {% for tool in tools %} {{ tool | tojson(ensure_ascii=False) }} {% endfor %} For each function call, output the function name and arguments within the following XML format: {function-name} {arg-key-1} {arg-value-1} {arg-key-2} {arg-value-2} ... {%- endif -%} {%- macro visible_text(content) -%} {%- if content is string -%} {{- content }} {%- elif content is iterable and content is not mapping -%} {%- for item in content -%} {%- if item is mapping and item.type == 'text' -%} {{- item.text }} {%- elif item is string -%} {{- item }} {%- endif -%} {%- endfor -%} {%- else -%} {{- content }} {%- endif -%} {%- endmacro -%} {%- set ns = namespace(last_user_index=-1) %} {%- for m in messages %} {%- if m.role == 'user' %} {% set ns.last_user_index = loop.index0 -%} {%- endif %} {%- endfor %} {% for m in messages %} {%- if m.role == 'user' -%}<|user|> {{ visible_text(m.content) }} {{- '/nothink' if (enable_thinking is defined and not enable_thinking and not visible_text(m.content).endswith("/nothink")) else '' -}} {%- elif m.role == 'assistant' -%} <|assistant|> {%- set reasoning_content = '' %} {%- set content = visible_text(m.content) %} {%- if m.reasoning_content is string %} {%- set reasoning_content = m.reasoning_content %} {%- else %} {%- if '' in content %} {%- set reasoning_content = content.split('')[0].rstrip('\n').split('')[-1].lstrip('\n') %} {%- set content = content.split('')[-1].lstrip('\n') %} {%- endif %} {%- endif %} {%- if loop.index0 > ns.last_user_index and reasoning_content -%} {{ '\n' + reasoning_content.strip() + ''}} {%- else -%} {{ '\n' }} {%- endif -%} {%- if content.strip() -%} {{ '\n' + content.strip() }} {%- endif -%} {% if m.tool_calls %} {% for tc in m.tool_calls %} {%- if tc.function %} {%- set tc = tc.function %} {%- endif %} {{ '\n' + tc.name }} {% set _args = tc.arguments %} {% for k, v in _args.items() %} {{ k }} {{ v | tojson(ensure_ascii=False) if v is not string else v }} {% endfor %} {% endfor %} {% endif %} {%- elif m.role == 'tool' -%} {%- if m.content is string -%} {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %} {{- '<|observation|>' }} {%- endif %} {{- '\n\n' }} {{- m.content }} {{- '\n' }} {%- else -%} <|observation|>{% for tr in m.content %} {{ tr.output if tr.output is defined else tr }} {% endfor -%} {% endif -%} {%- elif m.role == 'system' -%} <|system|> {{ visible_text(m.content) }} {%- endif -%} {%- endfor -%} {%- if add_generation_prompt -%} <|assistant|>{{- '\n' if (enable_thinking is defined and not enable_thinking) else '' -}} {%- endif -%} INFO:gguf.gguf_writer:Writing the following files: INFO:gguf.gguf_writer:upload-GLM_4.5_Air/GLM-4.5-Air-BF16-00001-of-00002.gguf: n_tensors = 0, total_size = negligible - metadata only INFO:gguf.gguf_writer:upload-GLM_4.5_Air/GLM-4.5-Air-BF16-00002-of-00002.gguf: n_tensors = 780, total_size = 213.8G Shard (0/2): 0.00byte [00:00, ?byte/s] Writing: 0%| | 0.00/214G [00:00 BF16, shape = {4096, 151552} INFO:hf-to-gguf:blk.46.nextn.embed_tokens.weight, torch.bfloat16 --> BF16, shape = {4096, 151552} INFO:hf-to-gguf:output.weight, torch.bfloat16 --> BF16, shape = {4096, 151552} INFO:hf-to-gguf:blk.46.nextn.shared_head_head.weight, torch.bfloat16 --> BF16, shape = {4096, 151552} INFO:hf-to-gguf:output_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.46.nextn.eh_proj.weight, torch.bfloat16 --> BF16, shape = {8192, 4096} INFO:hf-to-gguf:blk.46.nextn.enorm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.46.nextn.hnorm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.46.attn_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.46.ffn_down_exps.weight, torch.bfloat16 --> BF16, shape = {1408, 4096, 128} INFO:hf-to-gguf:blk.46.ffn_gate_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.46.ffn_up_exps.weight, torch.bfloat16 --> BF16, shape = {4096, 1408, 128} INFO:hf-to-gguf:blk.46.exp_probs_b.bias, torch.float32 --> F32, shape = {128} INFO:hf-to-gguf:blk.46.ffn_gate_inp.weight, torch.bfloat16 --> F32, shape = {4096, 128} INFO:hf-to-gguf:blk.46.ffn_down_shexp.weight, torch.bfloat16 --> BF16, shape = {1408, 4096} INFO:hf-to-gguf:blk.46.ffn_gate_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.46.ffn_up_shexp.weight, torch.bfloat16 --> BF16, shape = {4096, 1408} INFO:hf-to-gguf:blk.46.post_attention_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:blk.46.attn_k.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.46.attn_k.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.46.attn_output.weight, torch.bfloat16 --> BF16, shape = {12288, 4096} INFO:hf-to-gguf:blk.46.attn_q.bias, torch.bfloat16 --> F32, shape = {12288} INFO:hf-to-gguf:blk.46.attn_q.weight, torch.bfloat16 --> BF16, shape = {4096, 12288} INFO:hf-to-gguf:blk.46.attn_v.bias, torch.bfloat16 --> F32, shape = {1024} INFO:hf-to-gguf:blk.46.attn_v.weight, torch.bfloat16 --> BF16, shape = {4096, 1024} INFO:hf-to-gguf:blk.46.nextn.shared_head_norm.weight, torch.bfloat16 --> F32, shape = {4096} INFO:hf-to-gguf:Set meta model INFO:hf-to-gguf:Set model parameters INFO:hf-to-gguf:gguf: context length = 131072 INFO:hf-to-gguf:gguf: embedding length = 4096 INFO:hf-to-gguf:gguf: feed forward length = 10944 INFO:hf-to-gguf:gguf: head count = 96 INFO:hf-to-gguf:gguf: key-value head count = 8 WARNING:hf-to-gguf:Unknown RoPE type: default INFO:hf-to-gguf:gguf: rope scaling type = NONE INFO:hf-to-gguf:gguf: rope theta = 1000000 INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05 INFO:hf-to-gguf:gguf: expert count = 128 INFO:hf-to-gguf:gguf: experts used count = 8 INFO:hf-to-gguf:gguf: expert groups count = 1 INFO:hf-to-gguf:gguf: expert groups used count = 1 INFO:hf-to-gguf:gguf: file type = 32 WARNING:gguf.gguf_writer:Duplicated key name 'glm4moe.expert_count', overwriting it with new value 128 of type UINT32 INFO:hf-to-gguf:Set model quantization version INFO:hf-to-gguf:Set model tokenizer INFO:gguf.vocab:Adding 318088 merge(s). INFO:gguf.vocab:Setting special token type eos to 151329 INFO:gguf.vocab:Setting special token type pad to 151329 INFO:gguf.vocab:Setting special token type bos to 151331 INFO:gguf.vocab:Setting special token type eot to 151336 INFO:gguf.vocab:Setting special token type unk to 151329 INFO:gguf.vocab:Setting special token type eom to 151338 INFO:gguf.vocab:Setting chat_template to [gMASK] {%- if tools -%} <|system|> # Tools You may call one or more functions to assist with the user query. You are provided with function signatures within XML tags: {% for tool in tools %} {{ tool | tojson(ensure_ascii=False) }} {% endfor %} For each function call, output the function name and arguments within the following XML format: {function-name} {arg-key-1} {arg-value-1} {arg-key-2} {arg-value-2} ... {%- endif -%} {%- macro visible_text(content) -%} {%- if content is string -%} {{- content }} {%- elif content is iterable and content is not mapping -%} {%- for item in content -%} {%- if item is mapping and item.type == 'text' -%} {{- item.text }} {%- elif item is string -%} {{- item }} {%- endif -%} {%- endfor -%} {%- else -%} {{- content }} {%- endif -%} {%- endmacro -%} {%- set ns = namespace(last_user_index=-1) %} {%- for m in messages %} {%- if m.role == 'user' %} {% set ns.last_user_index = loop.index0 -%} {%- endif %} {%- endfor %} {% for m in messages %} {%- if m.role == 'user' -%}<|user|> {{ visible_text(m.content) }} {{- '/nothink' if (enable_thinking is defined and not enable_thinking and not visible_text(m.content).endswith("/nothink")) else '' -}} {%- elif m.role == 'assistant' -%} <|assistant|> {%- set reasoning_content = '' %} {%- set content = visible_text(m.content) %} {%- if m.reasoning_content is string %} {%- set reasoning_content = m.reasoning_content %} {%- else %} {%- if '' in content %} {%- set reasoning_content = content.split('')[0].rstrip('\n').split('')[-1].lstrip('\n') %} {%- set content = content.split('')[-1].lstrip('\n') %} {%- endif %} {%- endif %} {%- if loop.index0 > ns.last_user_index and reasoning_content -%} {{ '\n' + reasoning_content.strip() + ''}} {%- else -%} {{ '\n' }} {%- endif -%} {%- if content.strip() -%} {{ '\n' + content.strip() }} {%- endif -%} {% if m.tool_calls %} {% for tc in m.tool_calls %} {%- if tc.function %} {%- set tc = tc.function %} {%- endif %} {{ '\n' + tc.name }} {% set _args = tc.arguments %} {% for k, v in _args.items() %} {{ k }} {{ v | tojson(ensure_ascii=False) if v is not string else v }} {% endfor %} {% endfor %} {% endif %} {%- elif m.role == 'tool' -%} {%- if m.content is string -%} {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %} {{- '<|observation|>' }} {%- endif %} {{- '\n\n' }} {{- m.content }} {{- '\n' }} {%- else -%} <|observation|>{% for tr in m.content %} {{ tr.output if tr.output is defined else tr }} {% endfor -%} {% endif -%} {%- elif m.role == 'system' -%} <|system|> {{ visible_text(m.content) }} {%- endif -%} {%- endfor -%} {%- if add_generation_prompt -%} <|assistant|>{{- '\n' if (enable_thinking is defined and not enable_thinking) else '' -}} {%- endif -%} INFO:gguf.gguf_writer:Writing the following files: INFO:gguf.gguf_writer:upload-GLM_4.5_Air/mtp-GLM-4.5-Air-BF16.gguf: n_tensors = 26, total_size = 9.7G Writing: 0%| | 0.00/9.72G [00:00\n{%- if tools -%}\n<|syste... llama_model_loader: - kv 42: split.no u16 = 0 llama_model_loader: - kv 43: split.count u16 = 2 llama_model_loader: - kv 44: split.tensors.count i32 = 780 llama_model_loader: - type f32: 321 tensors llama_model_loader: - type bf16: 459 tensors [ 1/ 780] token_embd.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q8_0 .. size = 1184.00 MiB -> 629.00 MiB [ 2/ 780] blk.0.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 3/ 780] blk.0.ffn_down.weight - [ 10944, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 85.50 MiB -> 45.42 MiB [ 4/ 780] blk.0.ffn_gate.weight - [ 4096, 10944, 1, 1], type = bf16, converting to q8_0 .. size = 85.50 MiB -> 45.42 MiB [ 5/ 780] blk.0.ffn_up.weight - [ 4096, 10944, 1, 1], type = bf16, converting to q8_0 .. size = 85.50 MiB -> 45.42 MiB [ 6/ 780] blk.0.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 7/ 780] blk.0.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 8/ 780] blk.0.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 9/ 780] blk.0.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 10/ 780] blk.0.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 11/ 780] blk.0.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 12/ 780] blk.0.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 13/ 780] blk.0.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 14/ 780] blk.1.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 15/ 780] blk.1.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 16/ 780] blk.1.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 17/ 780] blk.1.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 18/ 780] blk.1.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 19/ 780] blk.1.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 20/ 780] blk.1.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 21/ 780] blk.1.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 22/ 780] blk.1.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 23/ 780] blk.1.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 24/ 780] blk.1.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 25/ 780] blk.1.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 26/ 780] blk.1.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 27/ 780] blk.1.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 28/ 780] blk.1.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 29/ 780] blk.1.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 30/ 780] blk.1.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 31/ 780] blk.2.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 32/ 780] blk.2.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 33/ 780] blk.2.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 34/ 780] blk.2.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 35/ 780] blk.2.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 36/ 780] blk.2.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 37/ 780] blk.2.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 38/ 780] blk.2.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 39/ 780] blk.2.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 40/ 780] blk.2.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 41/ 780] blk.2.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 42/ 780] blk.2.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 43/ 780] blk.2.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 44/ 780] blk.2.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 45/ 780] blk.2.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 46/ 780] blk.2.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 47/ 780] blk.2.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 48/ 780] blk.3.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 49/ 780] blk.3.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 50/ 780] blk.3.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 51/ 780] blk.3.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 52/ 780] blk.3.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 53/ 780] blk.3.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 54/ 780] blk.3.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 55/ 780] blk.3.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 56/ 780] blk.3.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 57/ 780] blk.3.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 58/ 780] blk.3.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 59/ 780] blk.3.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 60/ 780] blk.3.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 61/ 780] blk.3.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 62/ 780] blk.3.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 63/ 780] blk.3.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 64/ 780] blk.3.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 65/ 780] blk.4.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 66/ 780] blk.4.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 67/ 780] blk.4.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 68/ 780] blk.4.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 69/ 780] blk.4.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 70/ 780] blk.4.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 71/ 780] blk.4.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 72/ 780] blk.4.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 73/ 780] blk.4.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 74/ 780] blk.4.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 75/ 780] blk.4.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 76/ 780] blk.4.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 77/ 780] blk.4.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 78/ 780] blk.4.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 79/ 780] blk.4.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 80/ 780] blk.4.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 81/ 780] blk.4.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 82/ 780] blk.5.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 83/ 780] blk.5.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 84/ 780] blk.5.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 85/ 780] blk.5.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 86/ 780] blk.5.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 87/ 780] blk.5.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 88/ 780] blk.5.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 89/ 780] blk.5.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 90/ 780] blk.5.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 91/ 780] blk.5.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 92/ 780] blk.5.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 93/ 780] blk.5.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 94/ 780] blk.5.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 95/ 780] blk.5.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 96/ 780] blk.5.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 97/ 780] blk.5.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 98/ 780] blk.5.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 99/ 780] blk.6.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 100/ 780] blk.6.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 101/ 780] blk.6.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 102/ 780] blk.6.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 103/ 780] blk.6.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 104/ 780] blk.6.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 105/ 780] blk.6.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 106/ 780] blk.6.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 107/ 780] blk.6.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 108/ 780] blk.6.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 109/ 780] blk.6.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 110/ 780] blk.6.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 111/ 780] blk.6.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 112/ 780] blk.6.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 113/ 780] blk.6.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 114/ 780] blk.6.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 115/ 780] blk.6.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 116/ 780] blk.7.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 117/ 780] blk.7.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 118/ 780] blk.7.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 119/ 780] blk.7.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 120/ 780] blk.7.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 121/ 780] blk.7.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 122/ 780] blk.7.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 123/ 780] blk.7.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 124/ 780] blk.7.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 125/ 780] blk.7.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 126/ 780] blk.7.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 127/ 780] blk.7.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 128/ 780] blk.7.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 129/ 780] blk.7.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 130/ 780] blk.7.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 131/ 780] blk.7.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 132/ 780] blk.7.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 133/ 780] blk.8.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 134/ 780] blk.8.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 135/ 780] blk.8.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 136/ 780] blk.8.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 137/ 780] blk.8.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 138/ 780] blk.8.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 139/ 780] blk.8.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 140/ 780] blk.8.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 141/ 780] blk.8.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 142/ 780] blk.8.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 143/ 780] blk.8.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 144/ 780] blk.8.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 145/ 780] blk.8.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 146/ 780] blk.8.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 147/ 780] blk.8.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 148/ 780] blk.8.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 149/ 780] blk.8.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 150/ 780] blk.9.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 151/ 780] blk.9.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 152/ 780] blk.9.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 153/ 780] blk.9.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 154/ 780] blk.9.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 155/ 780] blk.9.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 156/ 780] blk.9.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 157/ 780] blk.9.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 158/ 780] blk.9.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 159/ 780] blk.9.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 160/ 780] blk.9.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 161/ 780] blk.9.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 162/ 780] blk.9.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 163/ 780] blk.9.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 164/ 780] blk.9.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 165/ 780] blk.9.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 166/ 780] blk.9.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 167/ 780] blk.10.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 168/ 780] blk.10.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 169/ 780] blk.10.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 170/ 780] blk.10.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 171/ 780] blk.10.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 172/ 780] blk.10.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 173/ 780] blk.10.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 174/ 780] blk.10.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 175/ 780] blk.10.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 176/ 780] blk.10.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 177/ 780] blk.10.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 178/ 780] blk.10.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 179/ 780] blk.10.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 180/ 780] blk.10.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 181/ 780] blk.10.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 182/ 780] blk.10.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 183/ 780] blk.10.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 184/ 780] blk.11.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 185/ 780] blk.11.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 186/ 780] blk.11.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 187/ 780] blk.11.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 188/ 780] blk.11.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 189/ 780] blk.11.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 190/ 780] blk.11.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 191/ 780] blk.11.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 192/ 780] blk.11.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 193/ 780] blk.11.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 194/ 780] blk.11.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 195/ 780] blk.11.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 196/ 780] blk.11.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 197/ 780] blk.11.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 198/ 780] blk.11.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 199/ 780] blk.11.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 200/ 780] blk.11.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 201/ 780] blk.12.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 202/ 780] blk.12.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 203/ 780] blk.12.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 204/ 780] blk.12.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 205/ 780] blk.12.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 206/ 780] blk.12.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 207/ 780] blk.12.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 208/ 780] blk.12.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 209/ 780] blk.12.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 210/ 780] blk.12.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 211/ 780] blk.12.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 212/ 780] blk.12.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 213/ 780] blk.12.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 214/ 780] blk.12.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 215/ 780] blk.12.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 216/ 780] blk.12.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 217/ 780] blk.12.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 218/ 780] blk.13.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 219/ 780] blk.13.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 220/ 780] blk.13.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 221/ 780] blk.13.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 222/ 780] blk.13.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 223/ 780] blk.13.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 224/ 780] blk.13.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 225/ 780] blk.13.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 226/ 780] blk.13.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 227/ 780] blk.13.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 228/ 780] blk.13.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 229/ 780] blk.13.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 230/ 780] blk.13.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 231/ 780] blk.13.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 232/ 780] blk.13.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 233/ 780] blk.13.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 234/ 780] blk.13.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 235/ 780] blk.14.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 236/ 780] blk.14.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 237/ 780] blk.14.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 238/ 780] blk.14.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 239/ 780] blk.14.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 240/ 780] blk.14.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 241/ 780] blk.14.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 242/ 780] blk.14.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 243/ 780] blk.14.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 244/ 780] blk.14.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 245/ 780] blk.14.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 246/ 780] blk.14.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 247/ 780] blk.14.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 248/ 780] blk.14.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 249/ 780] blk.14.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 250/ 780] blk.14.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 251/ 780] blk.14.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 252/ 780] blk.15.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 253/ 780] blk.15.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 254/ 780] blk.15.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 255/ 780] blk.15.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 256/ 780] blk.15.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 257/ 780] blk.15.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 258/ 780] blk.15.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 259/ 780] blk.15.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 260/ 780] blk.15.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 261/ 780] blk.15.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 262/ 780] blk.15.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 263/ 780] blk.15.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 264/ 780] blk.15.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 265/ 780] blk.15.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 266/ 780] blk.15.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 267/ 780] blk.15.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 268/ 780] blk.15.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 269/ 780] blk.16.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 270/ 780] blk.16.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 271/ 780] blk.16.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 272/ 780] blk.16.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 273/ 780] blk.16.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 274/ 780] blk.16.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 275/ 780] blk.16.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 276/ 780] blk.16.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 277/ 780] blk.16.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 278/ 780] blk.16.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 279/ 780] blk.16.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 280/ 780] blk.16.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 281/ 780] blk.16.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 282/ 780] blk.16.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 283/ 780] blk.16.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 284/ 780] blk.16.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 285/ 780] blk.16.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 286/ 780] blk.17.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 287/ 780] blk.17.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 288/ 780] blk.17.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 289/ 780] blk.17.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 290/ 780] blk.17.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 291/ 780] blk.17.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 292/ 780] blk.17.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 293/ 780] blk.17.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 294/ 780] blk.17.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 295/ 780] blk.17.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 296/ 780] blk.17.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 297/ 780] blk.17.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 298/ 780] blk.17.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 299/ 780] blk.17.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 300/ 780] blk.17.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 301/ 780] blk.17.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 302/ 780] blk.17.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 303/ 780] blk.18.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 304/ 780] blk.18.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 305/ 780] blk.18.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 306/ 780] blk.18.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 307/ 780] blk.18.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 308/ 780] blk.18.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 309/ 780] blk.18.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 310/ 780] blk.18.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 311/ 780] blk.18.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 312/ 780] blk.18.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 313/ 780] blk.18.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 314/ 780] blk.18.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 315/ 780] blk.18.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 316/ 780] blk.18.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 317/ 780] blk.18.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 318/ 780] blk.18.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 319/ 780] blk.18.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 320/ 780] blk.19.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 321/ 780] blk.19.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 322/ 780] blk.19.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 323/ 780] blk.19.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 324/ 780] blk.19.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 325/ 780] blk.19.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 326/ 780] blk.19.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 327/ 780] blk.19.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 328/ 780] blk.19.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 329/ 780] blk.19.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 330/ 780] blk.19.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 331/ 780] blk.19.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 332/ 780] blk.19.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 333/ 780] blk.19.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 334/ 780] blk.19.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 335/ 780] blk.19.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 336/ 780] blk.19.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 337/ 780] blk.20.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 338/ 780] blk.20.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 339/ 780] blk.20.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 340/ 780] blk.20.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 341/ 780] blk.20.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 342/ 780] blk.20.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 343/ 780] blk.20.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 344/ 780] blk.20.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 345/ 780] blk.20.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 346/ 780] blk.20.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 347/ 780] blk.20.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 348/ 780] blk.20.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 349/ 780] blk.20.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 350/ 780] blk.20.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 351/ 780] blk.20.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 352/ 780] blk.20.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 353/ 780] blk.20.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 354/ 780] blk.21.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 355/ 780] blk.21.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 356/ 780] blk.21.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 357/ 780] blk.21.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 358/ 780] blk.21.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 359/ 780] blk.21.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 360/ 780] blk.21.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 361/ 780] blk.21.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 362/ 780] blk.21.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 363/ 780] blk.21.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 364/ 780] blk.21.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 365/ 780] blk.21.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 366/ 780] blk.21.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 367/ 780] blk.21.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 368/ 780] blk.21.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 369/ 780] blk.21.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 370/ 780] blk.21.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 371/ 780] blk.22.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 372/ 780] blk.22.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 373/ 780] blk.22.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 374/ 780] blk.22.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 375/ 780] blk.22.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 376/ 780] blk.22.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 377/ 780] blk.22.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 378/ 780] blk.22.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 379/ 780] blk.22.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 380/ 780] blk.22.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 381/ 780] blk.22.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 382/ 780] blk.22.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 383/ 780] blk.22.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 384/ 780] blk.22.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 385/ 780] blk.22.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 386/ 780] blk.22.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 387/ 780] blk.22.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 388/ 780] blk.23.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 389/ 780] blk.23.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 390/ 780] blk.23.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 391/ 780] blk.23.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 392/ 780] blk.23.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 393/ 780] blk.23.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 394/ 780] blk.23.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 395/ 780] blk.23.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 396/ 780] blk.23.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 397/ 780] blk.23.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 398/ 780] blk.23.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 399/ 780] blk.23.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 400/ 780] blk.23.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 401/ 780] blk.23.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 402/ 780] blk.23.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 403/ 780] blk.23.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 404/ 780] blk.23.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 405/ 780] blk.24.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 406/ 780] blk.24.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 407/ 780] blk.24.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 408/ 780] blk.24.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 409/ 780] blk.24.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 410/ 780] blk.24.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 411/ 780] blk.24.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 412/ 780] blk.24.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 413/ 780] blk.24.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 414/ 780] blk.24.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 415/ 780] blk.24.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 416/ 780] blk.24.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 417/ 780] blk.24.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 418/ 780] blk.24.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 419/ 780] blk.24.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 420/ 780] blk.24.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 421/ 780] blk.24.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 422/ 780] blk.25.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 423/ 780] blk.25.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 424/ 780] blk.25.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 425/ 780] blk.25.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 426/ 780] blk.25.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 427/ 780] blk.25.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 428/ 780] blk.25.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 429/ 780] blk.25.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 430/ 780] blk.25.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 431/ 780] blk.25.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 432/ 780] blk.25.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 433/ 780] blk.25.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 434/ 780] blk.25.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 435/ 780] blk.25.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 436/ 780] blk.25.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 437/ 780] blk.25.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 438/ 780] blk.25.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 439/ 780] blk.26.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 440/ 780] blk.26.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 441/ 780] blk.26.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 442/ 780] blk.26.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 443/ 780] blk.26.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 444/ 780] blk.26.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 445/ 780] blk.26.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 446/ 780] blk.26.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 447/ 780] blk.26.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 448/ 780] blk.26.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 449/ 780] blk.26.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 450/ 780] blk.26.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 451/ 780] blk.26.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 452/ 780] blk.26.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 453/ 780] blk.26.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 454/ 780] blk.26.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 455/ 780] blk.26.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 456/ 780] blk.27.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 457/ 780] blk.27.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 458/ 780] blk.27.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 459/ 780] blk.27.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 460/ 780] blk.27.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 461/ 780] blk.27.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 462/ 780] blk.27.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 463/ 780] blk.27.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 464/ 780] blk.27.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 465/ 780] blk.27.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 466/ 780] blk.27.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 467/ 780] blk.27.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 468/ 780] blk.27.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 469/ 780] blk.27.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 470/ 780] blk.27.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 471/ 780] blk.27.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 472/ 780] blk.27.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 473/ 780] blk.28.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 474/ 780] blk.28.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 475/ 780] blk.28.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 476/ 780] blk.28.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 477/ 780] blk.28.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 478/ 780] blk.28.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 479/ 780] blk.28.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 480/ 780] blk.28.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 481/ 780] blk.28.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 482/ 780] blk.28.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 483/ 780] blk.28.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 484/ 780] blk.28.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 485/ 780] blk.28.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 486/ 780] blk.28.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 487/ 780] blk.28.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 488/ 780] blk.28.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 489/ 780] blk.28.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 490/ 780] blk.29.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 491/ 780] blk.29.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 492/ 780] blk.29.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 493/ 780] blk.29.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 494/ 780] blk.29.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 495/ 780] blk.29.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 496/ 780] blk.29.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 497/ 780] blk.29.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 498/ 780] blk.29.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 499/ 780] blk.29.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 500/ 780] blk.29.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 501/ 780] blk.29.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 502/ 780] blk.29.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 503/ 780] blk.29.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 504/ 780] blk.29.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 505/ 780] blk.29.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 506/ 780] blk.29.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 507/ 780] blk.30.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 508/ 780] blk.30.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 509/ 780] blk.30.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 510/ 780] blk.30.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 511/ 780] blk.30.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 512/ 780] blk.30.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 513/ 780] blk.30.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 514/ 780] blk.30.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 515/ 780] blk.30.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 516/ 780] blk.30.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 517/ 780] blk.30.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 518/ 780] blk.30.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 519/ 780] blk.30.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 520/ 780] blk.30.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 521/ 780] blk.30.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 522/ 780] blk.30.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 523/ 780] blk.30.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 524/ 780] blk.31.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 525/ 780] blk.31.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 526/ 780] blk.31.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 527/ 780] blk.31.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 528/ 780] blk.31.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 529/ 780] blk.31.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 530/ 780] blk.31.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 531/ 780] blk.31.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 532/ 780] blk.31.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 533/ 780] blk.31.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 534/ 780] blk.31.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 535/ 780] blk.31.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 536/ 780] blk.31.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 537/ 780] blk.31.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 538/ 780] blk.31.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 539/ 780] blk.31.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 540/ 780] blk.31.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 541/ 780] blk.32.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 542/ 780] blk.32.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 543/ 780] blk.32.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 544/ 780] blk.32.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 545/ 780] blk.32.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 546/ 780] blk.32.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 547/ 780] blk.32.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 548/ 780] blk.32.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 549/ 780] blk.32.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 550/ 780] blk.32.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 551/ 780] blk.32.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 552/ 780] blk.32.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 553/ 780] blk.32.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 554/ 780] blk.32.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 555/ 780] blk.32.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 556/ 780] blk.32.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 557/ 780] blk.32.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 558/ 780] blk.33.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 559/ 780] blk.33.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 560/ 780] blk.33.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 561/ 780] blk.33.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 562/ 780] blk.33.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 563/ 780] blk.33.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 564/ 780] blk.33.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 565/ 780] blk.33.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 566/ 780] blk.33.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 567/ 780] blk.33.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 568/ 780] blk.33.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 569/ 780] blk.33.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 570/ 780] blk.33.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 571/ 780] blk.33.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 572/ 780] blk.33.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 573/ 780] blk.33.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 574/ 780] blk.33.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 575/ 780] blk.34.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 576/ 780] blk.34.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 577/ 780] blk.34.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 578/ 780] blk.34.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 579/ 780] blk.34.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 580/ 780] blk.34.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 581/ 780] blk.34.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 582/ 780] blk.34.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 583/ 780] blk.34.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 584/ 780] blk.34.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 585/ 780] blk.34.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 586/ 780] blk.34.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 587/ 780] blk.34.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 588/ 780] blk.34.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 589/ 780] blk.34.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 590/ 780] blk.34.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 591/ 780] blk.34.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 592/ 780] blk.35.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 593/ 780] blk.35.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 594/ 780] blk.35.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 595/ 780] blk.35.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 596/ 780] blk.35.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 597/ 780] blk.35.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 598/ 780] blk.35.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 599/ 780] blk.35.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 600/ 780] blk.35.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 601/ 780] blk.35.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 602/ 780] blk.35.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 603/ 780] blk.35.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 604/ 780] blk.35.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 605/ 780] blk.35.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 606/ 780] blk.35.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 607/ 780] blk.35.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 608/ 780] blk.35.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 609/ 780] blk.36.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 610/ 780] blk.36.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 611/ 780] blk.36.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 612/ 780] blk.36.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 613/ 780] blk.36.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 614/ 780] blk.36.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 615/ 780] blk.36.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 616/ 780] blk.36.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 617/ 780] blk.36.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 618/ 780] blk.36.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 619/ 780] blk.36.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 620/ 780] blk.36.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 621/ 780] blk.36.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 622/ 780] blk.36.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 623/ 780] blk.36.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 624/ 780] blk.36.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 625/ 780] blk.36.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 626/ 780] blk.37.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 627/ 780] blk.37.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 628/ 780] blk.37.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 629/ 780] blk.37.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 630/ 780] blk.37.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 631/ 780] blk.37.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 632/ 780] blk.37.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 633/ 780] blk.37.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 634/ 780] blk.37.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 635/ 780] blk.37.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 636/ 780] blk.37.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 637/ 780] blk.37.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 638/ 780] blk.37.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 639/ 780] blk.37.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 640/ 780] blk.37.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 641/ 780] blk.37.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 642/ 780] blk.37.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 643/ 780] blk.38.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 644/ 780] blk.38.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 645/ 780] blk.38.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 646/ 780] blk.38.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 647/ 780] blk.38.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 648/ 780] blk.38.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 649/ 780] blk.38.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 650/ 780] blk.38.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 651/ 780] blk.38.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 652/ 780] blk.38.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 653/ 780] blk.38.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 654/ 780] blk.38.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 655/ 780] blk.38.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 656/ 780] blk.38.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 657/ 780] blk.38.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 658/ 780] blk.38.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 659/ 780] blk.38.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 660/ 780] blk.39.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 661/ 780] blk.39.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 662/ 780] blk.39.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 663/ 780] blk.39.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 664/ 780] blk.39.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 665/ 780] blk.39.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 666/ 780] blk.39.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 667/ 780] blk.39.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 668/ 780] blk.39.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 669/ 780] blk.39.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 670/ 780] blk.39.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 671/ 780] blk.39.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 672/ 780] blk.39.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 673/ 780] blk.39.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 674/ 780] blk.39.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 675/ 780] blk.39.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 676/ 780] blk.39.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 677/ 780] blk.40.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 678/ 780] blk.40.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 679/ 780] blk.40.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 680/ 780] blk.40.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 681/ 780] blk.40.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 682/ 780] blk.40.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 683/ 780] blk.40.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 684/ 780] blk.40.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 685/ 780] blk.40.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 686/ 780] blk.40.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 687/ 780] blk.40.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 688/ 780] blk.40.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 689/ 780] blk.40.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 690/ 780] blk.40.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 691/ 780] blk.40.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 692/ 780] blk.40.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 693/ 780] blk.40.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 694/ 780] blk.41.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 695/ 780] blk.41.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 696/ 780] blk.41.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 697/ 780] blk.41.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 698/ 780] blk.41.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 699/ 780] blk.41.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 700/ 780] blk.41.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 701/ 780] blk.41.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 702/ 780] blk.41.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 703/ 780] blk.41.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 704/ 780] blk.41.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 705/ 780] blk.41.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 706/ 780] blk.41.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 707/ 780] blk.41.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 708/ 780] blk.41.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 709/ 780] blk.41.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 710/ 780] blk.41.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 711/ 780] blk.42.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 712/ 780] blk.42.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 713/ 780] blk.42.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 714/ 780] blk.42.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 715/ 780] blk.42.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 716/ 780] blk.42.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 717/ 780] blk.42.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 718/ 780] blk.42.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 719/ 780] blk.42.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 720/ 780] blk.42.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 721/ 780] blk.42.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 722/ 780] blk.42.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 723/ 780] blk.42.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 724/ 780] blk.42.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 725/ 780] blk.42.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 726/ 780] blk.42.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 727/ 780] blk.42.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 728/ 780] blk.43.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 729/ 780] blk.43.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 730/ 780] blk.43.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 731/ 780] blk.43.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 732/ 780] blk.43.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 733/ 780] blk.43.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 734/ 780] blk.43.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 735/ 780] blk.43.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 736/ 780] blk.43.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 737/ 780] blk.43.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 738/ 780] blk.43.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 739/ 780] blk.43.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 740/ 780] blk.43.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 741/ 780] blk.43.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 742/ 780] blk.43.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 743/ 780] blk.43.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 744/ 780] blk.43.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 745/ 780] blk.44.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 746/ 780] blk.44.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 747/ 780] blk.44.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 748/ 780] blk.44.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 749/ 780] blk.44.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 750/ 780] blk.44.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 751/ 780] blk.44.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 752/ 780] blk.44.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 753/ 780] blk.44.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 754/ 780] blk.44.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 755/ 780] blk.44.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 756/ 780] blk.44.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 757/ 780] blk.44.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 758/ 780] blk.44.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 759/ 780] blk.44.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 760/ 780] blk.44.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 761/ 780] blk.44.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 762/ 780] output.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q8_0 .. size = 1184.00 MiB -> 629.00 MiB [ 763/ 780] blk.45.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 764/ 780] blk.45.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 765/ 780] blk.45.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 766/ 780] blk.45.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 767/ 780] blk.45.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 768/ 780] blk.45.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 769/ 780] blk.45.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 770/ 780] blk.45.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 771/ 780] blk.45.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 772/ 780] blk.45.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 773/ 780] blk.45.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 774/ 780] blk.45.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 775/ 780] blk.45.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 776/ 780] blk.45.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 777/ 780] blk.45.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 778/ 780] blk.45.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 779/ 780] blk.45.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 780/ 780] output_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB llama_model_quantize_impl: model size = 203851.49 MiB (16.00 BPW) llama_model_quantize_impl: quant size = 108340.16 MiB (8.51 BPW) llama_quantize: quantize time = 759559.41 ms llama_quantize: total time = 759559.41 ms + ./llama.cpp/build/bin/llama-quantize --keep-split --pure --tensor-type token_embd.weight=q8_0 --tensor-type '^output.weight=q6_k' --tensor-type ffn_down_exps.weight=q4_0 --tensor-type ffn_down.weight=q8_0 --tensor-type ffn_gate.weight=q8_0 --tensor-type ffn_up.weight=q8_0 --tensor-type shexp=q8_0 --tensor-type attn_=q8_0 ./upload-GLM_4.5_Air/GLM-4.5-Air-BF16-00001-of-00002.gguf ./upload-GLM_4.5_Air/GLM-4.5-Air-Q4_K_M.gguf Q4_K_M version: 0.3.0-dev (build 1, commit c1d0e7a) built with GNU 14.2.0 for Linux x86_64 llama_quantize: quantizing './upload-GLM_4.5_Air/GLM-4.5-Air-BF16-00001-of-00002.gguf' to './upload-GLM_4.5_Air/GLM-4.5-Air-Q4_K_M' as Q4_K_M llama_model_loader: additional 1 GGUFs metadata loaded. llama_model_loader: loaded meta data with 45 key-value pairs and 780 tensors from ./upload-GLM_4.5_Air/GLM-4.5-Air-BF16-00001-of-00002.gguf (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = glm4moe llama_model_loader: - kv 1: general.type str = model llama_model_loader: - kv 2: general.name str = GLM-4.5-Air llama_model_loader: - kv 3: general.size_label str = 128x8.0B llama_model_loader: - kv 4: general.license str = mit llama_model_loader: - kv 5: general.tags arr[str,1] = ["text-generation"] llama_model_loader: - kv 6: general.languages arr[str,2] = ["en", "zh"] llama_model_loader: - kv 7: glm4moe.block_count u32 = 46 llama_model_loader: - kv 8: glm4moe.context_length u32 = 131072 llama_model_loader: - kv 9: glm4moe.embedding_length u32 = 4096 llama_model_loader: - kv 10: glm4moe.feed_forward_length u32 = 10944 llama_model_loader: - kv 11: glm4moe.attention.head_count u32 = 96 llama_model_loader: - kv 12: glm4moe.attention.head_count_kv u32 = 8 llama_model_loader: - kv 13: glm4moe.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 14: glm4moe.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 15: glm4moe.expert_count u32 = 128 llama_model_loader: - kv 16: glm4moe.expert_used_count u32 = 8 llama_model_loader: - kv 17: glm4moe.expert_group_count u32 = 1 llama_model_loader: - kv 18: glm4moe.expert_group_used_count u32 = 1 llama_model_loader: - kv 19: glm4moe.attention.key_length u32 = 128 llama_model_loader: - kv 20: glm4moe.attention.value_length u32 = 128 llama_model_loader: - kv 21: general.file_type u32 = 32 llama_model_loader: - kv 22: glm4moe.rope.dimension_count u32 = 64 llama_model_loader: - kv 23: glm4moe.expert_feed_forward_length u32 = 1408 llama_model_loader: - kv 24: glm4moe.expert_shared_count u32 = 1 llama_model_loader: - kv 25: glm4moe.leading_dense_block_count u32 = 1 llama_model_loader: - kv 26: glm4moe.expert_gating_func u32 = 2 llama_model_loader: - kv 27: glm4moe.expert_weights_scale f32 = 1.000000 llama_model_loader: - kv 28: glm4moe.expert_weights_norm bool = true llama_model_loader: - kv 29: general.quantization_version u32 = 2 llama_model_loader: - kv 30: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 31: tokenizer.ggml.pre str = glm4 llama_model_loader: - kv 32: tokenizer.ggml.tokens arr[str,151552] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 33: tokenizer.ggml.token_type arr[i32,151552] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 34: tokenizer.ggml.merges arr[str,318088] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "... llama_model_loader: - kv 35: tokenizer.ggml.eos_token_id u32 = 151329 llama_model_loader: - kv 36: tokenizer.ggml.padding_token_id u32 = 151329 llama_model_loader: - kv 37: tokenizer.ggml.bos_token_id u32 = 151331 llama_model_loader: - kv 38: tokenizer.ggml.eot_token_id u32 = 151336 llama_model_loader: - kv 39: tokenizer.ggml.unknown_token_id u32 = 151329 llama_model_loader: - kv 40: tokenizer.ggml.eom_token_id u32 = 151338 llama_model_loader: - kv 41: tokenizer.chat_template str = [gMASK]\n{%- if tools -%}\n<|syste... llama_model_loader: - kv 42: split.no u16 = 0 llama_model_loader: - kv 43: split.count u16 = 2 llama_model_loader: - kv 44: split.tensors.count i32 = 780 llama_model_loader: - type f32: 321 tensors llama_model_loader: - type bf16: 459 tensors llama_tensor_get_type: token_embd.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.0.ffn_down.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.0.ffn_gate.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.0.ffn_up.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.0.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.0.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.0.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.0.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.1.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.1.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.1.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.1.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.1.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.1.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.1.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.1.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.2.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.2.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.2.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.2.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.2.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.2.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.2.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.2.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.3.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.3.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.3.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.3.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.3.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.3.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.3.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.3.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.4.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.4.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.4.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.4.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.4.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.4.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.4.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.4.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.5.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.5.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.5.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.5.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.5.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.5.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.5.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.5.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.6.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.6.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.6.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.6.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.6.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.6.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.6.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.6.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.7.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.7.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.7.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.7.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.7.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.7.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.7.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.7.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.8.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.8.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.8.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.8.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.8.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.8.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.8.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.8.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.9.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.9.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.9.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.9.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.9.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.9.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.9.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.9.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.10.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.10.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.10.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.10.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.10.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.10.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.10.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.10.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.11.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.11.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.11.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.11.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.11.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.11.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.11.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.11.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.12.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.12.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.12.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.12.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.12.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.12.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.12.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.12.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.13.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.13.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.13.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.13.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.13.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.13.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.13.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.13.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.14.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.14.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.14.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.14.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.14.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.14.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.14.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.14.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.15.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.15.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.15.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.15.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.15.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.15.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.15.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.15.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.16.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.16.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.16.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.16.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.16.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.16.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.16.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.16.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.17.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.17.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.17.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.17.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.17.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.17.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.17.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.17.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.18.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.18.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.18.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.18.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.18.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.18.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.18.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.18.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.19.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.19.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.19.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.19.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.19.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.19.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.19.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.19.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.20.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.20.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.20.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.20.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.20.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.20.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.20.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.20.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.21.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.21.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.21.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.21.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.21.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.21.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.21.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.21.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.22.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.22.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.22.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.22.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.22.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.22.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.22.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.22.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.23.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.23.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.23.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.23.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.23.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.23.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.23.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.23.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.24.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.24.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.24.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.24.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.24.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.24.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.24.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.24.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.25.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.25.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.25.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.25.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.25.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.25.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.25.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.25.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.26.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.26.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.26.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.26.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.26.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.26.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.26.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.26.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.27.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.27.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.27.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.27.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.27.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.27.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.27.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.27.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.28.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.28.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.28.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.28.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.28.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.28.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.28.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.28.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.29.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.29.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.29.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.29.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.29.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.29.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.29.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.29.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.30.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.30.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.30.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.30.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.30.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.30.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.30.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.30.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.31.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.31.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.31.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.31.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.31.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.31.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.31.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.31.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.32.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.32.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.32.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.32.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.32.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.32.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.32.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.32.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.33.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.33.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.33.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.33.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.33.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.33.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.33.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.33.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.34.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.34.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.34.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.34.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.34.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.34.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.34.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.34.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.35.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.35.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.35.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.35.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.35.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.35.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.35.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.35.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.36.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.36.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.36.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.36.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.36.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.36.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.36.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.36.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.37.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.37.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.37.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.37.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.37.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.37.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.37.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.37.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.38.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.38.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.38.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.38.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.38.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.38.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.38.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.38.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.39.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.39.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.39.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.39.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.39.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.39.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.39.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.39.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.40.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.40.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.40.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.40.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.40.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.40.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.40.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.40.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.41.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.41.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.41.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.41.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.41.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.41.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.41.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.41.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.42.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.42.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.42.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.42.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.42.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.42.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.42.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.42.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.43.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.43.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.43.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.43.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.43.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.43.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.43.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.43.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.44.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.44.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.44.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.44.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.44.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.44.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.44.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.44.attn_v.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: output.weight - applying manual override: q4_K -> q6_K llama_tensor_get_type: blk.45.ffn_down_exps.weight - applying manual override: q4_K -> q4_0 llama_tensor_get_type: blk.45.ffn_down_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.45.ffn_gate_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.45.ffn_up_shexp.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.45.attn_k.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.45.attn_output.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.45.attn_q.weight - applying manual override: q4_K -> q8_0 llama_tensor_get_type: blk.45.attn_v.weight - applying manual override: q4_K -> q8_0 [ 1/ 780] token_embd.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q8_0 .. size = 1184.00 MiB -> 629.00 MiB [ 2/ 780] blk.0.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 3/ 780] blk.0.ffn_down.weight - [ 10944, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 85.50 MiB -> 45.42 MiB [ 4/ 780] blk.0.ffn_gate.weight - [ 4096, 10944, 1, 1], type = bf16, converting to q8_0 .. size = 85.50 MiB -> 45.42 MiB [ 5/ 780] blk.0.ffn_up.weight - [ 4096, 10944, 1, 1], type = bf16, converting to q8_0 .. size = 85.50 MiB -> 45.42 MiB [ 6/ 780] blk.0.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 7/ 780] blk.0.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 8/ 780] blk.0.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 9/ 780] blk.0.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 10/ 780] blk.0.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 11/ 780] blk.0.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 12/ 780] blk.0.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 13/ 780] blk.0.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 14/ 780] blk.1.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 15/ 780] blk.1.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 16/ 780] blk.1.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 17/ 780] blk.1.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 18/ 780] blk.1.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 19/ 780] blk.1.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 20/ 780] blk.1.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 21/ 780] blk.1.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 22/ 780] blk.1.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 23/ 780] blk.1.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 24/ 780] blk.1.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 25/ 780] blk.1.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 26/ 780] blk.1.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 27/ 780] blk.1.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 28/ 780] blk.1.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 29/ 780] blk.1.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 30/ 780] blk.1.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 31/ 780] blk.2.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 32/ 780] blk.2.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 33/ 780] blk.2.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 34/ 780] blk.2.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 35/ 780] blk.2.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 36/ 780] blk.2.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 37/ 780] blk.2.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 38/ 780] blk.2.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 39/ 780] blk.2.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 40/ 780] blk.2.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 41/ 780] blk.2.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 42/ 780] blk.2.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 43/ 780] blk.2.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 44/ 780] blk.2.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 45/ 780] blk.2.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 46/ 780] blk.2.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 47/ 780] blk.2.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 48/ 780] blk.3.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 49/ 780] blk.3.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 50/ 780] blk.3.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 51/ 780] blk.3.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 52/ 780] blk.3.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 53/ 780] blk.3.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 54/ 780] blk.3.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 55/ 780] blk.3.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 56/ 780] blk.3.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 57/ 780] blk.3.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 58/ 780] blk.3.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 59/ 780] blk.3.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 60/ 780] blk.3.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 61/ 780] blk.3.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 62/ 780] blk.3.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 63/ 780] blk.3.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 64/ 780] blk.3.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 65/ 780] blk.4.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 66/ 780] blk.4.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 67/ 780] blk.4.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 68/ 780] blk.4.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 69/ 780] blk.4.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 70/ 780] blk.4.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 71/ 780] blk.4.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 72/ 780] blk.4.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 73/ 780] blk.4.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 74/ 780] blk.4.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 75/ 780] blk.4.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 76/ 780] blk.4.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 77/ 780] blk.4.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 78/ 780] blk.4.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 79/ 780] blk.4.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 80/ 780] blk.4.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 81/ 780] blk.4.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 82/ 780] blk.5.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 83/ 780] blk.5.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 84/ 780] blk.5.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 85/ 780] blk.5.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 86/ 780] blk.5.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 87/ 780] blk.5.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 88/ 780] blk.5.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 89/ 780] blk.5.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 90/ 780] blk.5.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 91/ 780] blk.5.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 92/ 780] blk.5.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 93/ 780] blk.5.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 94/ 780] blk.5.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 95/ 780] blk.5.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 96/ 780] blk.5.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 97/ 780] blk.5.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 98/ 780] blk.5.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 99/ 780] blk.6.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 100/ 780] blk.6.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 101/ 780] blk.6.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 102/ 780] blk.6.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 103/ 780] blk.6.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 104/ 780] blk.6.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 105/ 780] blk.6.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 106/ 780] blk.6.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 107/ 780] blk.6.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 108/ 780] blk.6.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 109/ 780] blk.6.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 110/ 780] blk.6.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 111/ 780] blk.6.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 112/ 780] blk.6.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 113/ 780] blk.6.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 114/ 780] blk.6.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 115/ 780] blk.6.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 116/ 780] blk.7.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 117/ 780] blk.7.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 118/ 780] blk.7.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 119/ 780] blk.7.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 120/ 780] blk.7.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 121/ 780] blk.7.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 122/ 780] blk.7.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 123/ 780] blk.7.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 124/ 780] blk.7.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 125/ 780] blk.7.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 126/ 780] blk.7.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 127/ 780] blk.7.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 128/ 780] blk.7.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 129/ 780] blk.7.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 130/ 780] blk.7.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 131/ 780] blk.7.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 132/ 780] blk.7.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 133/ 780] blk.8.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 134/ 780] blk.8.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 135/ 780] blk.8.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 136/ 780] blk.8.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 137/ 780] blk.8.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 138/ 780] blk.8.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 139/ 780] blk.8.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 140/ 780] blk.8.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 141/ 780] blk.8.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 142/ 780] blk.8.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 143/ 780] blk.8.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 144/ 780] blk.8.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 145/ 780] blk.8.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 146/ 780] blk.8.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 147/ 780] blk.8.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 148/ 780] blk.8.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 149/ 780] blk.8.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 150/ 780] blk.9.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 151/ 780] blk.9.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 152/ 780] blk.9.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 153/ 780] blk.9.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 154/ 780] blk.9.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 155/ 780] blk.9.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 156/ 780] blk.9.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 157/ 780] blk.9.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 158/ 780] blk.9.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 159/ 780] blk.9.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 160/ 780] blk.9.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 161/ 780] blk.9.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 162/ 780] blk.9.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 163/ 780] blk.9.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 164/ 780] blk.9.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 165/ 780] blk.9.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 166/ 780] blk.9.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 167/ 780] blk.10.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 168/ 780] blk.10.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 169/ 780] blk.10.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 170/ 780] blk.10.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 171/ 780] blk.10.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 172/ 780] blk.10.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 173/ 780] blk.10.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 174/ 780] blk.10.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 175/ 780] blk.10.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 176/ 780] blk.10.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 177/ 780] blk.10.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 178/ 780] blk.10.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 179/ 780] blk.10.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 180/ 780] blk.10.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 181/ 780] blk.10.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 182/ 780] blk.10.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 183/ 780] blk.10.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 184/ 780] blk.11.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 185/ 780] blk.11.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 186/ 780] blk.11.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 187/ 780] blk.11.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 188/ 780] blk.11.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 189/ 780] blk.11.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 190/ 780] blk.11.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 191/ 780] blk.11.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 192/ 780] blk.11.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 193/ 780] blk.11.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 194/ 780] blk.11.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 195/ 780] blk.11.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 196/ 780] blk.11.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 197/ 780] blk.11.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 198/ 780] blk.11.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 199/ 780] blk.11.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 200/ 780] blk.11.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 201/ 780] blk.12.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 202/ 780] blk.12.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 203/ 780] blk.12.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 204/ 780] blk.12.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 205/ 780] blk.12.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 206/ 780] blk.12.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 207/ 780] blk.12.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 208/ 780] blk.12.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 209/ 780] blk.12.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 210/ 780] blk.12.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 211/ 780] blk.12.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 212/ 780] blk.12.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 213/ 780] blk.12.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 214/ 780] blk.12.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 215/ 780] blk.12.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 216/ 780] blk.12.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 217/ 780] blk.12.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 218/ 780] blk.13.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 219/ 780] blk.13.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 220/ 780] blk.13.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 221/ 780] blk.13.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 222/ 780] blk.13.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 223/ 780] blk.13.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 224/ 780] blk.13.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 225/ 780] blk.13.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 226/ 780] blk.13.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 227/ 780] blk.13.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 228/ 780] blk.13.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 229/ 780] blk.13.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 230/ 780] blk.13.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 231/ 780] blk.13.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 232/ 780] blk.13.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 233/ 780] blk.13.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 234/ 780] blk.13.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 235/ 780] blk.14.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 236/ 780] blk.14.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 237/ 780] blk.14.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 238/ 780] blk.14.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 239/ 780] blk.14.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 240/ 780] blk.14.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 241/ 780] blk.14.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 242/ 780] blk.14.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 243/ 780] blk.14.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 244/ 780] blk.14.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 245/ 780] blk.14.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 246/ 780] blk.14.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 247/ 780] blk.14.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 248/ 780] blk.14.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 249/ 780] blk.14.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 250/ 780] blk.14.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 251/ 780] blk.14.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 252/ 780] blk.15.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 253/ 780] blk.15.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 254/ 780] blk.15.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 255/ 780] blk.15.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 256/ 780] blk.15.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 257/ 780] blk.15.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 258/ 780] blk.15.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 259/ 780] blk.15.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 260/ 780] blk.15.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 261/ 780] blk.15.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 262/ 780] blk.15.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 263/ 780] blk.15.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 264/ 780] blk.15.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 265/ 780] blk.15.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 266/ 780] blk.15.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 267/ 780] blk.15.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 268/ 780] blk.15.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 269/ 780] blk.16.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 270/ 780] blk.16.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 271/ 780] blk.16.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 272/ 780] blk.16.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 273/ 780] blk.16.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 274/ 780] blk.16.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 275/ 780] blk.16.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 276/ 780] blk.16.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 277/ 780] blk.16.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 278/ 780] blk.16.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 279/ 780] blk.16.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 280/ 780] blk.16.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 281/ 780] blk.16.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 282/ 780] blk.16.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 283/ 780] blk.16.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 284/ 780] blk.16.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 285/ 780] blk.16.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 286/ 780] blk.17.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 287/ 780] blk.17.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 288/ 780] blk.17.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 289/ 780] blk.17.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 290/ 780] blk.17.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 291/ 780] blk.17.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 292/ 780] blk.17.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 293/ 780] blk.17.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 294/ 780] blk.17.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 295/ 780] blk.17.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 296/ 780] blk.17.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 297/ 780] blk.17.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 298/ 780] blk.17.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 299/ 780] blk.17.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 300/ 780] blk.17.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 301/ 780] blk.17.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 302/ 780] blk.17.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 303/ 780] blk.18.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 304/ 780] blk.18.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 305/ 780] blk.18.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 306/ 780] blk.18.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 307/ 780] blk.18.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 308/ 780] blk.18.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 309/ 780] blk.18.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 310/ 780] blk.18.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 311/ 780] blk.18.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 312/ 780] blk.18.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 313/ 780] blk.18.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 314/ 780] blk.18.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 315/ 780] blk.18.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 316/ 780] blk.18.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 317/ 780] blk.18.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 318/ 780] blk.18.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 319/ 780] blk.18.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 320/ 780] blk.19.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 321/ 780] blk.19.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 322/ 780] blk.19.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 323/ 780] blk.19.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 324/ 780] blk.19.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 325/ 780] blk.19.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 326/ 780] blk.19.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 327/ 780] blk.19.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 328/ 780] blk.19.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 329/ 780] blk.19.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 330/ 780] blk.19.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 331/ 780] blk.19.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 332/ 780] blk.19.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 333/ 780] blk.19.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 334/ 780] blk.19.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 335/ 780] blk.19.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 336/ 780] blk.19.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 337/ 780] blk.20.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 338/ 780] blk.20.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 339/ 780] blk.20.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 340/ 780] blk.20.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 341/ 780] blk.20.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 342/ 780] blk.20.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 343/ 780] blk.20.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 344/ 780] blk.20.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 345/ 780] blk.20.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 346/ 780] blk.20.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 347/ 780] blk.20.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 348/ 780] blk.20.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 349/ 780] blk.20.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 350/ 780] blk.20.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 351/ 780] blk.20.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 352/ 780] blk.20.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 353/ 780] blk.20.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 354/ 780] blk.21.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 355/ 780] blk.21.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 356/ 780] blk.21.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 357/ 780] blk.21.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 358/ 780] blk.21.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 359/ 780] blk.21.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 360/ 780] blk.21.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 361/ 780] blk.21.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 362/ 780] blk.21.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 363/ 780] blk.21.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 364/ 780] blk.21.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 365/ 780] blk.21.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 366/ 780] blk.21.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 367/ 780] blk.21.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 368/ 780] blk.21.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 369/ 780] blk.21.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 370/ 780] blk.21.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 371/ 780] blk.22.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 372/ 780] blk.22.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 373/ 780] blk.22.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 374/ 780] blk.22.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 375/ 780] blk.22.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 376/ 780] blk.22.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 377/ 780] blk.22.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 378/ 780] blk.22.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 379/ 780] blk.22.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 380/ 780] blk.22.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 381/ 780] blk.22.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 382/ 780] blk.22.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 383/ 780] blk.22.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 384/ 780] blk.22.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 385/ 780] blk.22.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 386/ 780] blk.22.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 387/ 780] blk.22.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 388/ 780] blk.23.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 389/ 780] blk.23.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 390/ 780] blk.23.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 391/ 780] blk.23.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 392/ 780] blk.23.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 393/ 780] blk.23.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 394/ 780] blk.23.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 395/ 780] blk.23.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 396/ 780] blk.23.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 397/ 780] blk.23.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 398/ 780] blk.23.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 399/ 780] blk.23.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 400/ 780] blk.23.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 401/ 780] blk.23.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 402/ 780] blk.23.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 403/ 780] blk.23.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 404/ 780] blk.23.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 405/ 780] blk.24.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 406/ 780] blk.24.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 407/ 780] blk.24.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 408/ 780] blk.24.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 409/ 780] blk.24.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 410/ 780] blk.24.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 411/ 780] blk.24.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 412/ 780] blk.24.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 413/ 780] blk.24.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 414/ 780] blk.24.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 415/ 780] blk.24.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 416/ 780] blk.24.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 417/ 780] blk.24.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 418/ 780] blk.24.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 419/ 780] blk.24.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 420/ 780] blk.24.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 421/ 780] blk.24.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 422/ 780] blk.25.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 423/ 780] blk.25.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 424/ 780] blk.25.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 425/ 780] blk.25.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 426/ 780] blk.25.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 427/ 780] blk.25.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 428/ 780] blk.25.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 429/ 780] blk.25.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 430/ 780] blk.25.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 431/ 780] blk.25.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 432/ 780] blk.25.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 433/ 780] blk.25.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 434/ 780] blk.25.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 435/ 780] blk.25.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 436/ 780] blk.25.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 437/ 780] blk.25.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 438/ 780] blk.25.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 439/ 780] blk.26.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 440/ 780] blk.26.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 441/ 780] blk.26.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 442/ 780] blk.26.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 443/ 780] blk.26.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 444/ 780] blk.26.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 445/ 780] blk.26.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 446/ 780] blk.26.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 447/ 780] blk.26.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 448/ 780] blk.26.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 449/ 780] blk.26.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 450/ 780] blk.26.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 451/ 780] blk.26.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 452/ 780] blk.26.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 453/ 780] blk.26.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 454/ 780] blk.26.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 455/ 780] blk.26.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 456/ 780] blk.27.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 457/ 780] blk.27.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 458/ 780] blk.27.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 459/ 780] blk.27.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 460/ 780] blk.27.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 461/ 780] blk.27.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 462/ 780] blk.27.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 463/ 780] blk.27.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 464/ 780] blk.27.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 465/ 780] blk.27.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 466/ 780] blk.27.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 467/ 780] blk.27.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 468/ 780] blk.27.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 469/ 780] blk.27.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 470/ 780] blk.27.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 471/ 780] blk.27.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 472/ 780] blk.27.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 473/ 780] blk.28.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 474/ 780] blk.28.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 475/ 780] blk.28.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 476/ 780] blk.28.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 477/ 780] blk.28.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 478/ 780] blk.28.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 479/ 780] blk.28.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 480/ 780] blk.28.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 481/ 780] blk.28.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 482/ 780] blk.28.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 483/ 780] blk.28.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 484/ 780] blk.28.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 485/ 780] blk.28.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 486/ 780] blk.28.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 487/ 780] blk.28.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 488/ 780] blk.28.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 489/ 780] blk.28.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 490/ 780] blk.29.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 491/ 780] blk.29.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 492/ 780] blk.29.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 493/ 780] blk.29.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 494/ 780] blk.29.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 495/ 780] blk.29.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 496/ 780] blk.29.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 497/ 780] blk.29.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 498/ 780] blk.29.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 499/ 780] blk.29.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 500/ 780] blk.29.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 501/ 780] blk.29.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 502/ 780] blk.29.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 503/ 780] blk.29.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 504/ 780] blk.29.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 505/ 780] blk.29.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 506/ 780] blk.29.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 507/ 780] blk.30.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 508/ 780] blk.30.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 509/ 780] blk.30.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 510/ 780] blk.30.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 511/ 780] blk.30.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 512/ 780] blk.30.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 513/ 780] blk.30.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 514/ 780] blk.30.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 515/ 780] blk.30.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 516/ 780] blk.30.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 517/ 780] blk.30.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 518/ 780] blk.30.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 519/ 780] blk.30.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 520/ 780] blk.30.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 521/ 780] blk.30.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 522/ 780] blk.30.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 523/ 780] blk.30.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 524/ 780] blk.31.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 525/ 780] blk.31.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 526/ 780] blk.31.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 527/ 780] blk.31.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 528/ 780] blk.31.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 529/ 780] blk.31.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 530/ 780] blk.31.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 531/ 780] blk.31.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 532/ 780] blk.31.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 533/ 780] blk.31.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 534/ 780] blk.31.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 535/ 780] blk.31.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 536/ 780] blk.31.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 537/ 780] blk.31.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 538/ 780] blk.31.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 539/ 780] blk.31.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 540/ 780] blk.31.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 541/ 780] blk.32.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 542/ 780] blk.32.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 543/ 780] blk.32.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 544/ 780] blk.32.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 545/ 780] blk.32.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 546/ 780] blk.32.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 547/ 780] blk.32.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 548/ 780] blk.32.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 549/ 780] blk.32.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 550/ 780] blk.32.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 551/ 780] blk.32.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 552/ 780] blk.32.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 553/ 780] blk.32.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 554/ 780] blk.32.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 555/ 780] blk.32.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 556/ 780] blk.32.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 557/ 780] blk.32.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 558/ 780] blk.33.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 559/ 780] blk.33.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 560/ 780] blk.33.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 561/ 780] blk.33.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 562/ 780] blk.33.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 563/ 780] blk.33.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 564/ 780] blk.33.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 565/ 780] blk.33.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 566/ 780] blk.33.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 567/ 780] blk.33.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 568/ 780] blk.33.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 569/ 780] blk.33.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 570/ 780] blk.33.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 571/ 780] blk.33.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 572/ 780] blk.33.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 573/ 780] blk.33.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 574/ 780] blk.33.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 575/ 780] blk.34.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 576/ 780] blk.34.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 577/ 780] blk.34.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 578/ 780] blk.34.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 579/ 780] blk.34.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 580/ 780] blk.34.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 581/ 780] blk.34.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 582/ 780] blk.34.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 583/ 780] blk.34.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 584/ 780] blk.34.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 585/ 780] blk.34.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 586/ 780] blk.34.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 587/ 780] blk.34.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 588/ 780] blk.34.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 589/ 780] blk.34.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 590/ 780] blk.34.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 591/ 780] blk.34.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 592/ 780] blk.35.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 593/ 780] blk.35.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 594/ 780] blk.35.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 595/ 780] blk.35.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 596/ 780] blk.35.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 597/ 780] blk.35.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 598/ 780] blk.35.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 599/ 780] blk.35.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 600/ 780] blk.35.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 601/ 780] blk.35.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 602/ 780] blk.35.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 603/ 780] blk.35.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 604/ 780] blk.35.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 605/ 780] blk.35.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 606/ 780] blk.35.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 607/ 780] blk.35.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 608/ 780] blk.35.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 609/ 780] blk.36.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 610/ 780] blk.36.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 611/ 780] blk.36.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 612/ 780] blk.36.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 613/ 780] blk.36.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 614/ 780] blk.36.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 615/ 780] blk.36.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 616/ 780] blk.36.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 617/ 780] blk.36.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 618/ 780] blk.36.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 619/ 780] blk.36.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 620/ 780] blk.36.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 621/ 780] blk.36.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 622/ 780] blk.36.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 623/ 780] blk.36.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 624/ 780] blk.36.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 625/ 780] blk.36.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 626/ 780] blk.37.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 627/ 780] blk.37.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 628/ 780] blk.37.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 629/ 780] blk.37.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 630/ 780] blk.37.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 631/ 780] blk.37.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 632/ 780] blk.37.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 633/ 780] blk.37.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 634/ 780] blk.37.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 635/ 780] blk.37.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 636/ 780] blk.37.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 637/ 780] blk.37.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 638/ 780] blk.37.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 639/ 780] blk.37.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 640/ 780] blk.37.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 641/ 780] blk.37.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 642/ 780] blk.37.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 643/ 780] blk.38.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 644/ 780] blk.38.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 645/ 780] blk.38.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 646/ 780] blk.38.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 647/ 780] blk.38.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 648/ 780] blk.38.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 649/ 780] blk.38.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 650/ 780] blk.38.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 651/ 780] blk.38.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 652/ 780] blk.38.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 653/ 780] blk.38.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 654/ 780] blk.38.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 655/ 780] blk.38.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 656/ 780] blk.38.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 657/ 780] blk.38.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 658/ 780] blk.38.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 659/ 780] blk.38.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 660/ 780] blk.39.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 661/ 780] blk.39.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 662/ 780] blk.39.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 663/ 780] blk.39.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 664/ 780] blk.39.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 665/ 780] blk.39.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 666/ 780] blk.39.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 667/ 780] blk.39.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 668/ 780] blk.39.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 669/ 780] blk.39.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 670/ 780] blk.39.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 671/ 780] blk.39.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 672/ 780] blk.39.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 673/ 780] blk.39.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 674/ 780] blk.39.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 675/ 780] blk.39.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 676/ 780] blk.39.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 677/ 780] blk.40.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 678/ 780] blk.40.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 679/ 780] blk.40.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 680/ 780] blk.40.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 681/ 780] blk.40.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 682/ 780] blk.40.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 683/ 780] blk.40.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 684/ 780] blk.40.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 685/ 780] blk.40.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 686/ 780] blk.40.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 687/ 780] blk.40.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 688/ 780] blk.40.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 689/ 780] blk.40.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 690/ 780] blk.40.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 691/ 780] blk.40.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 692/ 780] blk.40.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 693/ 780] blk.40.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 694/ 780] blk.41.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 695/ 780] blk.41.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 696/ 780] blk.41.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 697/ 780] blk.41.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 698/ 780] blk.41.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 699/ 780] blk.41.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 700/ 780] blk.41.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 701/ 780] blk.41.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 702/ 780] blk.41.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 703/ 780] blk.41.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 704/ 780] blk.41.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 705/ 780] blk.41.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 706/ 780] blk.41.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 707/ 780] blk.41.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 708/ 780] blk.41.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 709/ 780] blk.41.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 710/ 780] blk.41.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 711/ 780] blk.42.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 712/ 780] blk.42.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 713/ 780] blk.42.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 714/ 780] blk.42.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 715/ 780] blk.42.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 716/ 780] blk.42.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 717/ 780] blk.42.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 718/ 780] blk.42.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 719/ 780] blk.42.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 720/ 780] blk.42.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 721/ 780] blk.42.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 722/ 780] blk.42.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 723/ 780] blk.42.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 724/ 780] blk.42.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 725/ 780] blk.42.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 726/ 780] blk.42.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 727/ 780] blk.42.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 728/ 780] blk.43.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 729/ 780] blk.43.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 730/ 780] blk.43.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 731/ 780] blk.43.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 732/ 780] blk.43.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 733/ 780] blk.43.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 734/ 780] blk.43.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 735/ 780] blk.43.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 736/ 780] blk.43.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 737/ 780] blk.43.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 738/ 780] blk.43.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 739/ 780] blk.43.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 740/ 780] blk.43.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 741/ 780] blk.43.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 742/ 780] blk.43.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 743/ 780] blk.43.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 744/ 780] blk.43.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 745/ 780] blk.44.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 746/ 780] blk.44.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 747/ 780] blk.44.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 748/ 780] blk.44.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 749/ 780] blk.44.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 750/ 780] blk.44.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 751/ 780] blk.44.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 752/ 780] blk.44.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 753/ 780] blk.44.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 754/ 780] blk.44.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 755/ 780] blk.44.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 756/ 780] blk.44.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 757/ 780] blk.44.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 758/ 780] blk.44.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 759/ 780] blk.44.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 760/ 780] blk.44.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 761/ 780] blk.44.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 762/ 780] output.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q6_K .. size = 1184.00 MiB -> 485.62 MiB [ 763/ 780] blk.45.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 764/ 780] blk.45.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 765/ 780] blk.45.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 766/ 780] blk.45.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_K .. size = 1408.00 MiB -> 396.00 MiB [ 767/ 780] blk.45.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 768/ 780] blk.45.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 769/ 780] blk.45.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 770/ 780] blk.45.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 771/ 780] blk.45.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 772/ 780] blk.45.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 773/ 780] blk.45.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 774/ 780] blk.45.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 775/ 780] blk.45.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 776/ 780] blk.45.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 777/ 780] blk.45.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 778/ 780] blk.45.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 779/ 780] blk.45.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 780/ 780] output_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB llama_model_quantize_impl: model size = 203851.49 MiB (16.00 BPW) llama_model_quantize_impl: quant size = 60676.79 MiB (4.76 BPW) llama_quantize: quantize time = 1076918.57 ms llama_quantize: total time = 1076918.57 ms + ./llama.cpp/build/bin/llama-quantize ./upload-GLM_4.5_Air/mtp-GLM-4.5-Air-BF16.gguf ./upload-GLM_4.5_Air/mtp-GLM-4.5-Air-Q8_0.gguf Q8_0 version: 0.3.0-dev (build 1, commit c1d0e7a) built with GNU 14.2.0 for Linux x86_64 llama_quantize: quantizing './upload-GLM_4.5_Air/mtp-GLM-4.5-Air-BF16.gguf' to './upload-GLM_4.5_Air/mtp-GLM-4.5-Air-Q8_0.gguf' as Q8_0 llama_model_loader: loaded meta data with 43 key-value pairs and 26 tensors from ./upload-GLM_4.5_Air/mtp-GLM-4.5-Air-BF16.gguf (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = glm4moe llama_model_loader: - kv 1: general.type str = model llama_model_loader: - kv 2: general.name str = GLM-4.5-Air llama_model_loader: - kv 3: general.size_label str = 128x2.7B llama_model_loader: - kv 4: general.license str = mit llama_model_loader: - kv 5: general.tags arr[str,1] = ["text-generation"] llama_model_loader: - kv 6: general.languages arr[str,2] = ["en", "zh"] llama_model_loader: - kv 7: glm4moe.block_count u32 = 47 llama_model_loader: - kv 8: glm4moe.context_length u32 = 131072 llama_model_loader: - kv 9: glm4moe.embedding_length u32 = 4096 llama_model_loader: - kv 10: glm4moe.feed_forward_length u32 = 10944 llama_model_loader: - kv 11: glm4moe.attention.head_count u32 = 96 llama_model_loader: - kv 12: glm4moe.attention.head_count_kv u32 = 8 llama_model_loader: - kv 13: glm4moe.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 14: glm4moe.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 15: glm4moe.expert_count u32 = 128 llama_model_loader: - kv 16: glm4moe.expert_used_count u32 = 8 llama_model_loader: - kv 17: glm4moe.expert_group_count u32 = 1 llama_model_loader: - kv 18: glm4moe.expert_group_used_count u32 = 1 llama_model_loader: - kv 19: glm4moe.attention.key_length u32 = 128 llama_model_loader: - kv 20: glm4moe.attention.value_length u32 = 128 llama_model_loader: - kv 21: general.file_type u32 = 32 llama_model_loader: - kv 22: glm4moe.rope.dimension_count u32 = 64 llama_model_loader: - kv 23: glm4moe.expert_feed_forward_length u32 = 1408 llama_model_loader: - kv 24: glm4moe.expert_shared_count u32 = 1 llama_model_loader: - kv 25: glm4moe.leading_dense_block_count u32 = 1 llama_model_loader: - kv 26: glm4moe.expert_gating_func u32 = 2 llama_model_loader: - kv 27: glm4moe.expert_weights_scale f32 = 1.000000 llama_model_loader: - kv 28: glm4moe.expert_weights_norm bool = true llama_model_loader: - kv 29: glm4moe.nextn_predict_layers u32 = 1 llama_model_loader: - kv 30: general.quantization_version u32 = 2 llama_model_loader: - kv 31: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 32: tokenizer.ggml.pre str = glm4 llama_model_loader: - kv 33: tokenizer.ggml.tokens arr[str,151552] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 34: tokenizer.ggml.token_type arr[i32,151552] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 35: tokenizer.ggml.merges arr[str,318088] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "... llama_model_loader: - kv 36: tokenizer.ggml.eos_token_id u32 = 151329 llama_model_loader: - kv 37: tokenizer.ggml.padding_token_id u32 = 151329 llama_model_loader: - kv 38: tokenizer.ggml.bos_token_id u32 = 151331 llama_model_loader: - kv 39: tokenizer.ggml.eot_token_id u32 = 151336 llama_model_loader: - kv 40: tokenizer.ggml.unknown_token_id u32 = 151329 llama_model_loader: - kv 41: tokenizer.ggml.eom_token_id u32 = 151338 llama_model_loader: - kv 42: tokenizer.chat_template str = [gMASK]\n{%- if tools -%}\n<|syste... llama_model_loader: - type f32: 11 tensors llama_model_loader: - type bf16: 15 tensors [ 1/ 26] output.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q8_0 .. size = 1184.00 MiB -> 629.00 MiB [ 2/ 26] output_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 3/ 26] token_embd.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q8_0 .. size = 1184.00 MiB -> 629.00 MiB [ 4/ 26] blk.46.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 5/ 26] blk.46.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 6/ 26] blk.46.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 7/ 26] blk.46.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 8/ 26] blk.46.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 9/ 26] blk.46.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q8_0 .. size = 96.00 MiB -> 51.00 MiB [ 10/ 26] blk.46.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 11/ 26] blk.46.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q8_0 .. size = 8.00 MiB -> 4.25 MiB [ 12/ 26] blk.46.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 13/ 26] blk.46.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 14/ 26] blk.46.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 15/ 26] blk.46.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 16/ 26] blk.46.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 17/ 26] blk.46.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 18/ 26] blk.46.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q8_0 .. size = 1408.00 MiB -> 748.00 MiB [ 19/ 26] blk.46.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q8_0 .. size = 11.00 MiB -> 5.84 MiB [ 20/ 26] blk.46.nextn.eh_proj.weight - [ 8192, 4096, 1, 1], type = bf16, converting to q8_0 .. size = 64.00 MiB -> 34.00 MiB [ 21/ 26] blk.46.nextn.embed_tokens.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q8_0 .. size = 1184.00 MiB -> 629.00 MiB [ 22/ 26] blk.46.nextn.enorm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 23/ 26] blk.46.nextn.hnorm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 24/ 26] blk.46.nextn.shared_head_head.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q8_0 .. size = 1184.00 MiB -> 629.00 MiB [ 25/ 26] blk.46.nextn.shared_head_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 26/ 26] blk.46.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB llama_model_quantize_impl: model size = 9267.15 MiB (16.00 BPW) llama_model_quantize_impl: quant size = 4924.18 MiB (8.50 BPW) llama_quantize: quantize time = 41421.57 ms llama_quantize: total time = 41421.57 ms + ./llama.cpp/build/bin/llama-quantize --pure ./upload-GLM_4.5_Air/mtp-GLM-4.5-Air-BF16.gguf ./upload-GLM_4.5_Air/mtp-GLM-4.5-Air-Q4_0.gguf Q4_0 version: 0.3.0-dev (build 1, commit c1d0e7a) built with GNU 14.2.0 for Linux x86_64 llama_quantize: quantizing './upload-GLM_4.5_Air/mtp-GLM-4.5-Air-BF16.gguf' to './upload-GLM_4.5_Air/mtp-GLM-4.5-Air-Q4_0.gguf' as Q4_0 llama_model_loader: loaded meta data with 43 key-value pairs and 26 tensors from ./upload-GLM_4.5_Air/mtp-GLM-4.5-Air-BF16.gguf (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = glm4moe llama_model_loader: - kv 1: general.type str = model llama_model_loader: - kv 2: general.name str = GLM-4.5-Air llama_model_loader: - kv 3: general.size_label str = 128x2.7B llama_model_loader: - kv 4: general.license str = mit llama_model_loader: - kv 5: general.tags arr[str,1] = ["text-generation"] llama_model_loader: - kv 6: general.languages arr[str,2] = ["en", "zh"] llama_model_loader: - kv 7: glm4moe.block_count u32 = 47 llama_model_loader: - kv 8: glm4moe.context_length u32 = 131072 llama_model_loader: - kv 9: glm4moe.embedding_length u32 = 4096 llama_model_loader: - kv 10: glm4moe.feed_forward_length u32 = 10944 llama_model_loader: - kv 11: glm4moe.attention.head_count u32 = 96 llama_model_loader: - kv 12: glm4moe.attention.head_count_kv u32 = 8 llama_model_loader: - kv 13: glm4moe.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 14: glm4moe.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 15: glm4moe.expert_count u32 = 128 llama_model_loader: - kv 16: glm4moe.expert_used_count u32 = 8 llama_model_loader: - kv 17: glm4moe.expert_group_count u32 = 1 llama_model_loader: - kv 18: glm4moe.expert_group_used_count u32 = 1 llama_model_loader: - kv 19: glm4moe.attention.key_length u32 = 128 llama_model_loader: - kv 20: glm4moe.attention.value_length u32 = 128 llama_model_loader: - kv 21: general.file_type u32 = 32 llama_model_loader: - kv 22: glm4moe.rope.dimension_count u32 = 64 llama_model_loader: - kv 23: glm4moe.expert_feed_forward_length u32 = 1408 llama_model_loader: - kv 24: glm4moe.expert_shared_count u32 = 1 llama_model_loader: - kv 25: glm4moe.leading_dense_block_count u32 = 1 llama_model_loader: - kv 26: glm4moe.expert_gating_func u32 = 2 llama_model_loader: - kv 27: glm4moe.expert_weights_scale f32 = 1.000000 llama_model_loader: - kv 28: glm4moe.expert_weights_norm bool = true llama_model_loader: - kv 29: glm4moe.nextn_predict_layers u32 = 1 llama_model_loader: - kv 30: general.quantization_version u32 = 2 llama_model_loader: - kv 31: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 32: tokenizer.ggml.pre str = glm4 llama_model_loader: - kv 33: tokenizer.ggml.tokens arr[str,151552] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 34: tokenizer.ggml.token_type arr[i32,151552] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 35: tokenizer.ggml.merges arr[str,318088] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "... llama_model_loader: - kv 36: tokenizer.ggml.eos_token_id u32 = 151329 llama_model_loader: - kv 37: tokenizer.ggml.padding_token_id u32 = 151329 llama_model_loader: - kv 38: tokenizer.ggml.bos_token_id u32 = 151331 llama_model_loader: - kv 39: tokenizer.ggml.eot_token_id u32 = 151336 llama_model_loader: - kv 40: tokenizer.ggml.unknown_token_id u32 = 151329 llama_model_loader: - kv 41: tokenizer.ggml.eom_token_id u32 = 151338 llama_model_loader: - kv 42: tokenizer.chat_template str = [gMASK]\n{%- if tools -%}\n<|syste... llama_model_loader: - type f32: 11 tensors llama_model_loader: - type bf16: 15 tensors [ 1/ 26] output.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q4_0 .. size = 1184.00 MiB -> 333.00 MiB [ 2/ 26] output_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 3/ 26] token_embd.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q4_0 .. size = 1184.00 MiB -> 333.00 MiB [ 4/ 26] blk.46.attn_k.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 5/ 26] blk.46.attn_k.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q4_0 .. size = 8.00 MiB -> 2.25 MiB [ 6/ 26] blk.46.attn_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 7/ 26] blk.46.attn_output.weight - [ 12288, 4096, 1, 1], type = bf16, converting to q4_0 .. size = 96.00 MiB -> 27.00 MiB [ 8/ 26] blk.46.attn_q.bias - [ 12288, 1, 1, 1], type = f32, size = 0.047 MiB [ 9/ 26] blk.46.attn_q.weight - [ 4096, 12288, 1, 1], type = bf16, converting to q4_0 .. size = 96.00 MiB -> 27.00 MiB [ 10/ 26] blk.46.attn_v.bias - [ 1024, 1, 1, 1], type = f32, size = 0.004 MiB [ 11/ 26] blk.46.attn_v.weight - [ 4096, 1024, 1, 1], type = bf16, converting to q4_0 .. size = 8.00 MiB -> 2.25 MiB [ 12/ 26] blk.46.exp_probs_b.bias - [ 128, 1, 1, 1], type = f32, size = 0.000 MiB [ 13/ 26] blk.46.ffn_down_exps.weight - [ 1408, 4096, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 14/ 26] blk.46.ffn_down_shexp.weight - [ 1408, 4096, 1, 1], type = bf16, converting to q4_0 .. size = 11.00 MiB -> 3.09 MiB [ 15/ 26] blk.46.ffn_gate_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 16/ 26] blk.46.ffn_gate_inp.weight - [ 4096, 128, 1, 1], type = f32, size = 2.000 MiB [ 17/ 26] blk.46.ffn_gate_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q4_0 .. size = 11.00 MiB -> 3.09 MiB [ 18/ 26] blk.46.ffn_up_exps.weight - [ 4096, 1408, 128, 1], type = bf16, converting to q4_0 .. size = 1408.00 MiB -> 396.00 MiB [ 19/ 26] blk.46.ffn_up_shexp.weight - [ 4096, 1408, 1, 1], type = bf16, converting to q4_0 .. size = 11.00 MiB -> 3.09 MiB [ 20/ 26] blk.46.nextn.eh_proj.weight - [ 8192, 4096, 1, 1], type = bf16, converting to q4_0 .. size = 64.00 MiB -> 18.00 MiB [ 21/ 26] blk.46.nextn.embed_tokens.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q4_0 .. size = 1184.00 MiB -> 333.00 MiB [ 22/ 26] blk.46.nextn.enorm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 23/ 26] blk.46.nextn.hnorm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 24/ 26] blk.46.nextn.shared_head_head.weight - [ 4096, 151552, 1, 1], type = bf16, converting to q4_0 .. size = 1184.00 MiB -> 333.00 MiB [ 25/ 26] blk.46.nextn.shared_head_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB [ 26/ 26] blk.46.post_attention_norm.weight - [ 4096, 1, 1, 1], type = f32, size = 0.016 MiB llama_model_quantize_impl: model size = 9267.15 MiB (16.00 BPW) llama_model_quantize_impl: quant size = 2607.93 MiB (4.50 BPW) llama_quantize: quantize time = 12801.47 ms llama_quantize: total time = 12801.47 ms + echo GLM-4.5-Air-Q8_0-00001-of-00002.gguf + echo GLM-4.5-Air-Q8_0-00002-of-00002.gguf + echo GLM-4.5-Air-Q4_K_M-00001-of-00002.gguf + echo GLM-4.5-Air-Q4_K_M-00002-of-00002.gguf + echo mtp-GLM-4.5-Air-Q8_0.gguf + echo mtp-GLM-4.5-Air-Q4_0.gguf