[INFO|2026-02-08 23:10:53] configuration_utils.py:667 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/config.json
[INFO|2026-02-08 23:10:53] configuration_utils.py:739 >> Model config Glm4Config {
  "architectures": [
    "Glm4ForCausalLM"
  ],
  "attention_bias": true,
  "attention_dropout": 0.0,
  "bos_token_id": null,
  "dtype": "bfloat16",
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 13696,
  "max_position_embeddings": 32768,
  "model_type": "glm4",
  "num_attention_heads": 32,
  "num_hidden_layers": 40,
  "num_key_value_heads": 2,
  "pad_token_id": 151329,
  "partial_rotary_factor": 0.5,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "partial_rotary_factor": 0.5,
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.0.0",
  "use_cache": true,
  "vocab_size": 151552
}

[INFO|2026-02-08 23:11:01] configuration_utils.py:667 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/config.json
[INFO|2026-02-08 23:11:01] configuration_utils.py:739 >> Model config Glm4Config {
  "architectures": [
    "Glm4ForCausalLM"
  ],
  "attention_bias": true,
  "attention_dropout": 0.0,
  "bos_token_id": null,
  "dtype": "bfloat16",
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 13696,
  "max_position_embeddings": 32768,
  "model_type": "glm4",
  "num_attention_heads": 32,
  "num_hidden_layers": 40,
  "num_key_value_heads": 2,
  "pad_token_id": 151329,
  "partial_rotary_factor": 0.5,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "partial_rotary_factor": 0.5,
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.0.0",
  "use_cache": true,
  "vocab_size": 151552
}

[INFO|2026-02-08 23:11:01] configuration_utils.py:667 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/config.json
[INFO|2026-02-08 23:11:01] configuration_utils.py:739 >> Model config Glm4Config {
  "architectures": [
    "Glm4ForCausalLM"
  ],
  "attention_bias": true,
  "attention_dropout": 0.0,
  "bos_token_id": null,
  "dtype": "bfloat16",
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 13696,
  "max_position_embeddings": 32768,
  "model_type": "glm4",
  "num_attention_heads": 32,
  "num_hidden_layers": 40,
  "num_key_value_heads": 2,
  "pad_token_id": 151329,
  "partial_rotary_factor": 0.5,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "partial_rotary_factor": 0.5,
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.0.0",
  "use_cache": true,
  "vocab_size": 151552
}

[INFO|2026-02-08 23:11:04] logging.py:144 >> Loading dataset train_split2.json...
[INFO|2026-02-08 23:11:12] configuration_utils.py:667 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/config.json
[INFO|2026-02-08 23:11:12] configuration_utils.py:739 >> Model config Glm4Config {
  "architectures": [
    "Glm4ForCausalLM"
  ],
  "attention_bias": true,
  "attention_dropout": 0.0,
  "bos_token_id": null,
  "dtype": "bfloat16",
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 13696,
  "max_position_embeddings": 32768,
  "model_type": "glm4",
  "num_attention_heads": 32,
  "num_hidden_layers": 40,
  "num_key_value_heads": 2,
  "pad_token_id": 151329,
  "partial_rotary_factor": 0.5,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "partial_rotary_factor": 0.5,
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.0.0",
  "use_cache": true,
  "vocab_size": 151552
}

[INFO|2026-02-08 23:11:12] logging.py:144 >> Quantizing model to 4 bit with bitsandbytes.
[INFO|2026-02-08 23:11:12] logging.py:144 >> KV cache is disabled during training.
[INFO|2026-02-08 23:11:20] configuration_utils.py:667 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/config.json
[INFO|2026-02-08 23:11:20] configuration_utils.py:739 >> Model config Glm4Config {
  "architectures": [
    "Glm4ForCausalLM"
  ],
  "attention_bias": true,
  "attention_dropout": 0.0,
  "bos_token_id": null,
  "dtype": "bfloat16",
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 13696,
  "max_position_embeddings": 32768,
  "model_type": "glm4",
  "num_attention_heads": 32,
  "num_hidden_layers": 40,
  "num_key_value_heads": 2,
  "pad_token_id": 151329,
  "partial_rotary_factor": 0.5,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "partial_rotary_factor": 0.5,
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.0.0",
  "use_cache": true,
  "vocab_size": 151552
}

[INFO|2026-02-08 23:11:22] configuration_utils.py:667 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/config.json
[INFO|2026-02-08 23:11:22] configuration_utils.py:739 >> Model config Glm4Config {
  "architectures": [
    "Glm4ForCausalLM"
  ],
  "attention_bias": true,
  "attention_dropout": 0.0,
  "bos_token_id": null,
  "dtype": "bfloat16",
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 13696,
  "max_position_embeddings": 32768,
  "model_type": "glm4",
  "num_attention_heads": 32,
  "num_hidden_layers": 40,
  "num_key_value_heads": 2,
  "pad_token_id": 151329,
  "partial_rotary_factor": 0.5,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "partial_rotary_factor": 0.5,
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.0.0",
  "use_cache": true,
  "vocab_size": 151552
}

[INFO|2026-02-08 23:11:31] configuration_utils.py:667 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/config.json
[INFO|2026-02-08 23:11:31] configuration_utils.py:739 >> Model config Glm4Config {
  "architectures": [
    "Glm4ForCausalLM"
  ],
  "attention_bias": true,
  "attention_dropout": 0.0,
  "bos_token_id": null,
  "dtype": "bfloat16",
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 13696,
  "max_position_embeddings": 32768,
  "model_type": "glm4",
  "num_attention_heads": 32,
  "num_hidden_layers": 40,
  "num_key_value_heads": 2,
  "pad_token_id": 151329,
  "partial_rotary_factor": 0.5,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "partial_rotary_factor": 0.5,
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.0.0",
  "use_cache": true,
  "vocab_size": 151552
}

[INFO|2026-02-08 23:11:31] modeling_utils.py:732 >> loading weights file model.safetensors from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/model.safetensors.index.json
[INFO|2026-02-08 23:12:17] configuration_utils.py:1014 >> Generate config GenerationConfig {
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "output_attentions": false,
  "output_hidden_states": false,
  "pad_token_id": 151329,
  "use_cache": true
}

[INFO|2026-02-08 23:12:24] configuration_utils.py:967 >> loading configuration file generation_config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/generation_config.json
[INFO|2026-02-08 23:12:24] configuration_utils.py:1014 >> Generate config GenerationConfig {
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "pad_token_id": 151329
}

[INFO|2026-02-08 23:12:24] dynamic_module_utils.py:406 >> Could not locate the custom_generate/generate.py inside zai-org/GLM-Z1-9B-0414.
[INFO|2026-02-08 23:12:25] configuration_utils.py:667 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/config.json
[INFO|2026-02-08 23:12:25] configuration_utils.py:739 >> Model config Glm4Config {
  "architectures": [
    "Glm4ForCausalLM"
  ],
  "attention_bias": true,
  "attention_dropout": 0.0,
  "bos_token_id": null,
  "dtype": "bfloat16",
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 13696,
  "max_position_embeddings": 32768,
  "model_type": "glm4",
  "num_attention_heads": 32,
  "num_hidden_layers": 40,
  "num_key_value_heads": 2,
  "pad_token_id": 151329,
  "partial_rotary_factor": 0.5,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "partial_rotary_factor": 0.5,
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.0.0",
  "use_cache": true,
  "vocab_size": 151552
}

[INFO|2026-02-08 23:12:31] logging.py:144 >> Gradient checkpointing enabled.
[INFO|2026-02-08 23:12:31] logging.py:144 >> Upcasting trainable params to float32.
[INFO|2026-02-08 23:12:31] logging.py:144 >> Fine-tuning method: LoRA
[INFO|2026-02-08 23:12:31] logging.py:144 >> Found linear modules: v_proj,down_proj,k_proj,gate_up_proj,q_proj,o_proj
[INFO|2026-02-08 23:12:35] logging.py:144 >> trainable params: 95,191,040 || all params: 9,495,470,080 || trainable%: 1.0025
[INFO|2026-02-08 23:12:36] trainer.py:1260 >> skipped Embedding(151552, 4096, padding_idx=151329): 592.0M params
[INFO|2026-02-08 23:12:36] trainer.py:1263 >> skipped: 592.0M params
[INFO|2026-02-08 23:12:36] trainer.py:2383 >> ***** Running training *****
[INFO|2026-02-08 23:12:36] trainer.py:2384 >>   Num examples = 3,716
[INFO|2026-02-08 23:12:36] trainer.py:2385 >>   Num Epochs = 3
[INFO|2026-02-08 23:12:36] trainer.py:2386 >>   Instantaneous batch size per device = 4
[INFO|2026-02-08 23:12:36] trainer.py:2389 >>   Total train batch size (w. parallel, distributed & accumulation) = 32
[INFO|2026-02-08 23:12:36] trainer.py:2390 >>   Gradient Accumulation steps = 8
[INFO|2026-02-08 23:12:36] trainer.py:2391 >>   Total optimization steps = 351
[INFO|2026-02-08 23:12:36] trainer.py:2392 >>   Number of trainable parameters = 95,191,040
[INFO|2026-02-08 23:14:12] logging.py:144 >> {'loss': 9.8036, 'learning_rate': 9.0000e-07, 'epoch': 0.09, 'throughput': 933.70}
[INFO|2026-02-08 23:15:17] logging.py:144 >> {'loss': 7.2789, 'learning_rate': 1.9000e-06, 'epoch': 0.17, 'throughput': 1117.95}
[INFO|2026-02-08 23:16:21] logging.py:144 >> {'loss': 4.2772, 'learning_rate': 2.9000e-06, 'epoch': 0.26, 'throughput': 1201.30}
[INFO|2026-02-08 23:17:25] logging.py:144 >> {'loss': 1.2959, 'learning_rate': 3.9000e-06, 'epoch': 0.34, 'throughput': 1250.75}
[INFO|2026-02-08 23:18:30] logging.py:144 >> {'loss': 0.3324, 'learning_rate': 4.9000e-06, 'epoch': 0.43, 'throughput': 1278.65}
[INFO|2026-02-08 23:19:34] logging.py:144 >> {'loss': 0.2573, 'learning_rate': 5.9000e-06, 'epoch': 0.52, 'throughput': 1297.77}
[INFO|2026-02-08 23:20:38] logging.py:144 >> {'loss': 0.2211, 'learning_rate': 6.9000e-06, 'epoch': 0.60, 'throughput': 1314.50}
[INFO|2026-02-08 23:21:41] logging.py:144 >> {'loss': 0.2148, 'learning_rate': 7.9000e-06, 'epoch': 0.69, 'throughput': 1327.02}
[INFO|2026-02-08 23:22:45] logging.py:144 >> {'loss': 0.1861, 'learning_rate': 8.9000e-06, 'epoch': 0.78, 'throughput': 1338.10}
[INFO|2026-02-08 23:23:49] logging.py:144 >> {'loss': 0.1617, 'learning_rate': 9.9000e-06, 'epoch': 0.86, 'throughput': 1344.54}
[INFO|2026-02-08 23:24:54] logging.py:144 >> {'loss': 0.1681, 'learning_rate': 1.0900e-05, 'epoch': 0.95, 'throughput': 1348.73}
[INFO|2026-02-08 23:25:53] logging.py:144 >> {'loss': 0.1580, 'learning_rate': 1.1900e-05, 'epoch': 1.03, 'throughput': 1352.22}
[INFO|2026-02-08 23:26:57] logging.py:144 >> {'loss': 0.1583, 'learning_rate': 1.2900e-05, 'epoch': 1.11, 'throughput': 1357.20}
[INFO|2026-02-08 23:28:01] logging.py:144 >> {'loss': 0.1407, 'learning_rate': 1.3900e-05, 'epoch': 1.20, 'throughput': 1360.95}
[INFO|2026-02-08 23:29:05] logging.py:144 >> {'loss': 0.1532, 'learning_rate': 1.4900e-05, 'epoch': 1.28, 'throughput': 1363.36}
[INFO|2026-02-08 23:30:09] logging.py:144 >> {'loss': 0.1475, 'learning_rate': 1.5900e-05, 'epoch': 1.37, 'throughput': 1365.75}
[INFO|2026-02-08 23:31:14] logging.py:144 >> {'loss': 0.1475, 'learning_rate': 1.6900e-05, 'epoch': 1.46, 'throughput': 1368.17}
[INFO|2026-02-08 23:32:18] logging.py:144 >> {'loss': 0.1384, 'learning_rate': 1.7900e-05, 'epoch': 1.54, 'throughput': 1370.15}
[INFO|2026-02-08 23:33:22] logging.py:144 >> {'loss': 0.1406, 'learning_rate': 1.8900e-05, 'epoch': 1.63, 'throughput': 1371.30}
[INFO|2026-02-08 23:34:26] logging.py:144 >> {'loss': 0.1450, 'learning_rate': 1.9900e-05, 'epoch': 1.71, 'throughput': 1373.15}
[INFO|2026-02-08 23:35:30] logging.py:144 >> {'loss': 0.1431, 'learning_rate': 2.0900e-05, 'epoch': 1.80, 'throughput': 1375.58}
[INFO|2026-02-08 23:36:34] logging.py:144 >> {'loss': 0.1484, 'learning_rate': 2.1900e-05, 'epoch': 1.89, 'throughput': 1377.21}
[INFO|2026-02-08 23:37:38] logging.py:144 >> {'loss': 0.1294, 'learning_rate': 2.2900e-05, 'epoch': 1.97, 'throughput': 1378.08}
[INFO|2026-02-08 23:38:38] logging.py:144 >> {'loss': 0.1000, 'learning_rate': 2.3900e-05, 'epoch': 2.05, 'throughput': 1378.78}
[INFO|2026-02-08 23:39:42] logging.py:144 >> {'loss': 0.0867, 'learning_rate': 2.4900e-05, 'epoch': 2.14, 'throughput': 1379.78}
[INFO|2026-02-08 23:40:46] logging.py:144 >> {'loss': 0.0835, 'learning_rate': 2.5900e-05, 'epoch': 2.22, 'throughput': 1381.74}
[INFO|2026-02-08 23:41:50] logging.py:144 >> {'loss': 0.0989, 'learning_rate': 2.6900e-05, 'epoch': 2.31, 'throughput': 1382.71}
[INFO|2026-02-08 23:42:54] logging.py:144 >> {'loss': 0.1173, 'learning_rate': 2.7900e-05, 'epoch': 2.40, 'throughput': 1383.72}
[INFO|2026-02-08 23:43:59] logging.py:144 >> {'loss': 0.1075, 'learning_rate': 2.8900e-05, 'epoch': 2.48, 'throughput': 1384.55}
[INFO|2026-02-08 23:45:03] logging.py:144 >> {'loss': 0.0904, 'learning_rate': 2.9900e-05, 'epoch': 2.57, 'throughput': 1385.48}
[INFO|2026-02-08 23:46:07] logging.py:144 >> {'loss': 0.1165, 'learning_rate': 3.0900e-05, 'epoch': 2.65, 'throughput': 1386.15}
[INFO|2026-02-08 23:47:11] logging.py:144 >> {'loss': 0.0846, 'learning_rate': 3.1900e-05, 'epoch': 2.74, 'throughput': 1386.83}
[INFO|2026-02-08 23:48:16] logging.py:144 >> {'loss': 0.0945, 'learning_rate': 3.2900e-05, 'epoch': 2.83, 'throughput': 1387.38}
[INFO|2026-02-08 23:49:19] logging.py:144 >> {'loss': 0.1040, 'learning_rate': 3.3900e-05, 'epoch': 2.91, 'throughput': 1387.74}
[INFO|2026-02-08 23:50:24] logging.py:144 >> {'loss': 0.0984, 'learning_rate': 3.4900e-05, 'epoch': 3.00, 'throughput': 1388.27}
[INFO|2026-02-08 23:50:24] trainer.py:4115 >> Saving model checkpoint to saves/GLM-Z1-0414-9B-Chat/lora/glm-treinado/checkpoint-351
[INFO|2026-02-08 23:50:25] configuration_utils.py:667 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/config.json
[INFO|2026-02-08 23:50:25] configuration_utils.py:739 >> Model config Glm4Config {
  "architectures": [
    "Glm4ForCausalLM"
  ],
  "attention_bias": true,
  "attention_dropout": 0.0,
  "bos_token_id": null,
  "dtype": "bfloat16",
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 13696,
  "max_position_embeddings": 32768,
  "model_type": "glm4",
  "num_attention_heads": 32,
  "num_hidden_layers": 40,
  "num_key_value_heads": 2,
  "pad_token_id": 151329,
  "partial_rotary_factor": 0.5,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "partial_rotary_factor": 0.5,
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.0.0",
  "use_cache": true,
  "vocab_size": 151552
}

[INFO|2026-02-08 23:50:26] trainer.py:2657 >> 

Training completed. Do not forget to share your model on huggingface.co/models =)


[INFO|2026-02-08 23:50:26] trainer.py:4115 >> Saving model checkpoint to saves/GLM-Z1-0414-9B-Chat/lora/glm-treinado
[INFO|2026-02-08 23:50:27] configuration_utils.py:667 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--zai-org--GLM-Z1-9B-0414/snapshots/b221b06fefb23ca320922cf6e68ab5f2fb82de81/config.json
[INFO|2026-02-08 23:50:27] configuration_utils.py:739 >> Model config Glm4Config {
  "architectures": [
    "Glm4ForCausalLM"
  ],
  "attention_bias": true,
  "attention_dropout": 0.0,
  "bos_token_id": null,
  "dtype": "bfloat16",
  "eos_token_id": [
    151329,
    151336,
    151338
  ],
  "head_dim": 128,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 13696,
  "max_position_embeddings": 32768,
  "model_type": "glm4",
  "num_attention_heads": 32,
  "num_hidden_layers": 40,
  "num_key_value_heads": 2,
  "pad_token_id": 151329,
  "partial_rotary_factor": 0.5,
  "rms_norm_eps": 1e-05,
  "rope_parameters": {
    "partial_rotary_factor": 0.5,
    "rope_theta": 10000.0,
    "rope_type": "default"
  },
  "tie_word_embeddings": false,
  "transformers_version": "5.0.0",
  "use_cache": true,
  "vocab_size": 151552
}

[WARNING|2026-02-08 23:50:28] logging.py:149 >> No metric eval_loss to plot.
[WARNING|2026-02-08 23:50:28] logging.py:149 >> No metric eval_accuracy to plot.
[INFO|2026-02-08 23:50:28] modelcard.py:266 >> Dropping the following result as it does not have all the necessary fields:
{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}