metadata
license: apache-2.0
base_model: zai-org/GLM-Z1-9B-0414
tags:
- quantized
- rtn
- 4-bit
- thinking
- reasoning
zai-org__GLM-Z1-9B-0414_RTN_w4g128
This is a 4-bit RTN (Round-To-Nearest) quantized version of zai-org/GLM-Z1-9B-0414.
Quantization Details
- Method: RTN (Round-To-Nearest)
- Bits: 4-bit
- Group Size: 128
- Base Model: zai-org/GLM-Z1-9B-0414
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "quantpa/zai-org__GLM-Z1-9B-0414_RTN_w4g128"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
# Use the model for inference
Model Details
- Quantization: RTN 4-bit
- Original Model: zai-org/GLM-Z1-9B-0414
- Quantized by: quantpa