README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-4B-Thinking-2507
tags:
- quantized
- rtn
- 4-bit
- thinking
- reasoning
---

# Qwen__Qwen3-4B-Thinking-2507_RTN_w4g128

This is a 4-bit RTN (Round-To-Nearest) quantized version of [Qwen/Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507).

## Quantization Details

- **Method**: RTN (Round-To-Nearest)
- **Bits**: 4-bit
- **Group Size**: 128
- **Base Model**: [Qwen/Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507)

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "quantpa/Qwen__Qwen3-4B-Thinking-2507_RTN_w4g128"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Use the model for inference
```

## Model Details

- **Quantization**: RTN 4-bit
- **Original Model**: Qwen/Qwen3-4B-Thinking-2507
- **Quantized by**: quantpa