---
base_model:
- Qwen/Qwen3-0.6B
pipeline_tag: text-generation
library_name: transformers
tags:
- FP8
- OCP
- AMD
- ROCM
- Quark
- vllm
---

# Qwen3-0.6B-FP8-KV

> **Lightweight OCP FP8_e4m3 quant of Qwen3-0.6B** with end-to-end KV-cache FP8 support, built with AMD Quark for ROCm.


## Introduction  
Qwen3-0.6B-FP8-KV is an OCP-standard FP8_e4m3 quantization of [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B), 
produced with **AMD Quark**. 


## Quantization Strategy  
- **Quantizer**: AMD Quark v0.9+  
- **Numeric Format**: OCP FP8_e4m3, symmetric per-tensor  
- **Scope**: All `Linear` layers (excl. `lm_head`), activations **and the KV cache**  
- **Block Size**: 128 (OCP-aligned)  
- **Calibration**: 128 Pile samples  
- **Metadata**: scales & block info in JSON; weights in SafeTensors  


## Performance Snapshot  

| Metric                 | FP16 Baseline | FP8_e4m3 Quantized |
|------------------------|--------------:|-------------------:|
| Wikitext2 Perplexity   |        ~22.1  |             ~25.8  |
| Memory Footprint       |         1.0×  |             0.50×  |
| Inference Throughput   |         1.0×  |             1.3×   |

## Evaluation
We measured perplexity on WikiText2:
- FP16 (Qwen3-0.6B)             →  22.1 PPL  
- FP8_e4m3 (this model)         →  25.8 PPL

## License
This model inherits the Qwen3-0.6B license.