README.md

---
license: other
license_name: qwen-research
license_link: https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I/blob/main/LICENSE
base_model:
- Qwen/Qwen-Image-2.1-PE-T2I
library_name: transformers
tags:
- text-generation-inference
- vllm
- 8-bit
- qwen
- prompt-rewriting
- image-editing
language:
- en
pipeline_tag: text-generation
---

# **Qwen-Image-2.1-PE-T2I-FP8**

> **Qwen-Image-2.1-PE-T2I-FP8** is an **FP8 dynamic-quantized** build of [Qwen/Qwen-Image-2.1-PE-T2I](https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I), the text-to-image **prompt rewriting model** for [Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1). The base model is a fine-tuned **Qwen3.5-VL 9B** that turns a brief image request in any language into a detailed English prompt plus a recommended aspect ratio. This checkpoint was compressed with **[llm-compressor](https://github.com/vllm-project/llm-compressor)** using the **FP8_DYNAMIC** scheme, stored in the **compressed-tensors** format, and is intended to be served with **[vLLM](https://github.com/vllm-project/vllm)**. The **Linear** layers are quantized to **FP8** weights with dynamic per-token FP8 activations, which lowers weight memory and improves serving throughput relative to the BF16 original. The **lm_head**, **embedding layers**, **vision modules**, and **linear attention** layers are excluded and kept in their original precision. No calibration data is required, because activation scales are computed at runtime.

> [!NOTE]
System Prompt — https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I/blob/main/system_prompt.txt

## Quantization Details

| Property | Value |
|---|---|
| Base model | `Qwen/Qwen-Image-2.1-PE-T2I` |
| Architecture | Qwen3.5-VL 9B (fine-tuned) |
| Quantization tool | `llm-compressor` |
| Modifier | `QuantizationModifier` |
| Scheme | `FP8_DYNAMIC` |
| Checkpoint format | `compressed-tensors` |
| Quantized modules | `Linear` layers |
| Excluded modules | `lm_head`, `embed_tokens`, `visual`, `linear_attn` |
| Calibration data | Not required |
| Inference engine | vLLM |

### Recipe

```yaml
default_stage:
  default_modifiers:
    QuantizationModifier:
      targets: [Linear]
      ignore: ['re:.*lm_head', 're:.*embed_tokens$', 're:.*visual.*', 're:.*model.visual.*',
        're:.*linear_attn.*']
      scheme: FP8_DYNAMIC
      bypass_divisibility_checks: false
      requires_calibration_data: false
```

### Reproduction

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from llmcompressor import oneshot

model_id = "Qwen/Qwen-Image-2.1-PE-T2I"
save_dir = "Qwen-Image-2.1-PE-T2I-FP8"

model = AutoModelForCausalLM.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# recipe.yaml is the recipe shown above
oneshot(model=model, recipe="recipe.yaml")

model.save_pretrained(save_dir, save_compressed=True)
tokenizer.save_pretrained(save_dir)
```

## Quick Start

### Installation

```bash
pip install vllm openai huggingface_hub
```

Use a recent vLLM release with support for Qwen3.5-VL and compressed-tensors FP8 checkpoints. Native FP8 compute requires a GPU with compute capability 8.9 or higher (Ada Lovelace, Hopper, Blackwell). On older GPUs vLLM falls back to weight-only FP8 kernels.

### Serve with vLLM

```bash
vllm serve prithivMLmods/Qwen-Image-2.1-PE-T2I-FP8 \
    --max-model-len 32768
```

Do not enable a reasoning parser, so the thinking block is returned in the response content and can be split from the JSON answer as shown below.

### Query the Server

```python
import json

import huggingface_hub
from openai import OpenAI

model_id = "prithivMLmods/Qwen-Image-2.1-PE-T2I-FP8"
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

# System prompt shipped with the base model
sys_prompt_path = huggingface_hub.hf_hub_download(
    "Qwen/Qwen-Image-2.1-PE-T2I", "system_prompt.txt"
)
system_prompt = open(sys_prompt_path).read().strip()

user_prompt = "一只在雨中弹吉他的柯基"

response = client.chat.completions.create(
    model=model_id,
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_prompt},
    ],
    max_tokens=16256,
    temperature=1.0,
    top_p=0.95,
    seed=42,
    extra_body={"top_k": 20},
)

gen = response.choices[0].message.content

# Split thinking from the answer
thinking, _, answer = gen.partition("</think>")
result = json.loads(answer.strip())
print(result)
# {"rewritten_prompt": "<long detailed English prompt>", "wh_ratio": "16:9"}
```

### Integration with Diffusers

```python
import torch
from diffusers import QwenImage21Pipeline

WH_RATIO_TO_SIZE = {
    "1:1": (2048, 2048), "4:3": (2400, 1792), "3:4": (1792, 2400),
    "3:2": (2528, 1696), "2:3": (1696, 2528), "16:9": (2752, 1536),
    "9:16": (1536, 2752),
}

# `result` from the vLLM call above
prompt = result["rewritten_prompt"]
width, height = WH_RATIO_TO_SIZE.get(result["wh_ratio"], (2048, 2048))

pipe = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16
).to("cuda")

image = pipe(
    prompt=prompt,
    width=width, height=height,
    num_inference_steps=40,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]

image.save("rewritten_t2i.png")
```

## Output Format

The model emits a JSON object after a `<think>` reasoning block:

```json
{
  "rewritten_prompt": "<long detailed English prompt describing the finished image>",
  "wh_ratio": "16:9"
}
```

| Field | Description |
|---|---|
| `rewritten_prompt` | Expanded English prompt to pass to the image generation model |
| `wh_ratio` | Recommended aspect ratio for rendering |

The aspect ratios used in the Diffusers example map to the following output sizes:

| `wh_ratio` | Width x Height |
|---|---|
| `1:1` | 2048 x 2048 |
| `4:3` | 2400 x 1792 |
| `3:4` | 1792 x 2400 |
| `3:2` | 2528 x 1696 |
| `2:3` | 1696 x 2528 |
| `16:9` | 2752 x 1536 |
| `9:16` | 1536 x 2752 |

## Notes

- FP8 quantization introduces small numerical differences from the BF16 original, so rewritten prompts may differ slightly in wording. Validate output quality on your own generation workloads before replacing the base model.
- Always parse the answer with a JSON loader and handle malformed output, since sampling at `temperature=1.0` can occasionally produce invalid JSON.
- Input requests can be written in any language, and the rewritten prompt is always English.

## License

This model inherits the license of the base model and is distributed under the [Qwen Research License Agreement](https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I/blob/main/LICENSE).

## Acknowledgements

Base model by the Qwen team: [Qwen/Qwen-Image-2.1-PE-T2I](https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I). Quantization with [llm-compressor](https://github.com/vllm-project/llm-compressor). Serving with [vLLM](https://github.com/vllm-project/vllm). See the [Qwen-Image-2.1 GitHub repo](https://github.com/QwenLM/Qwen-Image-2.1) and [blog](https://qwen.ai/blog?id=qwen-image-2.1) for details on the full pipeline.