Update README.md

README.md

@@ -15,4 +15,191 @@ tags:
 language:
 - en
 pipeline_tag: text-generation
----
+---
+
+# **Qwen-Image-2.1-PE-T2I-FP8**
+
+> **Qwen-Image-2.1-PE-T2I-FP8** is an **FP8 dynamic-quantized** build of [Qwen/Qwen-Image-2.1-PE-T2I](https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I), the text-to-image **prompt rewriting model** for [Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1). The base model is a fine-tuned **Qwen3.5-VL 9B** that turns a brief image request in any language into a detailed English prompt plus a recommended aspect ratio. This checkpoint was compressed with **[llm-compressor](https://github.com/vllm-project/llm-compressor)** using the **FP8_DYNAMIC** scheme, stored in the **compressed-tensors** format, and is intended to be served with **[vLLM](https://github.com/vllm-project/vllm)**. The **Linear** layers are quantized to **FP8** weights with dynamic per-token FP8 activations, which lowers weight memory and improves serving throughput relative to the BF16 original. The **lm_head**, **embedding layers**, **vision modules**, and **linear attention** layers are excluded and kept in their original precision. No calibration data is required, because activation scales are computed at runtime.
+
+## Quantization Details
+
+| Property | Value |
+|---|---|
+| Base model | `Qwen/Qwen-Image-2.1-PE-T2I` |
+| Architecture | Qwen3.5-VL 9B (fine-tuned) |
+| Quantization tool | `llm-compressor` |
+| Modifier | `QuantizationModifier` |
+| Scheme | `FP8_DYNAMIC` |
+| Checkpoint format | `compressed-tensors` |
+| Quantized modules | `Linear` layers |
+| Excluded modules | `lm_head`, `embed_tokens`, `visual`, `linear_attn` |
+| Calibration data | Not required |
+| Inference engine | vLLM |
+
+### Recipe
+
+```yaml
+default_stage:
+  default_modifiers:
+    QuantizationModifier:
+      targets: [Linear]
+      ignore: ['re:.*lm_head', 're:.*embed_tokens$', 're:.*visual.*', 're:.*model.visual.*',
+        're:.*linear_attn.*']
+      scheme: FP8_DYNAMIC
+      bypass_divisibility_checks: false
+      requires_calibration_data: false
+```
+
+### Reproduction
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from llmcompressor import oneshot
+
+model_id = "Qwen/Qwen-Image-2.1-PE-T2I"
+save_dir = "Qwen-Image-2.1-PE-T2I-FP8"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_id, dtype=torch.bfloat16, device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+
+# recipe.yaml is the recipe shown above
+oneshot(model=model, recipe="recipe.yaml")
+
+model.save_pretrained(save_dir, save_compressed=True)
+tokenizer.save_pretrained(save_dir)
+```
+
+## Quick Start
+
+### Installation
+
+```bash
+pip install vllm openai huggingface_hub
+```
+
+Use a recent vLLM release with support for Qwen3.5-VL and compressed-tensors FP8 checkpoints. Native FP8 compute requires a GPU with compute capability 8.9 or higher (Ada Lovelace, Hopper, Blackwell). On older GPUs vLLM falls back to weight-only FP8 kernels.
+
+### Serve with vLLM
+
+```bash
+vllm serve prithivMLmods/Qwen-Image-2.1-PE-T2I-FP8 \
+    --max-model-len 32768
+```
+
+Do not enable a reasoning parser, so the thinking block is returned in the response content and can be split from the JSON answer as shown below.
+
+### Query the Server
+
+```python
+import json
+
+import huggingface_hub
+from openai import OpenAI
+
+model_id = "prithivMLmods/Qwen-Image-2.1-PE-T2I-FP8"
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
+
+# System prompt shipped with the base model
+sys_prompt_path = huggingface_hub.hf_hub_download(
+    "Qwen/Qwen-Image-2.1-PE-T2I", "system_prompt.txt"
+)
+system_prompt = open(sys_prompt_path).read().strip()
+
+user_prompt = "一只在雨中弹吉他的柯基"
+
+response = client.chat.completions.create(
+    model=model_id,
+    messages=[
+        {"role": "system", "content": system_prompt},
+        {"role": "user", "content": user_prompt},
+    ],
+    max_tokens=16256,
+    temperature=1.0,
+    top_p=0.95,
+    seed=42,
+    extra_body={"top_k": 20},
+)
+
+gen = response.choices[0].message.content
+
+# Split thinking from the answer
+thinking, _, answer = gen.partition("</think>")
+result = json.loads(answer.strip())
+print(result)
+# {"rewritten_prompt": "<long detailed English prompt>", "wh_ratio": "16:9"}
+```
+
+### Integration with Diffusers
+
+```python
+import torch
+from diffusers import QwenImage21Pipeline
+
+WH_RATIO_TO_SIZE = {
+    "1:1": (2048, 2048), "4:3": (2400, 1792), "3:4": (1792, 2400),
+    "3:2": (2528, 1696), "2:3": (1696, 2528), "16:9": (2752, 1536),
+    "9:16": (1536, 2752),
+}
+
+# `result` from the vLLM call above
+prompt = result["rewritten_prompt"]
+width, height = WH_RATIO_TO_SIZE.get(result["wh_ratio"], (2048, 2048))
+
+pipe = QwenImage21Pipeline.from_pretrained(
+    "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16
+).to("cuda")
+
+image = pipe(
+    prompt=prompt,
+    width=width, height=height,
+    num_inference_steps=40,
+    generator=torch.Generator("cuda").manual_seed(42),
+).images[0]
+
+image.save("rewritten_t2i.png")
+```
+
+## Output Format
+
+The model emits a JSON object after a `<think>` reasoning block:
+
+```json
+{
+  "rewritten_prompt": "<long detailed English prompt describing the finished image>",
+  "wh_ratio": "16:9"
+}
+```
+
+| Field | Description |
+|---|---|
+| `rewritten_prompt` | Expanded English prompt to pass to the image generation model |
+| `wh_ratio` | Recommended aspect ratio for rendering |
+
+The aspect ratios used in the Diffusers example map to the following output sizes:
+
+| `wh_ratio` | Width x Height |
+|---|---|
+| `1:1` | 2048 x 2048 |
+| `4:3` | 2400 x 1792 |
+| `3:4` | 1792 x 2400 |
+| `3:2` | 2528 x 1696 |
+| `2:3` | 1696 x 2528 |
+| `16:9` | 2752 x 1536 |
+| `9:16` | 1536 x 2752 |
+
+## Notes
+
+- FP8 quantization introduces small numerical differences from the BF16 original, so rewritten prompts may differ slightly in wording. Validate output quality on your own generation workloads before replacing the base model.
+- Always parse the answer with a JSON loader and handle malformed output, since sampling at `temperature=1.0` can occasionally produce invalid JSON.
+- Input requests can be written in any language, and the rewritten prompt is always English.
+
+## License
+
+This model inherits the license of the base model and is distributed under the [Qwen Research License Agreement](./LICENSE).
+
+## Acknowledgements
+
+Base model by the Qwen team: [Qwen/Qwen-Image-2.1-PE-T2I](https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I). Quantization with [llm-compressor](https://github.com/vllm-project/llm-compressor). Serving with [vLLM](https://github.com/vllm-project/vllm). See the [Qwen-Image-2.1 GitHub repo](https://github.com/QwenLM/Qwen-Image-2.1) and [blog](https://qwen.ai/blog?id=qwen-image-2.1) for details on the full pipeline.