naykun commited on
Commit
79e6e58
·
verified ·
1 Parent(s): d0bf173

Update README with header, introduction, and usage

Browse files
Files changed (1) hide show
  1. README.md +139 -0
README.md ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: qwen-research
4
+ license_link: LICENSE
5
+ tags:
6
+ - qwen
7
+ - prompt-rewriting
8
+ - text-to-image
9
+ ---
10
+
11
+ <p align="center">
12
+ <img src="https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/image2.1/logo.png" width="400"/>
13
+ </p>
14
+ <p align="center">
15
+ 🤖 <a href="https://modelscope.cn/models/Qwen/Qwen-Image-2.1">ModelScope</a>&nbsp;&nbsp;|
16
+ &nbsp;&nbsp;🤗 <a href="https://huggingface.co/Qwen/Qwen-Image-2.1">HuggingFace</a>&nbsp;&nbsp;|
17
+ &nbsp;&nbsp;📑 <a href="https://qwen.ai/blog?id=qwen-image-2.1">Blog</a>&nbsp;&nbsp;|
18
+ &nbsp;&nbsp;🖥️ <a href="https://huggingface.co/spaces/Qwen/Qwen-Image-2.1">Demo</a>&nbsp;&nbsp;|
19
+ &nbsp;&nbsp;🫨 <a href="https://discord.gg/CV4E9rpNSD">Discord</a>
20
+ </p>
21
+
22
+ ## Introduction
23
+
24
+ We are excited to open-source **Qwen-Image-2.1**, a unified text-to-image generation and image editing model in the Qwen family. With just **7B parameters in its visual generation component** (32 Single-Stream DiT layers), Qwen-Image-2.1 balances generation quality, inference efficiency, and versatility.
25
+
26
+ Four key improvements define this release:
27
+
28
+ - **Compact and Efficient** — A lightweight architecture with mixed-granularity attention and prefix KV cache reuse delivers strong image quality at low computational cost.
29
+ - **Native Transparency, Unified Creation and Editing** — Generate regular or transparent (RGBA) images from text, edit transparent layers, and extract subjects from photographs—all in one model.
30
+ - **Versatile Editing** — Support up to **10 reference images**, specify local edits via circles, painted annotations, or separate masks, and preserve identity for people and products.
31
+ - **Realistic Textures and Refined Aesthetics** — Improved typography, portrait lighting, and fine details for more visually compelling results.
32
+
33
+ <p align="center">
34
+ <img src="https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen-Image/image2.1/images/example-01.png" width="100%"/>
35
+ </p>
36
+
37
+ # Qwen-Image-2.1-PE-T2I
38
+
39
+ Text-to-image **prompt rewriting model** for [Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1). A fine-tuned Qwen3.5-VL 9B that turns a brief image request in any language into a detailed English prompt plus a recommended aspect ratio.
40
+
41
+ For more details, see the [GitHub repo](https://github.com/QwenLM/Qwen-Image-2.1) and [Blog](https://qwen.ai/blog?id=qwen-image-2.1).
42
+
43
+ ## Quick Start
44
+
45
+ ### Installation
46
+
47
+ ```bash
48
+ pip install transformers>=5.4.0 torch>=2.4.0 accelerate pillow
49
+ ```
50
+
51
+ ### Usage with Transformers
52
+
53
+ ```python
54
+ import json
55
+ import torch
56
+ from transformers import AutoModelForCausalLM, AutoTokenizer
57
+
58
+ model_id = "Qwen/Qwen-Image-2.1-PE-T2I"
59
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
60
+ model = AutoModelForCausalLM.from_pretrained(
61
+ model_id, dtype=torch.bfloat16, device_map="auto"
62
+ ).eval()
63
+
64
+ # Load the system prompt shipped with the model
65
+ import huggingface_hub
66
+ sys_prompt_path = huggingface_hub.hf_hub_download(model_id, "system_prompt.txt")
67
+ system_prompt = open(sys_prompt_path).read().strip()
68
+
69
+ user_prompt = "一只在雨中弹吉他的柯基"
70
+
71
+ text = tokenizer.apply_chat_template(
72
+ [{"role": "system", "content": system_prompt},
73
+ {"role": "user", "content": user_prompt}],
74
+ tokenize=False, add_generation_prompt=True, enable_thinking=True,
75
+ )
76
+ inputs = tokenizer(text, return_tensors="pt").to(model.device)
77
+
78
+ with torch.no_grad():
79
+ out = model.generate(
80
+ **inputs, max_new_tokens=16256,
81
+ do_sample=True, temperature=1.0, top_p=0.95, top_k=20,
82
+ )
83
+ gen = tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
84
+
85
+ # Split thinking from the answer
86
+ thinking, _, answer = gen.partition("</think>")
87
+ result = json.loads(answer.strip())
88
+ print(result)
89
+ # {"rewritten_prompt": "<long detailed English prompt>", "wh_ratio": "16:9"}
90
+ ```
91
+
92
+ ### Integration with Diffusers
93
+
94
+ ```python
95
+ import json
96
+ import torch
97
+ from diffusers import QwenImage21Pipeline
98
+
99
+ WH_RATIO_TO_SIZE = {
100
+ "1:1": (2048, 2048), "4:3": (2400, 1792), "3:4": (1792, 2400),
101
+ "3:2": (2528, 1696), "2:3": (1696, 2528), "16:9": (2752, 1536),
102
+ "9:16": (1536, 2752),
103
+ }
104
+
105
+ # Assuming `result` from above
106
+ prompt = result["rewritten_prompt"]
107
+ width, height = WH_RATIO_TO_SIZE.get(result["wh_ratio"], (2048, 2048))
108
+
109
+ pipe = QwenImage21Pipeline.from_pretrained(
110
+ "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16
111
+ ).to("cuda")
112
+
113
+ image = pipe(
114
+ prompt=prompt,
115
+ width=width, height=height,
116
+ num_inference_steps=40,
117
+ generator=torch.Generator("cuda").manual_seed(42),
118
+ ).images[0]
119
+
120
+ image.save("rewritten_t2i.png")
121
+ ```
122
+
123
+ ### Output Format
124
+
125
+ The model outputs a JSON object after a `<think>` reasoning block:
126
+
127
+ ```json
128
+ {
129
+ "rewritten_prompt": "<long detailed English prompt describing the finished image>",
130
+ "wh_ratio": "16:9"
131
+ }
132
+ ```
133
+
134
+ - `rewritten_prompt` — the expanded prompt to pass to the image generation model
135
+ - `wh_ratio` — the recommended aspect ratio for rendering
136
+
137
+ ## License
138
+
139
+ This model is licensed under the [Qwen Research License Agreement](./LICENSE).