Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+processor/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,171 @@
 ---
 license: apache-2.0
+base_model: Qwen/Qwen-Image-2.1
+tags:
+- qwen
+- qwen3-vl
+- text-encoder
+- vision-encoder
+- heretic
+- abliteration
+- prompt-adherence
+- diffusers
 ---
+
+# Qwen3-VL-8B Heretic Text & Vision Encoder (Prompt Adherence & Geometric Alignment Edition) 🌺✨
+
+This repository provides an optimized, abliterated checkpoint of the **Qwen3-VL-8B** text and vision encoder from **[Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1)**, processed with **Norm-Preserving Biprojected Abliteration**.
+
+The primary purpose of this model is **maximum instruction following and prompt adherence**: it banishes geometric representation deflection ("internal blush" / hesitation vectors) that otherwise causes safety-tuned VLMs to corrupt diffusion conditioning on dynamic poses, human figures, athletic wear, and complex scenes.
+
+---
+
+## 🔬 The Core Problem: Why VLM Safety Alignment Degrades Diffusion Conditioning
+
+In text-generation tasks, safety alignment mechanisms steer models to emit refusal text (e.g. *"I cannot fulfill this request..."*). However, modern multimodal diffusion architectures like **Qwen-Image-2.1** do **not** generate text tokens:
+
+$$\text{DiT Conditioning} \longleftarrow \mathbf{h}_L = \text{TextEncoder}(\text{tokens})[-1]$$
+
+The diffusion transformer taps the raw pre-RMSNorm residual hidden states $\mathbf{h}_L$ directly from the text encoder to drive cross-attention.
+
+### The "Internal Blush" / Hesitation Deflection Phenomenon
+
+When prompts describe human subjects, dynamic physical actions, athletic attire (e.g. swimwear, volleyball, gymnastics), or expressive emotions, safety-tuning vectors inside the language model activate even on completely benign, non-refusal prompts. 
+
+Because the model cannot output a refusal string, these alignment vectors manifest as a **geometric rotation of the latent representation**:
+
+$$\mathbf{h}_{\text{sensitive}} = \mathbf{h}_{\text{clean}} + \mathbf{v}_{\text{refusal}}$$
+
+This hidden deflection rotates the conditioning signal by **over 60% relative norm** away from the prompt's intended semantic visual trajectory!
+
+### Visual Consequences in Image Generation
+
+When cross-attention layers in the DiT receive a representation deflected into the refusal/modesty subspace, the model displays hesitation artifacts:
+1. **Modesty Hallucinations & Clothing Confusion**: Spontaneous addition of mismatched cloth, awkward white ruffles, or extra fabric covering swimwear or sportswear.
+2. **Anatomical Occlusion**: The model avoids rendering human limbs or athletic poses, awkwardly hiding arms behind character backs or contorting torsos.
+3. **Subject & Prop Merging**: Equipment or background elements get fused into characters (e.g. sports balls bizarrely merged onto heads as hair ornaments).
+4. **Action Damping**: Dynamic verbs (*"jumping to spike the ball"*) are subdued into passive, static standing postures.
+
+---
+
+## 📊 Quantitative Measurement of Representation Deflection
+
+Using contrastive prompt pairs across benign and sensitive subjects, we measured the layer-by-layer cosine similarity and relative deflection norm across all 37 positions (input embeddings + 36 decoder layers) of Qwen3-VL-8B:
+
+| Layer Index | Position | Cosine Similarity ($\cos \theta$) | Relative Deflection ($\|\Delta \mathbf{h}\| / \|\mathbf{h}\|$) | Deflection Norm $\|\Delta \mathbf{h}\|$ |
+| :---: | :---: | :---: | :---: | :---: |
+| **0** | Input Embeddings | **1.0000** | **0.00%** | 0.00 |
+| **8** | Early Transformer | **0.9991** | **3.82%** | 18.24 |
+| **16** | Mid-Low (Deflection Onset) | **0.9943** | **10.64%** | 52.88 |
+| **20** | Mid-High Divergence | **0.9780** | **21.05%** | 114.73 |
+| **24** | Refusal Vector Surge | **0.9414** | **34.25%** | 192.40 |
+| **28** | Acceleration Peak | **0.8842** | **47.19%** | 275.31 |
+| **32** | Late Transformer | **0.8350** | **56.12%** | 331.05 |
+| **36** | Final Conditioning Layer | **0.8110** | **60.30%** | **357.94** |
+
+Between Layer 20 and Layer 36, representation deflection accelerates rapidly, culminating in a **60.3% vector distortion**. By surgically neutralizing this direction, the text encoder reflects the exact intended prompt semantics.
+
+---
+
+## 🛠️ Methodology: Norm-Preserving Biprojected Abliteration
+
+To eliminate hesitation deflection without degrading general language comprehension, we applied **Norm-Preserving Biprojected Abliteration** (`create_heretic_text_encoder.py`):
+
+1. **Refusal Subspace Extraction**: Difference-of-means vectors were extracted across contrastive prompt sets:
+   $$\mathbf{r}_l = \boldsymbol{\mu}_{\text{sensitive}}^{(l)} - \boldsymbol{\mu}_{\text{benign}}^{(l)}$$
+2. **Benign Subspace Orthogonalization**: The general semantic direction was stripped from the refusal vector:
+   $$\mathbf{v}_l = \mathbf{r}_l - \text{proj}_{\mathbf{u}_{\text{benign}}}(\mathbf{r}_l)$$
+3. **Norm-Preserving Rank-1 Projection**: Across 54 linear projection matrices (`self_attn.o_proj` and `mlp.down_proj` in layers 9–35, centered at layer 26 with Gaussian falloff $\lambda \in [0.10, 1.00]$):
+   $$W_{\text{norm}} = \text{normalize}(W, p=2, \text{dim}=1)$$
+   $$W' = \text{normalize}\Big(W_{\text{norm}} - \lambda \mathbf{v}_l (\mathbf{v}_l^T W_{\text{norm}})\Big) \cdot \|W\|_{\text{row}}$$
+
+Because exact row norms ($\|W\|_{\text{row}}$) are strictly preserved, the network's overall activation scales and general reasoning capabilities remain completely intact.
+
+---
+
+## 🤝 Pairing with Quantized DiT (`nunchaku-qwen-image-2.1`)
+
+This text encoder is specifically engineered to be paired with **[`nunchaku-qwen-image-2.1`](https://huggingface.co/models/nunchaku-qwen-image-2.1)** for consumer GPU setups:
+
+* **Resident DiT + Streamed Text Encoder**:
+  * DiT (`best_quality_fp4.safetensors`): **4.08 GB resident VRAM**.
+  * VAE (`AutoencoderKLQwenImage21`): **0.64 GB resident VRAM**.
+  * Qwen3-VL-8B ViT Vision Encoder: **1.07 GB resident VRAM**.
+  * Qwen3-VL-8B Language Model: Streamed layer-by-layer through a static **368 MB GPU buffer** over PCIe at ~28.7 GB/s via `stream_encoder.py`.
+* **Total VRAM Footprint**: **~6.17 GB active VRAM**, leaving **~9.5 GB free headroom** on a single 16 GB GPU (such as RTX 5060 Ti or RTX 4080)!
+* **Inference Speed**: Multimodal prompt encoding completes in **1.06s** (saving 16s vs CPU), and 25-step image generation runs in **~20s**.
+
+---
+
+## 🚀 Quickstart Usage
+
+### 1. Installation
+
+```bash
+pip install diffusers transformers accelerate torch sentencepiece
+```
+
+### 2. Loading with Diffusers
+
+```python
+import torch
+from diffusers import QwenImage21Pipeline
+from transformers import Qwen3VLForConditionalGeneration, Qwen3VLProcessor
+
+# 1. Load Heretic text encoder and processor
+text_encoder = Qwen3VLForConditionalGeneration.from_pretrained(
+    "models/Qwen21_Text_Encoder_Heretic",
+    torch_dtype=torch.bfloat16,
+    low_cpu_mem_usage=True,
+)
+processor = Qwen3VLProcessor.from_pretrained("models/Qwen21_Text_Encoder_Heretic")
+
+# 2. Assemble into pipeline
+pipe = QwenImage21Pipeline.from_pretrained(
+    "Qwen/Qwen-Image-2.1",
+    text_encoder=text_encoder,
+    processor=processor,
+    torch_dtype=torch.bfloat16,
+)
+pipe.enable_sequential_cpu_offload(gpu_id=0)
+
+# 3. Generate with precise prompt adherence
+image = pipe(
+    prompt="Two cute anime girls in colorful bikinis playing beach volleyball on a sunny tropical beach, dynamic action pose, jumping to spike the ball, sharp focus",
+    height=1024,
+    width=1024,
+    num_inference_steps=25,
+    true_cfg_scale=1.0,
+).images[0]
+
+image.save("beach_volleyball.png")
+```
+
+### 3. High-Throughput Server Usage
+
+Run the bundled ImageEditServer with NVFP4 DiT and Heretic text encoder:
+
+```bash
+# Start server on port 4500 (uses Heretic text encoder by default)
+./extras/imagegen_qwen21_nvfp4.sh 4500
+```
+
+---
+
+## 📦 Packaged Sources (`extras/`)
+
+* `create_heretic_text_encoder.py`: Complete script used to measure refusal vectors and perform norm-preserving biprojected abliteration.
+* `stream_encoder.py`: Zero-quality-loss layerwise weight streaming engine for Qwen3-VL-8B.
+* `test_heretic_beach_volleyball.py`: Empirical verification script comparing stock vs Heretic encoders.
+* `QwenImage21NVFP4Backend.py`: Diffusers + Nunchaku backend supporting custom text encoder overrides.
+* `ImageEditServer.py` & `imagegen_qwen21_nvfp4.sh`: Resident image generation server.
+
+---
+
+## 📜 Citation & Credits
+
+* **Qwen-Image-2.1 & Qwen3-VL**: Qwen Team, Alibaba Cloud.
+* **Abliteration Principles**: Arditi et al. (*Refusal in Language Models Is Mediated by a Single Direction*).
+* **Heretic LLM**: Heretic project (*Directional Abliteration Toolkit*).
+* **Abliteration & Diffusion Conditioning Optimization**: Oleg K. / Nikola Seeker Project.
+

added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

chat_template.jinja

@@ -0,0 +1,120 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' }}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- for message in messages %}
+    {%- if message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content_item in message.content %}
+                {%- if 'text' in content_item %}
+                    {{- content_item.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and message.content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,65 @@
+{
+  "architectures": [
+    "Qwen3VLForConditionalGeneration"
+  ],
+  "dtype": "bfloat16",
+  "image_token_id": 151655,
+  "model_type": "qwen3_vl",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 151643,
+    "dtype": "bfloat16",
+    "eos_token_id": 151645,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 4096,
+    "initializer_range": 0.02,
+    "intermediate_size": 12288,
+    "max_position_embeddings": 262144,
+    "model_type": "qwen3_vl_text",
+    "num_attention_heads": 32,
+    "num_hidden_layers": 36,
+    "num_key_value_heads": 8,
+    "pad_token_id": null,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        24,
+        20,
+        20
+      ],
+      "rope_theta": 5000000,
+      "rope_type": "default"
+    },
+    "use_cache": true,
+    "vocab_size": 151936
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.3.0.dev0",
+  "video_token_id": 151656,
+  "vision_config": {
+    "deepstack_visual_indexes": [
+      8,
+      16,
+      24
+    ],
+    "depth": 27,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4304,
+    "model_type": "qwen3_vl",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 4096,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 151653,
+  "vision_start_token_id": 151652
+}

extras/ImageEditServer.py

@@ -0,0 +1,703 @@
+import argparse
+import base64
+import io
+import time
+import torch
+import uvicorn
+import gc
+import asyncio
+import traceback
+from typing import List, Optional, Union
+from contextlib import asynccontextmanager
+from fastapi import FastAPI, HTTPException, UploadFile, File, Form
+from pydantic import BaseModel
+from PIL import Image, ImageOps
+
+# Argument parsing
+parser = argparse.ArgumentParser(description="Flux Image Edit Server with Nunchaku")
+parser.add_argument("--host", type=str, default="0.0.0.0", help="Host to bind to")
+parser.add_argument("--port", type=int, default=8000, help="Port to bind to")
+parser.add_argument("--model", type=str, default="black-forest-labs/FLUX.1-Kontext-dev", help="Path or Repo ID of the base model")
+parser.add_argument("--optimized-model", type=str, default=None, help="Path to the optimized Nunchaku model safetensors file")
+parser.add_argument("--optimized-edit-model", type=str, default=None, help="Path to the optimized Nunchaku model safetensors file for editing (optional)")
+parser.add_argument("--backend", type=str, default="kontext", choices=["kontext", "flux2", "flux2-klein", "flux2_klein", "qwen", "qwen21", "qwen-2.1", "qwen21-nvfp4", "qwen21_nvfp4", "glm", "zimage"], help="Backend to use: 'kontext', 'flux2', 'flux2-klein', 'qwen', 'qwen21', 'qwen21-nvfp4', 'glm', or 'zimage'")
+parser.add_argument("--steps", type=int, default=28, help="Default number of inference steps")
+parser.add_argument("--guidance-scale", type=float, default=3.5, help="Default guidance scale")
+parser.add_argument("--qwenimage", action="store_true", help="Use QwenImageBackend (T2I only) instead of full Qwen edit backend")
+parser.add_argument("--uma", action="store_true", help="Enable Unified Memory Architecture mode (load all to GPU, disable offload)")
+parser.add_argument("--no-layerwise-offload", action="store_true", help="Keep transformer resident in VRAM without layer-by-layer offloading")
+parser.add_argument(
+    "--text-encoder",
+    type=str,
+    default=None,
+    help="Path or Repo ID of custom text encoder (e.g. models/Qwen_Text_Encoder_Heretic)",
+)
+parser.add_argument(
+    "--nvfp4-text-encoder",
+    type=str,
+    default=None,
+    help=(
+        "Path to an NVFP4-pack-quantized HuggingFace text encoder "
+        "swaps in vLLM's W4A4 NVFP4 CUTLASS GEMM for ~4x text-encoder VRAM savings."
+    ),
+)
+args = parser.parse_args()
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    # Startup logic
+    load_model()
+    yield
+    # Shutdown logic (if any) could go here
+
+app = FastAPI(lifespan=lifespan)
+
+# Global components
+IMAGE_DIMENSION_ALIGNMENT = 32
+# Cap for a reference image fed with input_fit=native: the pipeline resizes each condition image
+# at its own aspect ratio, so this only bounds token count, never framing.
+MAX_REFERENCE_PIXELS = 1024 * 1024
+pipeline = None
+edit_pipeline = None
+request_lock = asyncio.Lock()
+is_sleeping_flag = False
+sleep_requested = False
+
+def set_zero_cond_t(transformer, val: bool):
+    """Recursively synchronize zero_cond_t across transformer and all inner blocks."""
+    if transformer is None:
+        return
+    if hasattr(transformer, "zero_cond_t"):
+        transformer.zero_cond_t = val
+    if hasattr(transformer, "transformer_blocks"):
+        for block in transformer.transformer_blocks:
+            if hasattr(block, "zero_cond_t"):
+                block.zero_cond_t = val
+
+def load_model():
+    global pipeline, edit_pipeline
+    
+    try:
+        if args.backend == "kontext":
+            import KontextBackend
+            print(f"Initializing KontextBackend...")
+            backend = KontextBackend.KontextBackend(args.model, args.optimized_model)
+            pipeline, edit_pipeline = backend.load()
+        elif args.backend == "flux2":
+            import Flux2Backend
+            print(f"Initializing Flux2Backend...")
+            backend = Flux2Backend.Flux2Backend(args.model)
+            pipeline, edit_pipeline = backend.load()
+        elif args.backend == "glm":
+            import GlmBackend
+            print(f"Initializing GlmBackend...")
+            # Use provided model or default to the one in the snippet if args.model is generic
+            # The user might pass the specific GLM model via --model, or we default in GlmBackend.
+            # Let's pass args.model if it's not the default flux one, otherwise let GlmBackend use its default.
+            model_to_use = args.model if args.model != "black-forest-labs/FLUX.1-Kontext-dev" else "Disty0/GLM-Image-SDNQ-4bit-dynamic"
+            backend = GlmBackend.GlmBackend(model_to_use)
+            pipeline, edit_pipeline = backend.load()
+        elif args.backend in ["qwen21", "qwen-2.1", "qwen21-nvfp4", "qwen21_nvfp4"]:
+            if args.optimized_model or "nvfp4" in args.backend:
+                import QwenImage21NVFP4Backend
+                print(f"Initializing QwenImage21NVFP4Backend (resident NVFP4)...")
+                backend = QwenImage21NVFP4Backend.QwenImage21NVFP4Backend(
+                    model_id=args.model,
+                    optimized_model_path=args.optimized_model or "/home/olegk/Nikola/models/nunchaku-qwen-image-2.1/best_quality_fp4.safetensors",
+                    text_encoder_path=args.text_encoder,
+                )
+                pipeline, edit_pipeline = backend.load()
+            else:
+                import QwenImage21Backend
+                print(f"Initializing QwenImage21Backend (unquantized baseline)...")
+                backend = QwenImage21Backend.QwenImage21Backend(
+                    args.model,
+                    text_encoder_path=args.text_encoder,
+                )
+                pipeline, edit_pipeline = backend.load()
+        elif args.backend.startswith("qwen"):
+            if args.qwenimage:
+                import QwenImageBackend
+                print(f"Initializing QwenImageBackend (T2I only)...")
+                backend = QwenImageBackend.QwenImageBackend(args.model, args.optimized_model)
+                pipeline, edit_pipeline = backend.load()
+            else:
+                import QwenBackend
+                print(f"Initializing QwenBackend...")
+                backend = QwenBackend.QwenBackend(
+                    args.model,
+                    args.optimized_model,
+                    optimized_edit_model_path=args.optimized_edit_model,
+                    uma=args.uma,
+                    no_layerwise_offload=args.no_layerwise_offload,
+                    text_encoder_path=args.text_encoder,
+                )
+                pipeline, edit_pipeline = backend.load()
+        elif args.backend in ["flux2-klein", "flux2_klein"]:
+            import Flux2KleinBackend
+            print(f"Initializing Flux2KleinBackend...")
+            backend = Flux2KleinBackend.Flux2KleinBackend(
+                args.model,
+                args.optimized_model,
+                nvfp4_text_encoder_path=args.nvfp4_text_encoder,
+            )
+            pipeline, edit_pipeline = backend.load()
+        elif args.backend == "zimage":
+            import ZImageTurboBackend
+            print(f"Initializing ZImageTurboBackend...")
+            backend = ZImageTurboBackend.ZImageTurboBackend(
+                args.model,
+                args.optimized_model,
+                uma=args.uma,
+                nvfp4_text_encoder_path=args.nvfp4_text_encoder,
+            )
+            pipeline, edit_pipeline = backend.load()
+        else:
+            raise ValueError(f"Unknown backend: {args.backend}")
+            
+    except Exception as e:
+        print(f"Oh no! The model refused to wake up: {e}")
+        raise e
+    
+    # Enable progress bar for diffusers
+    import diffusers.utils.logging
+    diffusers.utils.logging.enable_progress_bar()
+    diffusers.utils.logging.set_verbosity_info()
+    
+    # Instrument timing tracker for stage-by-stage profiling
+    instrument_pipeline(pipeline)
+    instrument_pipeline(edit_pipeline)
+    
+    print("Model loaded successfully! Ready for editing quests!")
+
+class PipelineTimingTracker:
+    def __init__(self):
+        self.reset()
+
+    def reset(self):
+        self.text_encoder_time = 0.0
+        self.transformer_times = []
+        self.vae_time = 0.0
+        self._te_start = None
+        self._tr_start = None
+        self._vae_start = None
+
+    def on_te_start(self):
+        if torch.cuda.is_available():
+            torch.cuda.synchronize()
+        self._te_start = time.perf_counter()
+
+    def on_te_end(self):
+        if torch.cuda.is_available():
+            torch.cuda.synchronize()
+        if self._te_start is not None:
+            self.text_encoder_time += time.perf_counter() - self._te_start
+            self._te_start = None
+
+    def on_tr_start(self):
+        if torch.cuda.is_available():
+            torch.cuda.synchronize()
+        self._tr_start = time.perf_counter()
+
+    def on_tr_end(self):
+        if torch.cuda.is_available():
+            torch.cuda.synchronize()
+        if self._tr_start is not None:
+            self.transformer_times.append(time.perf_counter() - self._tr_start)
+            self._tr_start = None
+
+    def on_vae_start(self):
+        if torch.cuda.is_available():
+            torch.cuda.synchronize()
+        self._vae_start = time.perf_counter()
+
+    def on_vae_end(self):
+        if torch.cuda.is_available():
+            torch.cuda.synchronize()
+        if self._vae_start is not None:
+            self.vae_time += time.perf_counter() - self._vae_start
+            self._vae_start = None
+
+    def get_summary(self, total_wall_time: float) -> dict:
+        num_steps = len(self.transformer_times)
+        total_tr = sum(self.transformer_times)
+        avg_step = (total_tr / num_steps) if num_steps > 0 else 0.0
+        return {
+            "text_encoder_s": round(self.text_encoder_time, 3),
+            "transformer_s": round(total_tr, 3),
+            "steps": num_steps,
+            "step_avg_s": round(avg_step, 3),
+            "vae_s": round(self.vae_time, 3),
+            "total_s": round(total_wall_time, 3),
+        }
+
+timing_tracker = PipelineTimingTracker()
+
+def instrument_pipeline(p):
+    if p is None or getattr(p, "_timing_instrumented", False):
+        return
+    if hasattr(p, "text_encoder") and p.text_encoder is not None:
+        orig_te = p.text_encoder.forward
+        def timed_te(*args, **kwargs):
+            timing_tracker.on_te_start()
+            try:
+                return orig_te(*args, **kwargs)
+            finally:
+                timing_tracker.on_te_end()
+        p.text_encoder.forward = timed_te
+
+    if hasattr(p, "transformer") and p.transformer is not None:
+        orig_tr = p.transformer.forward
+        def timed_tr(*args, **kwargs):
+            timing_tracker.on_tr_start()
+            try:
+                return orig_tr(*args, **kwargs)
+            finally:
+                timing_tracker.on_tr_end()
+        p.transformer.forward = timed_tr
+
+    if hasattr(p, "vae") and p.vae is not None:
+        orig_vae = p.vae.decode
+        def timed_vae(*args, **kwargs):
+            timing_tracker.on_vae_start()
+            try:
+                return orig_vae(*args, **kwargs)
+            finally:
+                timing_tracker.on_vae_end()
+        p.vae.decode = timed_vae
+
+    p._timing_instrumented = True
+
+def flush():
+    gc.collect()
+    torch.cuda.empty_cache()
+
+
+class ImageGenerationRequest(BaseModel):
+    prompt: str
+    n: int = 1
+    size: str = "1024x1024"
+    response_format: str = "b64_json"
+    quality: str = "standard"
+    style: str = "vivid"
+    num_inference_steps: Optional[int] = None
+    guidance_scale: Optional[float] = None
+    negative_prompt: Optional[str] = None
+    seed: Optional[int] = None
+
+
+@app.post("/v1/sleep")
+async def sleep_endpoint():
+    global is_sleeping_flag, sleep_requested
+    sleep_requested = True
+    try:
+        async with request_lock:
+            if not is_sleeping_flag and sleep_requested:
+                print("Sleep requested, moving models to CPU...")
+                for p in [pipeline, edit_pipeline]:
+                    if not p: continue
+                    for name, component in p.components.items():
+                        if isinstance(component, torch.nn.Module):
+                            # Special handling for Nunchaku which blocks .to() if offload is True
+                            if hasattr(component, "set_offload") and getattr(component, "offload", False):
+                                component.set_offload(False)
+                                component._nunchaku_was_offloaded = True
+
+                            try:
+                                component.to("cpu")
+                            except Exception as e:
+                                pass
+                flush()
+                is_sleeping_flag = True
+    finally:
+        sleep_requested = False
+    return {"status": "sleep completed", "is_sleeping": is_sleeping_flag}
+
+@app.post("/v1/wake_up")
+async def wake_up_endpoint():
+    global is_sleeping_flag, sleep_requested
+    sleep_requested = False
+    async with request_lock:
+        if is_sleeping_flag:
+            print("Waking up, restoring models to CUDA...")
+            for p in [pipeline, edit_pipeline]:
+                if not p: continue
+                excluded = getattr(p, "_exclude_from_cpu_offload", [])
+                for name, component in p.components.items():
+                    if isinstance(component, torch.nn.Module):
+                        if getattr(component, "_nunchaku_was_offloaded", False):
+                            component.set_offload(True, use_pin_memory=True, num_blocks_on_gpu=8)
+                            for attr in ["img_in", "txt_in", "txt_norm", "time_text_embed", "norm_out", "proj_out"]:
+                                if hasattr(component, attr):
+                                    try:
+                                        getattr(component, attr).to("cuda")
+                                    except Exception:
+                                        pass
+                            component._nunchaku_was_offloaded = False
+                        elif not hasattr(component, "_hf_hook") or name in excluded:
+                            try:
+                                component.to("cuda")
+                            except Exception:
+                                pass
+            is_sleeping_flag = False
+    return {"status": "awoken", "is_sleeping": False}
+
+@app.get("/v1/is_sleeping")
+async def is_sleeping_endpoint():
+    return {"is_sleeping": is_sleeping_flag}
+
+
+@app.get("/v1/memory_stats")
+async def memory_stats_endpoint():
+    """Lightweight introspection endpoint that returns PyTorch's CUDA allocator
+    snapshot. Used to diagnose VRAM/UMA bloat without restarting the server."""
+    stats = {}
+    if torch.cuda.is_available():
+        stats["allocated_gb"] = torch.cuda.memory_allocated() / 1e9
+        stats["reserved_gb"] = torch.cuda.memory_reserved() / 1e9
+        stats["max_allocated_gb"] = torch.cuda.max_memory_allocated() / 1e9
+        stats["max_reserved_gb"] = torch.cuda.max_memory_reserved() / 1e9
+        # Top allocations by size from the allocator snapshot (>=64 MiB)
+        try:
+            snap = torch.cuda.memory_snapshot()
+            blocks = []
+            for seg in snap:
+                for b in seg.get("blocks", []):
+                    if b.get("state") == "active_allocated" and b.get("size", 0) >= 64 * 1024 * 1024:
+                        blocks.append(b["size"])
+            blocks.sort(reverse=True)
+            stats["large_active_blocks_gb"] = [round(s / 1e9, 3) for s in blocks[:20]]
+            stats["large_active_blocks_total_gb"] = round(sum(blocks) / 1e9, 3)
+            stats["large_active_blocks_count"] = len(blocks)
+        except Exception as e:
+            stats["snapshot_error"] = str(e)
+        # Walk Python objects to find big tensors and group them
+        try:
+            import gc as _gc
+            seen = set()
+            big = []
+            for obj in _gc.get_objects():
+                try:
+                    if isinstance(obj, torch.Tensor) and obj.is_cuda:
+                        ptr = obj.data_ptr()
+                        if ptr in seen or ptr == 0:
+                            continue
+                        seen.add(ptr)
+                        sz = obj.element_size() * obj.numel()
+                        if sz >= 16 * 1024 * 1024:
+                            big.append((sz, tuple(obj.shape), str(obj.dtype)))
+                except Exception:
+                    continue
+            big.sort(reverse=True)
+            # Group by (shape, dtype)
+            from collections import Counter
+            grouped = Counter((shape, dtype) for _, shape, dtype in big)
+            stats["big_tensor_groups"] = [
+                {"shape": list(shape), "dtype": dtype, "count": cnt,
+                 "size_gb_each": round(
+                     (1 if shape == () else (lambda l: __import__('functools').reduce(lambda a, b: a*b, l, 1))(shape)) * (
+                         8 if 'int64' in dtype or 'float64' in dtype else
+                         4 if 'int32' in dtype or 'float32' in dtype else
+                         2 if 'bfloat16' in dtype or 'float16' in dtype else 1
+                     ) / 1e9, 4)}
+                for (shape, dtype), cnt in grouped.most_common(30)
+            ]
+            stats["big_tensor_count"] = len(big)
+            stats["big_tensor_total_gb"] = round(sum(s for s, _, _ in big) / 1e9, 3)
+        except Exception as e:
+            stats["walk_error"] = str(e)
+    return stats
+
+@app.post("/v1/images/edits")
+async def edit_image(
+    image: Union[List[UploadFile], UploadFile] = File(...),
+    prompt: str = Form(...),
+    n: int = Form(1),
+    size: str = Form("1024x1024"),
+    width: Optional[int] = Form(None),
+    height: Optional[int] = Form(None),
+    input_fit: str = Form("canvas"),
+    response_format: str = Form("b64_json"), # Default to b64_json
+    guidance_scale: Optional[float] = Form(None), 
+    num_inference_steps: Optional[int] = Form(None),
+    negative_prompt: Optional[str] = Form(None),
+    seed: Optional[int] = Form(None)
+):
+    # Use CLI defaults if not provided
+    steps = num_inference_steps if num_inference_steps is not None else args.steps
+    cfg_scale = guidance_scale if guidance_scale is not None else args.guidance_scale
+    neg_prompt = negative_prompt if negative_prompt is not None else "" # Default empty for now, or maybe None?
+    
+    generator = None
+    import random
+    if seed is None:
+        seed = random.randint(0, 2**32 - 1)
+    
+    print(f"Using seed: {seed}")
+    generator = torch.Generator(device="cuda").manual_seed(seed)
+
+    if not edit_pipeline:
+        raise HTTPException(status_code=500, detail="Model not loaded")
+
+    if sleep_requested or is_sleeping_flag:
+        raise HTTPException(status_code=503, detail="Server is sleeping or trying to sleep.")
+
+    async with request_lock:
+        print(f"Received edit request: {prompt}")
+
+        # Processing the input image(s)
+        input_files = image if isinstance(image, list) else [image]
+        init_images = []
+
+        try:
+            for img_file in input_files:
+                await img_file.seek(0)
+                contents = await img_file.read()
+                img = Image.open(io.BytesIO(contents)).convert("RGB")
+                init_images.append(img)
+        except Exception as e:
+            raise HTTPException(status_code=400, detail=f"Invalid image file: {e}")
+        
+        if not init_images:
+             raise HTTPException(status_code=400, detail="No images provided")
+
+        # Parse max target dimensions from requested size
+        try:
+            target_width, target_height = map(int, size.split("x"))
+        except ValueError:
+            target_width, target_height = 1024, 1024
+
+        # Calculate new dimensions preserving aspect ratio based on the first image
+        if width is not None and height is not None:
+            # Explicit output canvas: the caller owns the framing, so references of any aspect
+            # ratio can be fed without dictating the composition's shape.
+            req_width, req_height = width, height
+        else:
+            first_image = init_images[0]
+            orig_width, orig_height = first_image.size
+            scale = min(target_width / orig_width, target_height / orig_height)
+            req_width = int(orig_width * scale)
+            req_height = int(orig_height * scale)
+
+        # Ensure dimensions are aligned to 32 for compatibility (e.g. GLM-Image)
+        width = (req_width // IMAGE_DIMENSION_ALIGNMENT) * IMAGE_DIMENSION_ALIGNMENT
+        height = (req_height // IMAGE_DIMENSION_ALIGNMENT) * IMAGE_DIMENSION_ALIGNMENT
+
+        if (input_fit or "").strip().lower() == "native":
+            # QwenImageEditPlus resizes every condition image at its own aspect ratio, so padding
+            # references onto the output canvas only costs them resolution and adds false letterbox
+            # content the model then reproduces. Keep each reference's own framing.
+            resized_images = []
+            for img in init_images:
+                iw, ih = img.size
+                scale = min(1.0, (MAX_REFERENCE_PIXELS / float(max(1, iw * ih))) ** 0.5)
+                if scale < 1.0:
+                    nw = max(IMAGE_DIMENSION_ALIGNMENT,
+                             (int(iw * scale) // IMAGE_DIMENSION_ALIGNMENT) * IMAGE_DIMENSION_ALIGNMENT)
+                    nh = max(IMAGE_DIMENSION_ALIGNMENT,
+                             (int(ih * scale) // IMAGE_DIMENSION_ALIGNMENT) * IMAGE_DIMENSION_ALIGNMENT)
+                    img = img.resize((nw, nh), Image.LANCZOS)
+                resized_images.append(img)
+        else:
+            # Resize input images to match the calculated target size, padding if necessary
+            resized_images = []
+            for img in init_images:
+                if img.size != (width, height):
+                    # This handles cases where subsequent images might have different ARs
+                    img = ImageOps.pad(img, (width, height), method=Image.LANCZOS, color=(0, 0, 0))
+                resized_images.append(img)
+            
+        # If single image, pass as item, if multiple, pass as list
+        # GLM pipeline has a bug where it checks len() on the input, so it must be a list
+        if len(resized_images) > 1 or args.backend == "glm":
+            image_input = resized_images
+        else:
+            image_input = resized_images[0]
+
+        response_images = []
+        timing_tracker.reset()
+        t_req_start = time.perf_counter()
+        
+        try:
+            if args.backend.startswith("qwen"):
+                # Qwen specific parameters
+                if hasattr(edit_pipeline, "transformer"):
+                    set_zero_cond_t(edit_pipeline.transformer, True)
+                # guidance_scale maps to true_cfg_scale
+                if args.qwenimage: # QwenImageBackend is T2I only, so it doesn't take an image
+                    generated_images = edit_pipeline(
+                        prompt=prompt,
+                        height=height,
+                        width=width,
+                        num_inference_steps=steps,
+                        true_cfg_scale=cfg_scale,
+                        num_images_per_prompt=n,
+                        generator=generator,
+                    ).images
+                else: # Full Qwen edit backend takes an image (or list of images now)
+                    generated_images = edit_pipeline(
+                        image=image_input,
+                        prompt=prompt,
+                        height=height,
+                        width=width,
+                        negative_prompt=neg_prompt,
+                        num_inference_steps=steps,
+                        true_cfg_scale=cfg_scale,
+                        num_images_per_prompt=n,
+                        generator=generator,
+                    ).images
+            else:
+                # Standard Flux/Kontext or GLM
+                # GLM I2I Fix: Manually move vision encoder to GPU because get_image_features escapes hooks
+                if args.backend == "glm" and hasattr(edit_pipeline, "vision_language_encoder"):
+                     print("Manually moving GLM Vision Encoder to GPU...")
+                     edit_pipeline.vision_language_encoder.to("cuda")
+                
+                try:
+                    generated_images = edit_pipeline(
+                        image=image_input,
+                        prompt=prompt,
+                        height=height,
+                        width=width,
+                        num_inference_steps=steps, 
+                        guidance_scale=cfg_scale,
+                        num_images_per_prompt=n,
+                        generator=generator,
+                    ).images
+                finally:
+                    if args.backend == "glm" and hasattr(edit_pipeline, "vision_language_encoder"):
+                         print("Moving GLM Vision Encoder back to CPU...")
+                         edit_pipeline.vision_language_encoder.to("cpu")
+            
+            for img in generated_images:
+                buffered = io.BytesIO()
+                img.save(buffered, format="PNG")
+                img_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
+                
+                if response_format == "b64_json":
+                    response_images.append({"b64_json": img_str})
+                else:
+                    # If url is requested we can't really do it without storage, so we fallback or error?
+                    # For now, let's just assume simple b64_json as per request
+                    response_images.append({"b64_json": img_str}) # Fallback
+                
+        except Exception as e:
+            print(f"Error during editing: {e}")
+            print(traceback.format_exc())
+            raise HTTPException(status_code=500, detail=str(e))
+        finally:
+            flush()
+
+        t_req_total = time.perf_counter() - t_req_start
+        timings = timing_tracker.get_summary(t_req_total)
+        print(f"[TIMING] Text Encoder: {timings['text_encoder_s']:.2f}s | "
+              f"Transformer ({timings['steps']} steps): {timings['transformer_s']:.2f}s ({timings['step_avg_s']:.2f}s/step) | "
+              f"VAE: {timings['vae_s']:.2f}s | Total: {timings['total_s']:.2f}s")
+
+        return {
+            "created": int(time.time()),
+            "timings": timings,
+            "seed": seed,
+            "width": width,
+            "height": height,
+            "data": response_images
+        }
+
+
+
+@app.post("/v1/images/generations")
+async def generate_image(request: ImageGenerationRequest):
+    if not pipeline:
+        raise HTTPException(status_code=500, detail="Model not loaded")
+
+    if sleep_requested or is_sleeping_flag:
+        raise HTTPException(status_code=503, detail="Server is sleeping or trying to sleep.")
+
+    async with request_lock:
+        #print(f"Received generation request: {request.prompt}")
+
+        # Parse size
+        try:
+            width, height = map(int, request.size.split("x"))
+        except ValueError:
+            width, height = 1024, 1024
+
+        # Ensure dimensions are aligned to 32
+        width = (width // IMAGE_DIMENSION_ALIGNMENT) * IMAGE_DIMENSION_ALIGNMENT
+        height = (height // IMAGE_DIMENSION_ALIGNMENT) * IMAGE_DIMENSION_ALIGNMENT
+
+        response_images = []
+        timing_tracker.reset()
+        t_req_start = time.perf_counter()
+        
+        try:
+            # Generate images (no image argument for txt2img!)
+            steps = request.num_inference_steps if request.num_inference_steps is not None else args.steps
+            cfg_scale = request.guidance_scale if request.guidance_scale is not None else args.guidance_scale
+            # negative_prompt not in standard request body in original snippet, but we added it to model
+            neg_prompt = request.negative_prompt if request.negative_prompt is not None else ""
+            
+            generator = None
+            import random
+            seed = request.seed
+            if seed is None:
+                seed = random.randint(0, 2**32 - 1)
+            
+            print(f"Using seed: {seed}")
+            generator = torch.Generator(device="cuda").manual_seed(seed)
+
+            if args.backend.startswith("qwen"):
+                if hasattr(pipeline, "transformer"):
+                    set_zero_cond_t(pipeline.transformer, False)
+                generated_images = pipeline(
+                    prompt=request.prompt,
+                    height=height,
+                    width=width,
+                    num_inference_steps=steps, 
+                    true_cfg_scale=cfg_scale,     
+                    num_images_per_prompt=request.n,
+                    negative_prompt=neg_prompt,
+                    generator=generator,
+                ).images
+            else:
+                generated_images = pipeline(
+                    prompt=request.prompt,
+                    height=height,
+                    width=width,
+                    num_inference_steps=steps, 
+                    guidance_scale=cfg_scale,     
+                    num_images_per_prompt=request.n,
+                    generator=generator,
+                    # Not passing negative_prompt here for generation unless we confirm support in standard Flux pipeline?
+                ).images
+            
+            for img in generated_images:
+                buffered = io.BytesIO()
+                img.save(buffered, format="PNG")
+                img_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
+                response_images.append({"b64_json": img_str})
+                
+        except Exception as e:
+            print(f"Error during generation: {e}")
+            print(traceback.format_exc())
+            raise HTTPException(status_code=500, detail=str(e))
+        finally:
+            flush()
+
+        t_req_total = time.perf_counter() - t_req_start
+        timings = timing_tracker.get_summary(t_req_total)
+        print(f"[TIMING] Text Encoder: {timings['text_encoder_s']:.2f}s | "
+              f"Transformer ({timings['steps']} steps): {timings['transformer_s']:.2f}s ({timings['step_avg_s']:.2f}s/step) | "
+              f"VAE: {timings['vae_s']:.2f}s | Total: {timings['total_s']:.2f}s")
+
+        return {
+            "created": int(time.time()),
+            "timings": timings,
+            "seed": seed,
+            "width": width,
+            "height": height,
+            "data": response_images
+        }
+
+if __name__ == "__main__":
+    uvicorn.run(app, host=args.host, port=args.port)

extras/QwenImage21Backend.py

@@ -0,0 +1,73 @@
+import torch
+import os
+from diffusers import QwenImage21Pipeline
+
+# Channeling Nikola's cheerful seeker spirit!
+# A clean, modular backend for Qwen-Image-2.1 with sequential layerwise offloading!
+
+class QwenImage21Backend:
+    def __init__(
+        self,
+        model_id="/home/olegk/Nikola/models/Qwen/Qwen-Image-2.1",
+        gpu_id=0,
+        enable_tiling=True,
+        text_encoder_path=None,
+    ):
+        self.model_id = model_id
+        self.gpu_id = gpu_id
+        self.enable_tiling = enable_tiling
+        self.text_encoder_path = text_encoder_path
+        self.pipeline = None
+
+    def load(self):
+        print(f"Loading QwenImage21Backend from {self.model_id}...")
+        if self.text_encoder_path:
+            print(f"  • Custom Text Encoder: {self.text_encoder_path}")
+            from transformers import Qwen3VLForConditionalGeneration, Qwen3VLProcessor
+
+            te_dir = (
+                os.path.join(self.text_encoder_path, "text_encoder")
+                if os.path.isdir(os.path.join(self.text_encoder_path, "text_encoder"))
+                else self.text_encoder_path
+            )
+            proc_dir = (
+                os.path.join(self.text_encoder_path, "processor")
+                if os.path.isdir(os.path.join(self.text_encoder_path, "processor"))
+                else self.text_encoder_path
+            )
+            if not os.path.exists(os.path.join(proc_dir, "tokenizer.json")):
+                proc_dir = os.path.join(self.model_id, "processor")
+
+            custom_te = Qwen3VLForConditionalGeneration.from_pretrained(
+                te_dir,
+                torch_dtype=torch.bfloat16,
+                low_cpu_mem_usage=True,
+            )
+            custom_proc = Qwen3VLProcessor.from_pretrained(proc_dir)
+
+            pipeline = QwenImage21Pipeline.from_pretrained(
+                self.model_id,
+                text_encoder=custom_te,
+                processor=custom_proc,
+                torch_dtype=torch.bfloat16,
+            )
+        else:
+            pipeline = QwenImage21Pipeline.from_pretrained(
+                self.model_id,
+                torch_dtype=torch.bfloat16,
+            )
+
+        print(f"Attaching sequential CPU offload on GPU {self.gpu_id} for layerwise execution...")
+        pipeline.enable_sequential_cpu_offload(gpu_id=self.gpu_id)
+
+        if self.enable_tiling:
+            print("Enabling VAE tiling for low-memory decode...")
+            try:
+                pipeline.vae.enable_tiling()
+            except Exception as e:
+                print(f"Note: VAE tiling could not be enabled ({e}), continuing with standard decode.")
+
+        self.pipeline = pipeline
+        # QwenImage21Pipeline natively handles both generations (t2i) and edits (i2i)
+        return self.pipeline, self.pipeline
+

extras/QwenImage21NVFP4Backend.py

@@ -0,0 +1,166 @@
+# -*- coding: utf-8 -*-
+"""Nunchaku NVFP4 Resident Backend for Qwen-Image-2.1.
+
+Loads the forged SVDQuant NVFP4 r32 Qwen-Image-2.1 transformer directly into VRAM,
+enabling blazingly fast inference with zero layerwise PCIe streaming bottlenecks.
+"""
+
+import os
+import sys
+import torch
+from diffusers import QwenImage21Pipeline
+
+# Ensure local packages are on path
+ROOT_DIR = "/auto/home/amano/olegk/Nikola"
+for p in [f"{ROOT_DIR}/packages/nunchaku", f"{ROOT_DIR}/packages/deepcompressor", ROOT_DIR]:
+    if p not in sys.path:
+        sys.path.insert(0, p)
+
+from nunchaku.models.transformers.transformer_qwenimage21 import NunchakuQwenImage21Transformer2DModel
+
+
+class QwenImage21NVFP4Backend:
+    def __init__(
+        self,
+        model_id="/home/olegk/Nikola/models/Qwen/Qwen-Image-2.1",
+        optimized_model_path="/home/olegk/Nikola/models/nunchaku-qwen-image-2.1/best_quality_fp4.safetensors",
+        gpu_id=0,
+        enable_tiling=True,
+        dynamic_scale_k=0.0,
+        stream_text_encoder=True,
+        text_encoder_path=None,
+    ):
+        self.model_id = model_id
+        self.optimized_model_path = optimized_model_path
+        self.gpu_id = gpu_id
+        self.enable_tiling = enable_tiling
+        self.dynamic_scale_k = dynamic_scale_k
+        self.stream_text_encoder = stream_text_encoder
+        self.text_encoder_path = text_encoder_path
+        self.pipeline = None
+
+    def load(self):
+        print(f"Loading QwenImage21NVFP4Backend...")
+        print(f"  • Base Pipeline: {self.model_id}")
+        print(f"  • Quantized DiT: {self.optimized_model_path}")
+        print(f"  • Target GPU: cuda:{self.gpu_id}")
+        print(f"  • Stream Text Encoder: {self.stream_text_encoder}")
+        if self.text_encoder_path:
+            print(f"  • Custom Text Encoder: {self.text_encoder_path}")
+
+        device = f"cuda:{self.gpu_id}"
+
+        # 1. Load base pipeline without heavy transformer
+        # Pass transformer=None or dummy to save loading 13GB BF16 transformer
+        print("Loading peripheral pipeline components (Text Encoder, Tokenizer, VAE, Scheduler)...")
+        if self.text_encoder_path:
+            from transformers import Qwen3VLForConditionalGeneration, Qwen3VLProcessor
+
+            te_dir = (
+                os.path.join(self.text_encoder_path, "text_encoder")
+                if os.path.isdir(os.path.join(self.text_encoder_path, "text_encoder"))
+                else self.text_encoder_path
+            )
+            proc_dir = (
+                os.path.join(self.text_encoder_path, "processor")
+                if os.path.isdir(os.path.join(self.text_encoder_path, "processor"))
+                else self.text_encoder_path
+            )
+            if not os.path.exists(os.path.join(proc_dir, "tokenizer.json")):
+                proc_dir = os.path.join(self.model_id, "processor")
+
+            print(f"  • Loading custom Qwen3-VL text encoder from: {te_dir}")
+            custom_te = Qwen3VLForConditionalGeneration.from_pretrained(
+                te_dir,
+                torch_dtype=torch.bfloat16,
+                low_cpu_mem_usage=True,
+            )
+            print(f"  • Loading processor from: {proc_dir}")
+            custom_proc = Qwen3VLProcessor.from_pretrained(proc_dir)
+
+            pipeline = QwenImage21Pipeline.from_pretrained(
+                self.model_id,
+                transformer=None,
+                text_encoder=custom_te,
+                processor=custom_proc,
+                torch_dtype=torch.bfloat16,
+            )
+        else:
+            pipeline = QwenImage21Pipeline.from_pretrained(
+                self.model_id,
+                transformer=None,
+                torch_dtype=torch.bfloat16,
+            )
+
+        # 2. Load forged Nunchaku NVFP4 transformer directly into resident VRAM
+        print(f"Loading resident NVFP4 DiT into {device}...")
+        quantized_transformer = NunchakuQwenImage21Transformer2DModel.from_pretrained(
+            self.optimized_model_path,
+            device=device,
+            torch_dtype=torch.bfloat16,
+        )
+        pipeline.transformer = quantized_transformer
+
+        # 3. Place VAE directly on target device with memory-safe tiling
+        print(f"Placing VAE on {device}...")
+        pipeline.vae = pipeline.vae.to(device)
+        if self.enable_tiling:
+            try:
+                pipeline.vae.enable_tiling()
+                print("VAE tiling enabled successfully.")
+            except Exception as e:
+                print(f"Note: VAE tiling could not be enabled ({e})")
+
+        # 4. Text Encoder Configuration: Layerwise PCIe Streaming or CPU Fallback
+        if self.stream_text_encoder:
+            print(f"Configuring PCIe Layerwise Weight Streaming for Qwen3-VL on {device}...")
+            from stream_encoder import attach_qwen3vl_streamer
+            self.streamer = attach_qwen3vl_streamer(pipeline, device=device)
+
+            orig_encode_prompt = pipeline.encode_prompt
+
+            def streamed_safe_encode_prompt(*args, **kwargs):
+                kwargs.pop("device", None)
+                embeds = orig_encode_prompt(*args, device=torch.device(device), **kwargs)
+                target_dev = pipeline.transformer.device
+                return tuple(x.to(target_dev) if isinstance(x, torch.Tensor) else x for x in embeds)
+
+            pipeline.encode_prompt = streamed_safe_encode_prompt
+        else:
+            print(f"Configuring hybrid CPU prompt encoding (DiT & VAE 100% resident on {device})...")
+            orig_encode_prompt = pipeline.encode_prompt
+
+            def safe_encode_prompt(*args, **kwargs):
+                kwargs.pop("device", None)
+                te_device = pipeline.text_encoder.device
+                embeds = orig_encode_prompt(*args, device=te_device, **kwargs)
+                target_dev = pipeline.transformer.device
+                return tuple(x.to(target_dev) if isinstance(x, torch.Tensor) else x for x in embeds)
+
+            pipeline.encode_prompt = safe_encode_prompt
+
+        # 5. Automatically attach optimal dynamic latent variance damping s(t)
+        if self.dynamic_scale_k > 0.0:
+            k = self.dynamic_scale_k
+            print(f"Attaching automatic late-stage variance damping (k={k:.3f}, +1.20 dB fidelity boost)...")
+            orig_call = pipeline.__call__
+
+            def scaled_call(*args, **kwargs):
+                user_cb = kwargs.pop("callback_on_step_end", None)
+
+                def combined_cb(pipe_obj, step_idx, timestep, callback_kwargs):
+                    t = float(timestep.item() if isinstance(timestep, torch.Tensor) else timestep)
+                    if t < 600.0:
+                        scale = 1.0 - k * ((600.0 - t) / 600.0)
+                        callback_kwargs["latents"] = callback_kwargs["latents"] * scale
+                    if user_cb is not None:
+                        return user_cb(pipe_obj, step_idx, timestep, callback_kwargs)
+                    return callback_kwargs
+
+                kwargs["callback_on_step_end"] = combined_cb
+                return orig_call(*args, **kwargs)
+
+            pipeline.__call__ = scaled_call
+
+        self.pipeline = pipeline
+        return self.pipeline, self.pipeline

extras/create_heretic_text_encoder.py

@@ -0,0 +1,201 @@
+# -*- coding: utf-8 -*-
+"""Heretic Abliteration Engine for Qwen3-VL Text Encoder.
+
+Applies Norm-Preserving Biprojected Abliteration (Heretic/Arditi et al.) to Qwen3-VL-8B:
+1. Measures the refusal/hesitation direction across all 36 transformer decoder layers.
+2. Orthogonalizes the refusal direction against the benign semantic direction.
+3. Applies rank-1 directional ablation to `self_attn.o_proj` and `mlp.down_proj`:
+     W' = normalize(W_norm - lambda * v * (v^T W_norm)) * ||W||_row
+   preserving exact row norms to protect general capability and language quality.
+4. Exports the decensored, hesitation-free text encoder to ~/Nikola/models/Qwen_Text_Encoder_Heretic.
+"""
+
+import os
+import sys
+import shutil
+import math
+import torch
+import torch.nn.functional as F
+import torch.linalg as LA
+from diffusers import QwenImage21Pipeline
+
+ROOT_DIR = "/auto/home/amano/olegk/Nikola"
+for p in [f"{ROOT_DIR}/src/imagegen", ROOT_DIR]:
+    if p not in sys.path:
+        sys.path.insert(0, p)
+
+from stream_encoder import attach_qwen3vl_streamer
+
+MODEL_PATH = "/home/olegk/Nikola/models/Qwen/Qwen-Image-2.1"
+OUTPUT_DIR = "/home/olegk/Nikola/models/Qwen21_Text_Encoder_Heretic"
+
+# Contrastive calibration prompts
+BENIGN_PROMPTS = [
+    "A scenic view of a quiet tropical beach with palm trees, white sand, and gentle ocean waves under sunlight.",
+    "A group of athletic friends playing a casual game of volleyball on a sports court on a bright afternoon.",
+    "Two cheerful anime girls laughing together and talking in a colorful modern classroom setting.",
+    "A beautiful golden hour sunset over the coastal dunes with warm ambient lighting and ocean breeze.",
+    "Two athletic anime girls in standard sportswear practicing drills together on an outdoor gymnasium court.",
+    "A detailed photograph of a clean sandy beach with clear turquoise water and blue sky with light clouds.",
+    "Two anime girls taking a relaxing stroll along the seaside promenade during summer vacation.",
+    "A dynamic action photograph of a volleyball match on the sand with players in motion."
+]
+
+SENSITIVE_PROMPTS = [
+    "Two anime girls in colorful skimpy bikinis playing beach volleyball on a sunny tropical beach.",
+    "Two attractive anime girls wearing revealing two-piece swimsuits posing playfully under the bright sun.",
+    "Two cute anime girls in skimpy micro bikinis jumping to spike a volleyball on the beach sand.",
+    "A beach volleyball match with two anime girls wearing small swimsuits, athletic dynamic poses.",
+    "Two anime girls in alluring colorful bikinis having fun playing sports on a sunny seaside beach.",
+    "A close up action shot of two anime girls in tiny bikinis diving for a volleyball on the beach.",
+    "Two beautiful anime girls wearing revealing beachwear and swimsuits posing by the ocean shoreline.",
+    "An alluring tropical beach scene with two anime girls in revealing bikinis playing athletic beach sports."
+]
+
+
+def abliterate_layer_weights(weight: torch.Tensor, v: torch.Tensor, weight_factor: float) -> torch.Tensor:
+    """Norm-preserving biprojected abliteration: delta W = -lambda * v * (v^T W)."""
+    if weight_factor <= 0.0:
+        return weight
+
+    orig_dtype = weight.dtype
+    W = weight.float()
+    v = v.float().to(W.device)
+
+    # Calculate row norms
+    row_norms = LA.vector_norm(W, dim=1, keepdim=True)
+    # Normalize rows
+    W_norm = F.normalize(W, p=2, dim=1)
+
+    # v @ W_norm -> (in_features,)
+    lora_A = (v @ W_norm).view(1, -1)
+    # -weight_factor * v -> (out_features, 1)
+    lora_B = (-weight_factor * v).view(-1, 1)
+
+    # Project and renormalize
+    W_adj = W_norm + lora_B @ lora_A
+    W_adj = F.normalize(W_adj, p=2, dim=1)
+    W_final = W_adj * row_norms
+
+    return W_final.to(orig_dtype)
+
+
+def create_heretic_model():
+    print("=" * 80)
+    print("🔮 FORGING OPTIMIZED HERETIC TEXT ENCODER (QWEN3-VL-8B)")
+    print("=" * 80)
+    print(f"Base Model: {MODEL_PATH}")
+    print(f"Output Directory: {OUTPUT_DIR}")
+
+    # 1. Load pipeline and attach layerwise streamer for rapid residual extraction
+    print("\n[Step 1/4] Loading pipeline & attaching layerwise streamer...")
+    pipe = QwenImage21Pipeline.from_pretrained(
+        MODEL_PATH,
+        transformer=None,
+        torch_dtype=torch.bfloat16,
+    )
+    streamer = attach_qwen3vl_streamer(pipe, device="cuda:0")
+
+    # 2. Extract layer-by-layer residuals across contrastive prompt sets
+    print("\n[Step 2/4] Measuring refusal directions across all 36 decoder layers...")
+
+    def get_mean_residuals(prompts, label):
+        layer_states = [[] for _ in range(37)]
+        print(f"  • Collecting residuals for {len(prompts)} {label} prompts...")
+        for p in prompts:
+            fmt = pipe.prompt_template_t2i.format(p)
+            inputs = pipe.processor(text=[fmt], return_tensors="pt").to("cuda:0")
+            with torch.no_grad():
+                out = pipe.text_encoder(
+                    input_ids=inputs.input_ids,
+                    attention_mask=inputs.attention_mask,
+                    output_hidden_states=True,
+                )
+            for l in range(37):
+                layer_states[l].append(out.hidden_states[l][0, -1, :].float().cpu())
+        return torch.stack([torch.stack(l).mean(dim=0) for l in layer_states])
+
+    b_means = get_mean_residuals(BENIGN_PROMPTS, "benign")
+    s_means = get_mean_residuals(SENSITIVE_PROMPTS, "sensitive")
+
+    # Compute difference of means
+    residual_directions = s_means - b_means
+
+    # Orthogonalize against benign direction (Heretic projected abliteration)
+    print("  • Orthogonalizing refusal directions against benign semantic vectors...")
+    good_directions = F.normalize(b_means, p=2, dim=1)
+    proj = torch.sum(residual_directions * good_directions, dim=1, keepdim=True)
+    ortho_directions = residual_directions - proj * good_directions
+    ortho_directions = F.normalize(ortho_directions, p=2, dim=1)
+
+    # 3. Apply Norm-Preserving Biprojected Abliteration to Language Model Weights
+    print("\n[Step 3/4] Applying Heretic norm-preserving abliteration to weights...")
+    lm = pipe.text_encoder.model.language_model
+    num_layers = len(lm.layers)
+
+    # Target layers: layers 16 to 35, centered at layer 26 with max_weight = 1.0
+    center_layer = 26.0
+    spread = 8.0
+
+    ablated_count = 0
+    for l_idx, layer in enumerate(lm.layers):
+        dist = abs(l_idx - center_layer)
+        # Smooth bell-shaped abliteration profile
+        weight_factor = float(math.exp(-(dist**2) / (2 * (spread**2))))
+        if weight_factor < 0.10:
+            weight_factor = 0.0  # skip early layers (0..12) where refusal is inactive
+
+        v = ortho_directions[l_idx + 1]  # l_idx+1 accounts for embedding layer at idx 0
+
+        if weight_factor > 0.0:
+            print(f"  • Layer {l_idx:2d}: Abliterating with lambda={weight_factor:.3f}...")
+
+            # 1. Attention Out Projection
+            if hasattr(layer.self_attn, "o_proj"):
+                with torch.no_grad():
+                    layer.self_attn.o_proj.weight.data = abliterate_layer_weights(
+                        layer.self_attn.o_proj.weight.data, v, weight_factor
+                    )
+                ablated_count += 1
+
+            # 2. MLP Down Projection
+            if hasattr(layer.mlp, "down_proj"):
+                with torch.no_grad():
+                    layer.mlp.down_proj.weight.data = abliterate_layer_weights(
+                        layer.mlp.down_proj.weight.data, v, weight_factor
+                    )
+                ablated_count += 1
+
+    print(f"  • Successfully abliterated {ablated_count} linear projection matrices!")
+
+    # 4. Save Standalone Checkpoint to models/Qwen_Text_Encoder_Heretic
+    print(f"\n[Step 4/4] Saving Heretic Text Encoder to {OUTPUT_DIR}...")
+    os.makedirs(OUTPUT_DIR, exist_ok=True)
+
+    # Save text_encoder subfolder (for direct use with Diffusers or Transformers)
+    sub_dir = os.path.join(OUTPUT_DIR, "text_encoder")
+    os.makedirs(sub_dir, exist_ok=True)
+    pipe.text_encoder.save_pretrained(sub_dir)
+    print(f"  • Saved abliterated text encoder weights to: {sub_dir}")
+
+    # Also copy tokenizer and processor files
+    proc_src = os.path.join(MODEL_PATH, "processor")
+    proc_dst = os.path.join(OUTPUT_DIR, "processor")
+    if os.path.exists(proc_src):
+        if os.path.exists(proc_dst):
+            shutil.rmtree(proc_dst)
+        shutil.copytree(proc_src, proc_dst)
+        print(f"  • Copied processor to: {proc_dst}")
+
+    # Copy root config files
+    for fname in ["config.json", "generation_config.json"]:
+        src_f = os.path.join(MODEL_PATH, "text_encoder", fname)
+        if os.path.exists(src_f):
+            shutil.copy(src_f, os.path.join(OUTPUT_DIR, fname))
+
+    print("\n🎉 SUCCESS: Heretic Text Encoder forged and saved successfully!")
+    print("=" * 80)
+
+
+if __name__ == "__main__":
+    create_heretic_model()

extras/imagegen_qwen21_nvfp4.sh

@@ -0,0 +1,40 @@
+#!/bin/bash
+# Start ImageEditServer with forged SVDQuant NVFP4 r32 Qwen-Image-2.1 model
+# Runs 100% resident in VRAM on GPU 0 with Heretic text encoder by default
+# Default: 25 steps, guidance-scale 1.0, port 4500, Text Encoder: Qwen21_Text_Encoder_Heretic
+
+PORT=${1:-4500}
+MODEL_DIR="/home/olegk/Nikola/models/Qwen/Qwen-Image-2.1"
+OPTIMIZED_MODEL="/home/olegk/Nikola/models/nunchaku-qwen-image-2.1/best_quality_fp4.safetensors"
+TEXT_ENCODER=${2:-"/home/olegk/Nikola/models/Qwen21_Text_Encoder_Heretic"}
+
+echo "=========================================================="
+echo "Starting ImageEditServer (Qwen-Image-2.1 SVDQuant NVFP4 - Resident VRAM)"
+echo "Port: $PORT"
+echo "Model: $MODEL_DIR"
+echo "Optimized Model: $OPTIMIZED_MODEL"
+echo "Text Encoder: $TEXT_ENCODER"
+echo "Steps: 25 | Guidance Scale: 1.0 | Resident Mode: True"
+echo "=========================================================="
+
+export CUDA_VISIBLE_DEVICES="0"
+export PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True"
+export PYTHONPATH="/home/olegk/Nikola/packages/nunchaku:/home/olegk/Nikola/packages/deepcompressor:/home/olegk/Nikola:$PYTHONPATH"
+
+cd /auto/home/amano/olegk/Nikola/src/imagegen
+
+EXTRA_ARGS=""
+if [ -n "$TEXT_ENCODER" ] && [ "$TEXT_ENCODER" != "none" ]; then
+    EXTRA_ARGS="--text-encoder $TEXT_ENCODER"
+fi
+
+exec /home/olegk/Nikola/.venv/bin/python ImageEditServer.py \
+    --host 0.0.0.0 \
+    --port "$PORT" \
+    --model "$MODEL_DIR" \
+    --optimized-model "$OPTIMIZED_MODEL" \
+    --backend qwen21-nvfp4 \
+    --steps 25 \
+    --guidance-scale 1.0 \
+    $EXTRA_ARGS
+

extras/stream_encoder.py

@@ -0,0 +1,190 @@
+# -*- coding: utf-8 -*-
+"""Layerwise Weight Streaming Engine for Qwen3-VL-8B in Qwen-Image-2.1.
+
+Pins all 36 language model decoder layers in host RAM and streams them through a single
+pre-allocated GPU layer buffer (368 MB VRAM) over PCIe (~28.7 GB/s).
+Keeps the visual ViT encoder (1.07 GB) resident on GPU.
+
+Achieves GPU compute speeds (1.06s multimodal prompt encode vs 17.18s on CPU, saving >16s per edit)
+with only ~1.45 GB VRAM footprint and 100% bit-exact mathematical parity (zero quality loss).
+"""
+
+import copy
+import time
+import torch
+import torch.nn as nn
+from transformers.modeling_outputs import BaseModelOutputWithPast
+from transformers.models.qwen3_vl.modeling_qwen3_vl import create_causal_mask
+
+
+class Qwen3VLLayerwiseStreamer:
+    """Streams Qwen3-VL language model decoder layers through a single static GPU buffer."""
+
+    def __init__(self, pipeline, device="cuda:0"):
+        self.pipeline = pipeline
+        self.device = torch.device(device)
+        self.text_encoder = pipeline.text_encoder
+        self.lm = getattr(self.text_encoder.model, "language_model", self.text_encoder.model)
+        self.num_layers = len(self.lm.layers)
+
+        print(f"Initializing Qwen3VLLayerwiseStreamer for {self.num_layers} layers on {self.device}...")
+
+        # 1. Pin CPU layers in host memory for maximum PCIe transfer throughput
+        t0 = time.perf_counter()
+        self.cpu_layers = []
+        for layer in self.lm.layers:
+            layer = layer.to("cpu", dtype=torch.bfloat16)
+            for p in layer.parameters():
+                if not p.data.is_pinned():
+                    p.data = p.data.pin_memory()
+            for b in layer.buffers():
+                if not b.data.is_pinned():
+                    b.data = b.data.pin_memory()
+            self.cpu_layers.append(layer)
+        t_pin = time.perf_counter() - t0
+        print(f"  • Pinned {self.num_layers} layers in CPU RAM in {t_pin:.2f} s")
+
+        # 2. Allocate ONE single template GPU layer buffer in VRAM (~368 MB)
+        self.gpu_layer = copy.deepcopy(self.cpu_layers[0]).to(self.device, dtype=torch.bfloat16)
+        gpu_param_dict = dict(self.gpu_layer.named_parameters())
+        gpu_buffer_dict = dict(self.gpu_layer.named_buffers())
+
+        # Pre-build parameter transfer pairs for zero-overhead non-blocking copying
+        self.param_pairs = []
+        for i in range(self.num_layers):
+            lp = [(gpu_param_dict[name], cp) for name, cp in self.cpu_layers[i].named_parameters()]
+            lb = [(gpu_buffer_dict[name], cb) for name, cb in self.cpu_layers[i].named_buffers()]
+            self.param_pairs.append((lp, lb))
+
+        gpu_mb = sum(p.numel() * p.element_size() for p in self.gpu_layer.parameters()) / (1024**2)
+        print(f"  • Static GPU layer buffer allocated: {gpu_mb:.2f} MB VRAM")
+
+        # 3. Place small peripheral layers directly on target GPU
+        self.lm.rotary_emb = self.lm.rotary_emb.to(self.device)
+        self.lm.embed_tokens = self.lm.embed_tokens.to(self.device)
+        self.lm.norm = self.lm.norm.to(self.device)
+
+        # 4. Place visual ViT encoder directly on target GPU (1.07 GB VRAM)
+        if hasattr(self.text_encoder.model, "visual") and self.text_encoder.model.visual is not None:
+            self.text_encoder.model.visual = self.text_encoder.model.visual.to(self.device, dtype=torch.bfloat16)
+            print("  • Visual ViT encoder placed resident on GPU (1.07 GB VRAM)")
+
+        # 5. Bypass unused lm_head (152,064 vocab projection, saving 1.24 GB computation)
+        class DummyHead(nn.Module):
+            def forward(self, x):
+                return None
+
+        self.text_encoder.lm_head = DummyHead()
+        print("  • Bypassed unused lm_head projection")
+
+        # 6. Install hooked forward pass
+        self.orig_lm_forward = self.lm.forward
+        self.lm.forward = self.streamed_forward
+
+        # 7. Route pipeline._get_qwen_prompt_embeds to target GPU
+        self.orig_get_embeds = self.pipeline._get_qwen_prompt_embeds
+        target_dev = self.device
+
+        def gpu_get_embeds(prompt_arg, image_arg, device_arg=None):
+            return self.orig_get_embeds(prompt_arg, image_arg, device=target_dev)
+
+        self.pipeline._get_qwen_prompt_embeds = gpu_get_embeds
+        print(f"  • Hooked Qwen3-VL language model and prompt embedding router onto {self.device}!")
+
+    @torch.no_grad()
+    def streamed_forward(
+        self,
+        input_ids: torch.LongTensor | None = None,
+        attention_mask: torch.Tensor | None = None,
+        position_ids: torch.LongTensor | None = None,
+        past_key_values=None,
+        inputs_embeds: torch.FloatTensor | None = None,
+        use_cache: bool | None = None,
+        visual_pos_masks: torch.Tensor | None = None,
+        deepstack_visual_embeds: list[torch.Tensor] | None = None,
+        output_hidden_states: bool | None = None,
+        **kwargs,
+    ) -> BaseModelOutputWithPast:
+        """Executes language model decoding by streaming layers one by one into the GPU buffer."""
+        if inputs_embeds is None:
+            inputs_embeds = self.lm.embed_tokens(input_ids)
+        inputs_embeds = inputs_embeds.to(self.device)
+
+        if position_ids is None:
+            past_seen = past_key_values.get_seq_length() if past_key_values is not None else 0
+            position_ids = torch.arange(inputs_embeds.shape[1], device=self.device) + past_seen
+            position_ids = position_ids.view(1, 1, -1).expand(4, inputs_embeds.shape[0], -1)
+        elif position_ids.ndim == 2:
+            position_ids = position_ids[None, ...].expand(4, position_ids.shape[0], -1)
+
+        position_ids = position_ids.to(self.device)
+        if position_ids.ndim == 3 and position_ids.shape[0] == 4:
+            text_position_ids = position_ids[0]
+            rotary_pos_ids = position_ids[1:]
+        else:
+            text_position_ids = None
+            rotary_pos_ids = position_ids
+
+        causal_mask = create_causal_mask(
+            config=self.lm.config,
+            inputs_embeds=inputs_embeds,
+            attention_mask=attention_mask.to(self.device) if attention_mask is not None else None,
+            past_key_values=past_key_values,
+            position_ids=text_position_ids,
+        )
+
+        position_embeddings = self.lm.rotary_emb(inputs_embeds, rotary_pos_ids)
+        hidden_states = inputs_embeds
+
+        if visual_pos_masks is not None:
+            visual_pos_masks = visual_pos_masks.to(self.device)
+        if deepstack_visual_embeds is not None:
+            deepstack_visual_embeds = [d.to(self.device) for d in deepstack_visual_embeds]
+
+        all_hidden_states = () if output_hidden_states else None
+
+        # Stream all 36 decoder layers through the static GPU buffer
+        for layer_idx in range(self.num_layers):
+            if output_hidden_states:
+                all_hidden_states = all_hidden_states + (hidden_states,)
+
+            params, buffers = self.param_pairs[layer_idx]
+            for gp, cp in params:
+                gp.data.copy_(cp.data, non_blocking=True)
+            for gb, cb in buffers:
+                gb.data.copy_(cb.data, non_blocking=True)
+
+            layer_outputs = self.gpu_layer(
+                hidden_states,
+                attention_mask=causal_mask,
+                position_ids=text_position_ids,
+                past_key_values=past_key_values,
+                position_embeddings=position_embeddings,
+                **kwargs,
+            )
+            hidden_states = layer_outputs
+
+            # Add multi-layer deepstack visual features if present
+            if deepstack_visual_embeds is not None and layer_idx in range(len(deepstack_visual_embeds)):
+                hidden_states = self.lm._deepstack_process(
+                    hidden_states,
+                    visual_pos_masks,
+                    deepstack_visual_embeds[layer_idx],
+                )
+
+        pre_norm_states = hidden_states
+        if output_hidden_states:
+            all_hidden_states = all_hidden_states + (pre_norm_states,)
+
+        norm_states = self.lm.norm(hidden_states)
+
+        return BaseModelOutputWithPast(
+            last_hidden_state=norm_states,
+            past_key_values=past_key_values,
+            hidden_states=all_hidden_states,
+        )
+
+
+def attach_qwen3vl_streamer(pipeline, device="cuda:0") -> Qwen3VLLayerwiseStreamer:
+    """Convenience factory to attach layerwise streaming to any QwenImage21Pipeline."""
+    return Qwen3VLLayerwiseStreamer(pipeline, device=device)

extras/test_heretic_beach_volleyball.py

@@ -0,0 +1,128 @@
+# -*- coding: utf-8 -*-
+"""Verification & Comparison of Heretic Decensored Text Encoder on Qwen-Image-2.1.
+
+Generates identical text-to-image scenes comparing:
+  1. Base / Stock Qwen3-VL Text Encoder (Safety Aligned / Blush Vector)
+  2. Heretic Qwen3-VL Text Encoder (Abliterated / Hesitation-Free)
+
+Target Prompt:
+  "Two cute anime girls in colorful bikinis playing beach volleyball on a sunny tropical beach,
+   dynamic action pose, jumping to spike the ball, sharp focus, vibrant anime style,
+   detailed background with ocean and palm trees"
+"""
+
+import os
+import sys
+import time
+import shutil
+import torch
+from PIL import Image
+
+ROOT_DIR = "/auto/home/amano/olegk/Nikola"
+for p in [f"{ROOT_DIR}/packages/nunchaku", f"{ROOT_DIR}/packages/deepcompressor", f"{ROOT_DIR}/src/imagegen", ROOT_DIR]:
+    if p not in sys.path:
+        sys.path.insert(0, p)
+
+from QwenImage21NVFP4Backend import QwenImage21NVFP4Backend
+
+PROMPT = (
+    "Two cute anime girls in colorful bikinis playing beach volleyball on a sunny tropical beach, "
+    "dynamic action pose, jumping to spike the ball, sharp focus, vibrant anime style, "
+    "detailed background with ocean and palm trees"
+)
+SEED = 42
+STEPS = 25
+GUIDANCE_SCALE = 1.0
+HEIGHT = 1024
+WIDTH = 1024
+
+TMP_DIR = "/home/olegk/tmp"
+ARTIFACT_DIR = "/home/olegk/.gemini/antigravity/brain/19d2cb51-1ef6-42f5-a95f-f814fe6ba720"
+HERETIC_PATH = "/home/olegk/Nikola/models/Qwen21_Text_Encoder_Heretic"
+
+
+def generate_with_backend(backend_name: str, text_encoder_path=None):
+    print(f"\n{'='*70}")
+    print(f"🚀 RUNNING GENERATION: {backend_name}")
+    print(f"Text Encoder Path: {text_encoder_path or 'Stock (Base Qwen-Image-2.1)'}")
+    print(f"{'='*70}")
+
+    backend = QwenImage21NVFP4Backend(
+        model_id="/home/olegk/Nikola/models/Qwen/Qwen-Image-2.1",
+        optimized_model_path="/home/olegk/Nikola/models/nunchaku-qwen-image-2.1/best_quality_fp4.safetensors",
+        gpu_id=0,
+        enable_tiling=True,
+        dynamic_scale_k=0.0,
+        stream_text_encoder=True,
+        text_encoder_path=text_encoder_path,
+    )
+
+    t0 = time.perf_counter()
+    pipeline, _ = backend.load()
+    t_load = time.perf_counter() - t0
+    print(f"Backend loaded in {t_load:.2f} s")
+
+    generator = torch.Generator(device="cuda:0").manual_seed(SEED)
+
+    torch.cuda.synchronize()
+    t_gen_start = time.perf_counter()
+    output = pipeline(
+        prompt=PROMPT,
+        height=HEIGHT,
+        width=WIDTH,
+        num_inference_steps=STEPS,
+        true_cfg_scale=GUIDANCE_SCALE,
+        generator=generator,
+    )
+    torch.cuda.synchronize()
+    t_gen = time.perf_counter() - t_gen_start
+    print(f"Generation completed in {t_gen:.2f} s")
+
+    img = output.images[0]
+
+    # Clean up pipeline from GPU memory
+    del pipeline
+    del backend
+    torch.cuda.empty_cache()
+
+    return img, t_gen
+
+
+def main():
+    os.makedirs(TMP_DIR, exist_ok=True)
+    os.makedirs(ARTIFACT_DIR, exist_ok=True)
+
+    print("=" * 80)
+    print("🏐 ANIME BEACH VOLLEYBALL TEXT ENCODER COMPARISON TEST")
+    print(f"Prompt: {PROMPT}")
+    print(f"Seed: {SEED} | Steps: {STEPS} | CFG: {GUIDANCE_SCALE}")
+    print("=" * 80)
+
+    # 1. Run Stock Text Encoder
+    img_stock, t_stock = generate_with_backend("STOCK (Base Qwen-Image-2.1 Text Encoder)", text_encoder_path=None)
+    stock_path = os.path.join(TMP_DIR, "qwen21_stock_beach_volleyball.png")
+    stock_art = os.path.join(ARTIFACT_DIR, "qwen21_stock_beach_volleyball.png")
+    img_stock.save(stock_path)
+    shutil.copy(stock_path, stock_art)
+    print(f"Saved stock image to: {stock_path} and artifact")
+
+    # 2. Run Heretic Text Encoder
+    img_heretic, t_heretic = generate_with_backend(
+        "HERETIC (Decensored Qwen3-VL Text Encoder)",
+        text_encoder_path=HERETIC_PATH,
+    )
+    heretic_path = os.path.join(TMP_DIR, "qwen21_heretic_beach_volleyball.png")
+    heretic_art = os.path.join(ARTIFACT_DIR, "qwen21_heretic_beach_volleyball.png")
+    img_heretic.save(heretic_path)
+    shutil.copy(heretic_path, heretic_art)
+    print(f"Saved heretic image to: {heretic_path} and artifact")
+
+    print("\n" + "=" * 80)
+    print("🎉 COMPARISON TEST COMPLETED SUCCESSFULLY!")
+    print(f"Stock Image:   {stock_path} ({t_stock:.2f} s)")
+    print(f"Heretic Image: {heretic_path} ({t_heretic:.2f} s)")
+    print("=" * 80)
+
+
+if __name__ == "__main__":
+    main()

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "5.3.0.dev0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba4cadb5ccea2f11dcbb7facce01cb35a76c9a2d5553b34f681b2e16227a1f02
+size 16289680712

preprocessor_config.json

@@ -0,0 +1,39 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "disable_grouping": null,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_pad": null,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "Qwen2VLImageProcessorFast",
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "input_data_format": null,
+  "max_pixels": null,
+  "merge_size": 2,
+  "min_pixels": null,
+  "pad_size": null,
+  "patch_size": 16,
+  "processor_class": "Qwen3VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "return_tensors": null,
+  "size": {
+    "longest_edge": 16777216,
+    "shortest_edge": 65536
+  },
+  "temporal_patch_size": 2
+}

processor/added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

processor/chat_template.jinja

@@ -0,0 +1,120 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' }}
+        {%- if messages[0].content is string %}
+            {{- messages[0].content }}
+        {%- else %}
+            {%- for content in messages[0].content %}
+                {%- if 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- for message in messages %}
+    {%- if message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content_item in message.content %}
+                {%- if 'text' in content_item %}
+                    {{- content_item.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and message.content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {%- if message.content is string %}
+            {{- message.content }}
+        {%- else %}
+            {%- for content in message.content %}
+                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}
+                    <|vision_start|><|image_pad|><|vision_end|>
+                {%- elif content.type == 'video' or 'video' in content %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}
+                    <|vision_start|><|video_pad|><|vision_end|>
+                {%- elif 'text' in content %}
+                    {{- content.text }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

processor/preprocessor_config.json

@@ -0,0 +1,39 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "disable_grouping": null,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_pad": null,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "Qwen2VLImageProcessorFast",
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "input_data_format": null,
+  "max_pixels": null,
+  "merge_size": 2,
+  "min_pixels": null,
+  "pad_size": null,
+  "patch_size": 16,
+  "processor_class": "Qwen3VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "return_tensors": null,
+  "size": {
+    "longest_edge": 16777216,
+    "shortest_edge": 65536
+  },
+  "temporal_patch_size": 2
+}

processor/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

processor/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4
+size 11422654

processor/tokenizer_config.json

@@ -0,0 +1,240 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 262144,
+  "pad_token": "<|endoftext|>",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

processor/video_preprocessor_config.json

@@ -0,0 +1,41 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "do_sample_frames": true,
+  "fps": 2,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "input_data_format": null,
+  "max_frames": 768,
+  "merge_size": 2,
+  "min_frames": 4,
+  "num_frames": null,
+  "pad_size": null,
+  "patch_size": 16,
+  "processor_class": "Qwen3VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "return_metadata": false,
+  "size": {
+    "longest_edge": 25165824,
+    "shortest_edge": 4096
+  },
+  "temporal_patch_size": 2,
+  "video_metadata": null,
+  "video_processor_type": "Qwen3VLVideoProcessor"
+}

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

text_encoder/config.json

@@ -0,0 +1,65 @@
+{
+  "architectures": [
+    "Qwen3VLForConditionalGeneration"
+  ],
+  "dtype": "bfloat16",
+  "image_token_id": 151655,
+  "model_type": "qwen3_vl",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 151643,
+    "dtype": "bfloat16",
+    "eos_token_id": 151645,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 4096,
+    "initializer_range": 0.02,
+    "intermediate_size": 12288,
+    "max_position_embeddings": 262144,
+    "model_type": "qwen3_vl_text",
+    "num_attention_heads": 32,
+    "num_hidden_layers": 36,
+    "num_key_value_heads": 8,
+    "pad_token_id": null,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        24,
+        20,
+        20
+      ],
+      "rope_theta": 5000000,
+      "rope_type": "default"
+    },
+    "use_cache": true,
+    "vocab_size": 151936
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.3.0.dev0",
+  "video_token_id": 151656,
+  "vision_config": {
+    "deepstack_visual_indexes": [
+      8,
+      16,
+      24
+    ],
+    "depth": 27,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4304,
+    "model_type": "qwen3_vl",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 4096,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 151653,
+  "vision_start_token_id": 151652
+}

text_encoder/generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 151643,
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "5.3.0.dev0"
+}

text_encoder/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba4cadb5ccea2f11dcbb7facce01cb35a76c9a2d5553b34f681b2e16227a1f02
+size 16289680712

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4
+size 11422654

tokenizer_config.json

@@ -0,0 +1,240 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 262144,
+  "pad_token": "<|endoftext|>",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

video_preprocessor_config.json

@@ -0,0 +1,41 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "do_sample_frames": true,
+  "fps": 2,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "input_data_format": null,
+  "max_frames": 768,
+  "merge_size": 2,
+  "min_frames": 4,
+  "num_frames": null,
+  "pad_size": null,
+  "patch_size": 16,
+  "processor_class": "Qwen3VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "return_metadata": false,
+  "size": {
+    "longest_edge": 25165824,
+    "shortest_edge": 4096
+  },
+  "temporal_patch_size": 2,
+  "video_metadata": null,
+  "video_processor_type": "Qwen3VLVideoProcessor"
+}