Branded model card with banner

README.md

@@ -1,23 +1,75 @@
 ---
 license: apache-2.0
-pipeline_tag: image-text-to-text
 library_name: transformers
+pipeline_tag: image-text-to-text
+language:
+- en
+- zh
+- ja
+- ko
+- de
+- fr
+- es
+- pt
+- ru
 tags:
+- qwen
+- qwen3
+- qwen3-vl
+- vision-language
+- vlm
+- multimodal
+- image-text-to-text
+- video
+- ocr
+- grounding
+- agent
 - heretic
 - uncensored
 - decensored
 - abliterated
 - reproducible
+- gguf
+- roleplay
+- local-llm
+- consumer-gpu
+base_model: Qwen/Qwen3-VL-2B-Instruct
 ---
-# This is a decensored version of [Qwen/Qwen3-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct), made using [Heretic](https://heretic-project.org) v1.4.0
 
-> [!TIP]
-> **This model is reproducible!**
->
-> See the [README](reproduce/README.md) in the `reproduce` directory for more information.
+# Qwen3-VL-2B-Instruct-heretic
+
+<div align="center">
+  <img src="https://photu.kashyalabanavli.site/racer-is-op.png" alt="RACER IS OP" width="100%">
+</div>
+
+<br>
+
+A decensored variant of [Qwen/Qwen3-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct), produced with [Heretic](https://github.com/p-e-w/heretic) v1.4.0 (directional ablation / "abliteration"). Qwen3-VL is the vision-language branch of the Qwen3 family — image, video, and OCR input with spatial grounding — and refusal behaviour is suppressed here via targeted weight edits to the attention output and MLP down-projections rather than fine-tuning, so the vision tower and grounding behaviour are left largely intact.
+
+**Who this is for:** developers who want uncensored vision-language understanding on hardware they already own. At 2B this is the smallest model in the collection that still sees images, and it answers directly instead of declining — refusals drop from 97/100 to 5/100. Best for local OCR, screenshot and document Q&A, UI-to-code from a reference image, and vision agents on CPU or a low-VRAM GPU. If you need text-only at this size, [Qwen2.5-3B-Instruct-heretic](https://huggingface.co/saidutta69/Qwen2.5-3B-Instruct-heretic) is the denser sibling.
+
+<!-- racer-gpu-matrix -->
+## Runs on your gaming PC
+
+Full GGUF ladder included - pick the quant that fits your card:
+
+| Your GPU | Recommended quant | Weights |
+| :--- | :--- | :--- |
+| RTX 4090 / 5090 (24 GB) | Q8_0 | 2.05 GB |
+| RTX 4080 / 5080 / 4060 Ti 16G (16 GB) | Q6_K | 1.62 GB |
+| RTX 3060 / 4070 / 5070 (12 GB) | Q5_K_M | 1.45 GB |
+| RTX 4060 / 3070 (8 GB) | Q4_K_M | 1.26 GB |
+| GTX 1660 Super / 2060 / 3050 laptop (6 GB) | IQ4_XS | 1.15 GB |
+| CPU-only / Apple Silicon | Q4_K_M | fits in system RAM |
+
+Weights only, at this model's native 2B size; add ~1 GB per 32K of context. Multimodal context
+carries image tokens too, so budget more KV cache than a text-only model of the same size.
+OOM? Drop one quant level. Headroom to spare? Go one up.
 
 ## Abliteration parameters
 
+Trial 74 of a 200-trial Heretic run (seed `1736045380`). `direction_index` was selected per layer.
+
 | Parameter | Value |
 | :-------- | :---: |
 | **direction_index** | per layer |
@@ -37,192 +89,109 @@ tags:
 | **KL divergence** | 0.0528 | 0 *(by definition)* |
 | **Refusals** | 5/100 | 97/100 |
 
------
-
-<a href="https://huggingface.co/spaces/akhaliq/Qwen3-VL-2B-Instruct" target="_blank" style="margin: 2px;">
-    <img alt="Demo" src="https://img.shields.io/badge/Demo-536af5" style="display: inline-block; vertical-align: middle;"/>
-</a>
-
-
-# Qwen3-VL-2B-Instruct
-
-
-Meet Qwen3-VL — the most powerful vision-language model in the Qwen series to date.
-
-This generation delivers comprehensive upgrades across the board: superior text understanding & generation, deeper visual perception & reasoning, extended context length, enhanced spatial and video dynamics comprehension, and stronger agent interaction capabilities.
-
-Available in Dense and MoE architectures that scale from edge to cloud, with Instruct and reasoning‑enhanced Thinking editions for flexible, on‑demand deployment.
-
-
-#### Key Enhancements:
-
-* **Visual Agent**: Operates PC/mobile GUIs—recognizes elements, understands functions, invokes tools, completes tasks.
-
-* **Visual Coding Boost**: Generates Draw.io/HTML/CSS/JS from images/videos.
+KL divergence of 0.0528 with refusals at 5/100 is one of the stronger trade-offs in this batch:
+the harmful evaluation set drops from 97/100 to 5/100 while the vision tower and grounding
+behaviour stay close to the original. Small models concentrate refusal behaviour into fewer
+directions, which is why a 2B edit can land this cleanly.
 
-* **Advanced Spatial Perception**: Judges object positions, viewpoints, and occlusions; provides stronger 2D grounding and enables 3D grounding for spatial reasoning and embodied AI.
+## Why abliteration instead of fine-tuning
 
-* **Long Context & Video Understanding**: Native 256K context, expandable to 1M; handles books and hours-long video with full recall and second-level indexing.
+Fine-tuning a "helpful" persona on top of RLHF'd refusals fights the base model's training and tends to degrade coherence. Abliteration instead finds and edits the specific weight directions responsible for refusal, leaving the rest of the network (and its capabilities) untouched. See the [Heretic repo](https://github.com/p-e-w/heretic) and the [original abliteration writeup](https://huggingface.co/blog/mlabonne/abliteration) for the mechanism.
 
-* **Enhanced Multimodal Reasoning**: Excels in STEM/Math—causal analysis and logical, evidence-based answers.
+> Made with ❤️ by **RACER IS OP** — follow for more uncensored models
 
-* **Upgraded Visual Recognition**: Broader, higher-quality pretraining is able to “recognize everything”—celebrities, anime, products, landmarks, flora/fauna, etc.
+## Files
 
-* **Expanded OCR**: Supports 32 languages (up from 19); robust in low light, blur, and tilt; better with rare/ancient characters and jargon; improved long-document structure parsing.
+### Safetensors
 
-* **Text Understanding on par with pure LLMs**: Seamless text–vision fusion for lossless, unified comprehension.
+| File | Size |
+|---|---|
+| `model.safetensors` | 3.96 GB |
 
+BF16, ~2B. The `reproduce/` directory carries the full Heretic recipe -
+`config.toml`, `requirements.txt`, the Optuna study journal, and SHA-256 sums - so this exact
+model can be regenerated bit-for-bit. Reproduce it with `heretic --reproduce reproduce/reproduce.json`.
 
-#### Model Architecture Updates:
+### GGUF quantizations
+Full quantization set (14 quants + F16) produced with [llama.cpp](https://github.com/ggml-org/llama.cpp).
 
-<p align="center">
-    <img src="https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/qwen3vl_arc.jpg" width="80%"/>
-<p>
+<!-- racer-gguf-table -->
 
+Qwen3-VL architecture (`qwen3vl`) with its vision encoder - loads natively in llama.cpp / LM Studio / Jan.
 
-1. **Interleaved-MRoPE**: Full‑frequency allocation over time, width, and height via robust positional embeddings, enhancing long‑horizon video reasoning.
+Run `llama serve -hf saidutta69/Qwen3-VL-2B-Instruct-heretic` to pull the default quant.
 
-2. **DeepStack**: Fuses multi‑level ViT features to capture fine‑grained details and sharpen image–text alignment.
-
-3. **Text–Timestamp Alignment:** Moves beyond T‑RoPE to precise, timestamp‑grounded event localization for stronger video temporal modeling.
-
-This is the weight repository for Qwen3-VL-2B-Instruct.
-
-
----
+## Quickstart
 
-## Model Performance
+```bash
+# llama.cpp - the vision projector ships with the GGUF
+llama serve -hf saidutta69/Qwen3-VL-2B-Instruct-heretic
+```
 
-**Multimodal performance**
+```python
+# transformers
+from transformers import AutoProcessor, AutoModelForImageTextToText
+
+model_name = "saidutta69/Qwen3-VL-2B-Instruct-heretic"
+model = AutoModelForImageTextToText.from_pretrained(model_name, dtype="auto", device_map="auto")
+processor = AutoProcessor.from_pretrained(model_name)
+
+messages = [{"role": "user", "content": [
+    {"type": "image", "image": "https://example.com/screenshot.png"},
+    {"type": "text", "text": "Transcribe all text in this image and describe the layout."},
+]}]
+inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,
+                                        return_dict=True, return_tensors="pt").to(model.device)
+out = model.generate(**inputs, max_new_tokens=1024)
+print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
 
-![](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/qwen3vl_2b_32b_vl_instruct.jpg)
+### Grounding and OCR
 
-**Pure text performance**
-![](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3-VL/qwen3vl_2b_32b_text_instruct.jpg)
+Qwen3-VL emits bounding boxes in a normalised coordinate space, so you can ask for element
+positions directly and get coordinates back rather than prose descriptions:
 
-## Quickstart
+```python
+messages = [{"role": "user", "content": [
+    {"type": "image", "image": "https://example.com/page.jpg"},
+    {"type": "text", "text": "Give the bounding box of the submit button."},
+]}]
+```
 
-Below, we provide simple examples to show how to use Qwen3-VL with 🤖 ModelScope and 🤗 Transformers.
+This is the practical path for UI automation and for building training data from screenshots.
 
-The code of Qwen3-VL has been in the latest Hugging Face transformers and we advise you to build from source with command:
-```
-pip install git+https://github.com/huggingface/transformers
-# pip install transformers==4.57.0 # currently, V4.57.0 is not released
-```
+## Model details
 
-### Using 🤗 Transformers to Chat
+| | |
+|---|---|
+| **Architecture** | `Qwen3VLForConditionalGeneration` (vision-language decoder) |
+| **Parameters** | ~2B |
+| **Layers / heads** | 28 layers, 16 attention heads, 8 KV heads, head dim 128 |
+| **Hidden / intermediate** | 2048 / 6144 |
+| **Position embedding** | mRoPE (interleaved, sections 24/20/20), theta = 5,000,000 |
+| **Context length** | 262,144 |
+| **Vocab** | 151,936 |
+| **Precision** | bfloat16 |
+| **Modalities** | Text and image/video in, text out |
+| **Base model** | [Qwen/Qwen3-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct) |
 
-Here we show a code snippet to show how to use the chat model with `transformers`:
+## About Qwen3-VL
 
-```python
-from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
-
-# default: Load the model on the available device(s)
-model = Qwen3VLForConditionalGeneration.from_pretrained(
-    "Qwen/Qwen3-VL-2B-Instruct", dtype="auto", device_map="auto"
-)
-
-# We recommend enabling flash_attention_2 for better acceleration and memory saving, especially in multi-image and video scenarios.
-# model = Qwen3VLForConditionalGeneration.from_pretrained(
-#     "Qwen/Qwen3-VL-2B-Instruct",
-#     dtype=torch.bfloat16,
-#     attn_implementation="flash_attention_2",
-#     device_map="auto",
-# )
-
-processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-2B-Instruct")
-
-messages = [
-    {
-        "role": "user",
-        "content": [
-            {
-                "type": "image",
-                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
-            },
-            {"type": "text", "text": "Describe this image."},
-        ],
-    }
-]
-
-# Preparation for inference
-inputs = processor.apply_chat_template(
-    messages,
-    tokenize=True,
-    add_generation_prompt=True,
-    return_dict=True,
-    return_tensors="pt"
-)
-inputs = inputs.to(model.device)
-
-# Inference: Generation of the output
-generated_ids = model.generate(**inputs, max_new_tokens=128)
-generated_ids_trimmed = [
-    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
-]
-output_text = processor.batch_decode(
-    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
-)
-print(output_text)
-```
+Qwen3-VL is the vision-language branch of the Qwen series, available in dense and MoE sizes with
+Instruct and Thinking editions. This generation brings deeper visual perception and reasoning,
+extended context, stronger spatial and video understanding, and improved agent interaction — OCR,
+grounding, and video temporal reasoning are the headline capabilities over the earlier Qwen2.5-VL.
 
-### Generation Hyperparameters
-#### VL
-```bash
-export greedy='false'
-export top_p=0.8
-export top_k=20
-export temperature=0.7
-export repetition_penalty=1.0
-export presence_penalty=1.5
-export out_seq_length=16384
-```
+## Responsible use
 
-#### Text
-```bash
-export greedy='false'
-export top_p=1.0
-export top_k=40
-export repetition_penalty=1.0
-export presence_penalty=2.0
-export temperature=1.0
-export out_seq_length=32768
-```
+Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it shouldn't. There is no safety filtering layered on top. You are responsible for how you deploy it — don't put this behind an unmoderated public-facing endpoint serving third parties. It inherits Qwen3-VL's factual limitations and biases; abliteration removes refusal directions, it doesn't add capability or judgment.
 
+## License
 
-## Citation
+Inherits the [`apache-2.0`](https://www.apache.org/licenses/LICENSE-2.0) license from the base model.
 
-If you find our work helpful, feel free to give us a cite.
+## Related
 
-```
-@misc{qwen3technicalreport,
-      title={Qwen3 Technical Report}, 
-      author={Qwen Team},
-      year={2025},
-      eprint={2505.09388},
-      archivePrefix={arXiv},
-      primaryClass={cs.CL},
-      url={https://arxiv.org/abs/2505.09388}, 
-}
-
-@article{Qwen2.5-VL,
-  title={Qwen2.5-VL Technical Report},
-  author={Bai, Shuai and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Song, Sibo and Dang, Kai and Wang, Peng and Wang, Shijie and Tang, Jun and Zhong, Humen and Zhu, Yuanzhi and Yang, Mingkun and Li, Zhaohai and Wan, Jianqiang and Wang, Pengfei and Ding, Wei and Fu, Zheren and Xu, Yiheng and Ye, Jiabo and Zhang, Xi and Xie, Tianbao and Cheng, Zesen and Zhang, Hang and Yang, Zhibo and Xu, Haiyang and Lin, Junyang},
-  journal={arXiv preprint arXiv:2502.13923},
-  year={2025}
-}
-
-@article{Qwen2VL,
-  title={Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution},
-  author={Wang, Peng and Bai, Shuai and Tan, Sinan and Wang, Shijie and Fan, Zhihao and Bai, Jinze and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Fan, Yang and Dang, Kai and Du, Mengfei and Ren, Xuancheng and Men, Rui and Liu, Dayiheng and Zhou, Chang and Zhou, Jingren and Lin, Junyang},
-  journal={arXiv preprint arXiv:2409.12191},
-  year={2024}
-}
-
-@article{Qwen-VL,
-  title={Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond},
-  author={Bai, Jinze and Bai, Shuai and Yang, Shusheng and Wang, Shijie and Tan, Sinan and Wang, Peng and Lin, Junyang and Zhou, Chang and Zhou, Jingren},
-  journal={arXiv preprint arXiv:2308.12966},
-  year={2023}
-}
-```
+- [Qwen/Qwen3-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct) — the base model
+- [saidutta69/gemma-4-E4B-it-heretic](https://huggingface.co/saidutta69/gemma-4-E4B-it-heretic) — the larger multimodal sibling in this collection
+- [RACER IS OP — Heretic Models — full collection](https://huggingface.co/collections/saidutta69/racer-is-op-heretic-models)