genevera/Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit

Qwen3.6-35B-Abliterated-Heretic — 4-bit AWQ Quantized Multimodal MoE

A 4-bit AWQ-quantized multimodal Mixture-of-Experts (MoE) model derived from Youssofal/Qwen3.6-35B-A3B-Abliterated-Heretic-BF16. This is a quantized variant of Youssofal's Abliterated-Heretic-Uncensored model — an unrestricted, uncensored iteration of the Qwen3.6-35B family — compressed to 4-bit via Activation-aware Weight Quantization for deployment-friendly inference.

Model Type & Architecture

Property	Value
Architecture	Qwen3_5MoeForConditionalGeneration
Model Family	Qwen3.6-35B Abliterated-Heretic-Uncensored (quantized)
Total Parameters	~36.2B
Text Hidden Size	2,048
Vision Hidden Size	1,152
Text Layers	40 (hybrid: 3 linear + 1 full attention repeating)
Attention	Grouped Query Attention (GQA), 16 query heads, 2 KV heads (ratio 8:1)
Head Dimension	256
Vocabulary Size	248,320
Max Context Length	262,144 tokens
MoE Experts	256 per layer, 8 activated per token
MoE Intermediate Size	512
RoPE Theta	10,000,000 (long-context optimized)
RoPE	Interleaved multi-RoPE sections [11, 11, 10] for multimodal
Activation	SiLU (text), GELU-Tanh (vision)
Transformers Version	5.6.0.dev0

Vision Encoder

Property	Value
Layers	27
Hidden Size	1,152
Patches	16×16
Temporal Patch Size	2
Spatial Merge Size	2
Output Dimension	2,048 (matches text dimension)
Max Frames	768
FPS (video)	2

Quantization Details

Property	Value
Method	AWQ (Activation-aware Weight Quantization)
Bits	4-bit signed int
Group Size	128
Provider	auto-round v0.10.2
Quantization Target	model.language_model.layers
Preserved (full precision)	lm_head, all shared_expert_gate layers (39), vision merger (linear_fc1, linear_fc2), entire vision encoder (model.visual.blocks)
Symmetric	Yes
Zero Point	False
Version	gemm

The quantization strategy preserves critical components in full precision — including the language model head, all shared expert gating mechanisms across 40 layers, the vision-language merger, and the entire vision encoder — while quantizing only the language model layer weights. This hybrid approach maintains output quality while achieving significant memory reduction.

Multimodal Capabilities

This model supports multimodal inputs including:

• Text: Full text generation, understanding, and chat
• Images: Image-to-text via <|image|> tokens (ID: 248056)
• Video: Video understanding with up to 768 frames at 2 FPS via <|video|> tokens (ID: 248057)
• Audio: Audio token support (BOS/EOS defined)
• Tool Calling: Full function/tool calling with XML-style tool call formatting
• Reasoning: Chain-of-thought with <think>/</think> blocks

Chat Template

The model uses a sophisticated multimodal chat template with:

• Full tool/function calling support with XML-style formatting
• Reasoning block support (<thinking> / </think>)
• Image/video token insertion with optional numbering
• System message handling
• Multi-turn conversation with tool results

See chat_template.jinja for the complete template.

Tokenizer

Property	Value
Class	TokenizersBackend (HuggingFace tokenizers)
Processor	Qwen3VLProcessor
BOS / PAD	248044 (<bos>)
EOS	248046 (</s>)
Image Token	248056 (`<
Video Token	248057 (`<
Vision BOS	248053 (`<
Vision EOS	248054 (`<
Audio BOS/EOS	Defined
Max Length	262,144 tokens

See tokenizer_config.json for full configuration.

Generation Parameters

Property	Value
Sampling	Enabled (do_sample: true)
Temperature	1.0
Top-K	20
Top-P	0.95
EOS Tokens	[248046, 248044]

Quick Start — Inference Examples

Basic Text Generation

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "./Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit"

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

# Simple text generation
messages = [
    {"role": "user", "content": "Hello, can you tell me a story?"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Multimodal (Image) Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
from PIL import Image

model_id = "./Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

# Load image
image = Image.open("path/to/image.jpg").convert("RGB")

# Multimodal conversation
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": "Describe this image in detail."}
        ]
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

# Note: For actual image processing, use Qwen3VLProcessor
# from transformers import Qwen3VLProcessor
# processor = Qwen3VLProcessor.from_pretrained(model_id)

inputs = processor(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
result = processor.decode(outputs[0], skip_special_tokens=True)
print(result)

Chat with Tool Calling

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "./Qwen3.6-35B-A3B-Abliterated-Heretic-AWQ-4bit"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

# Define tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name or coordinates"}
                },
                "required": ["location"]
            }
        }
    }
]

messages = [
    {"role": "system", "content": "You are a helpful assistant with weather lookup capabilities."},
    {"role": "user", "content": "What's the weather in Tokyo?"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    tools=tools,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)
# The model may output a tool call in XML format:
# <function=get_weather>