Wondernutts/Chimera-X-26B-A4B-int4-ov

OpenVINO INT4 AWQ - asymmetric 4-bit weight compression, verified group size 64.

Chimera-X 26B-A4B — OpenVINO INT4 AWQ

OpenVINO INT4 conversion of Vortex5/Chimera-X-26B-A4B, optimized for fast local inference on Intel Arc GPUs.

This repository is a quantized deployment artifact. The original fine-tune and merge work belongs to Vortex5 and the creators of its component models; this conversion does not claim authorship of that work.

Model and credits

Chimera-X is a Gemma 4 26B-A4B mixture-of-experts model intended for roleplay, creative writing, storytelling, and brainstorming. The source model combines work from:

• Darkhn/Gemma-4-26B-A4B-Animus-V14.1-FFT
• Gryphe/Pantheon-Reasoning-26B-A4B-1.1
• Vortex5/G4-Moonlight-Dusk-26B-A4B
• zerofata/G4-MeroMero-26B-A4B

See the source model card for its full description, merge history, intended use, and original credits.

OpenVINO conversion

• Weight format: asymmetric INT4
• Compression: AWQ, group size 64, ratio 1.0
• MoE routing: router layers excluded from INT4 weight compression
• Graph: multimodal Gemma 4 OpenVINO IR (VLMPipeline)
• RoPE lookup-table optimization: 131,072 positions, clamped at index 131,071
• Source model context declaration: 262,144 tokens
• Serving window validated on this build: 16K tokens
• 32K serving is experimental and depends on KV-cache budget and runtime configuration

The 131,072-position RoPE table is the hard ceiling of this particular graph, even though the source configuration declares a larger window.

Measured Intel Arc performance

Single-request measurements on one Intel Arc Pro B70 using the custom OpenVINO 2026.4 PA/XMX runtime used by Arcanaeum:

Test	Result
Prompt processing	about 4,967 tokens/s
Short decode	about 102.8 tokens/s
Decode with about 6.7K tokens of context	about 87.7 tokens/s
First token with about 6.7K tokens of context	about 1.45 seconds

Results are workload- and runtime-dependent. These numbers are measurements of this deployment, not guarantees for every OpenVINO build or Intel GPU.

Inference with OpenVINO GenAI

Install a recent OpenVINO GenAI build and Hugging Face Hub client:

pip install -U openvino-genai huggingface-hub

Download the full repository and point VLMPipeline at the local snapshot:

import openvino_genai as ov_genai
from huggingface_hub import snapshot_download

model_dir = snapshot_download("Wondernutts/Chimera-X-26B-A4B-int4-ov")

pipe = ov_genai.VLMPipeline(
    model_dir,
    "GPU",
    DYNAMIC_QUANTIZATION_GROUP_SIZE=128,
)

config = ov_genai.GenerationConfig()
config.max_new_tokens = 512
config.do_sample = True
config.temperature = 0.9
config.top_p = 0.95
config.repetition_penalty = 1.2
config.apply_chat_template = False

prompt = (
    "<bos>"
    "<|turn>system\nYou are a vivid, consistent roleplay partner.<turn|>\n"
    "<|turn>user\nWrite a short scene in a moonlit inn.<turn|>\n"
    "<|turn>model\n"
    "<|channel>thought\n<channel|>"
)

result = pipe.generate(prompt, generation_config=config)
print(result)

The final empty thought channel disables visible reasoning for direct roleplay responses. To use the model's reasoning mode, remove that final pre-closed channel and budget additional generation tokens. Applications should hide internal reasoning from end users.

The model artifact includes vision embeddings. Native image input requires the multimodal VLMPipeline API and an OpenVINO GenAI build compatible with this Gemma 4 export. Text inference is the path benchmarked above.

Notes

• This is not a Transformers checkpoint; use OpenVINO/OpenVINO GenAI rather than AutoModelForCausalLM.
• The model is uncensored/creative by design. Deployers remain responsible for prompts, outputs, applicable law, and platform policy.
• License: Apache-2.0, inherited from the source model. Review the upstream card and component licenses before redistribution or commercial deployment.