---
license: apache-2.0
base_model:
  - Vortex5/Chimera-X-26B-A4B
base_model_relation: quantized
tags:
  - openvino
  - int4
  - awq
  - intel-arc
  - gemma4
  - roleplay
  - conversational
pipeline_tag: image-text-to-text
---

> **OpenVINO INT4 AWQ** - asymmetric 4-bit weight compression, verified group size 64.


# Chimera-X 26B-A4B — OpenVINO INT4 AWQ

OpenVINO INT4 conversion of [Vortex5/Chimera-X-26B-A4B](https://huggingface.co/Vortex5/Chimera-X-26B-A4B), optimized for fast local inference on Intel Arc GPUs.

This repository is a quantized deployment artifact. The original fine-tune and merge work belongs to Vortex5 and the creators of its component models; this conversion does not claim authorship of that work.

## Model and credits

Chimera-X is a Gemma 4 26B-A4B mixture-of-experts model intended for roleplay, creative writing, storytelling, and brainstorming. The source model combines work from:

- [Darkhn/Gemma-4-26B-A4B-Animus-V14.1-FFT](https://huggingface.co/Darkhn/Gemma-4-26B-A4B-Animus-V14.1-FFT)
- [Gryphe/Pantheon-Reasoning-26B-A4B-1.1](https://huggingface.co/Gryphe/Pantheon-Reasoning-26B-A4B-1.1)
- [Vortex5/G4-Moonlight-Dusk-26B-A4B](https://huggingface.co/Vortex5/G4-Moonlight-Dusk-26B-A4B)
- [zerofata/G4-MeroMero-26B-A4B](https://huggingface.co/zerofata/G4-MeroMero-26B-A4B)

See the [source model card](https://huggingface.co/Vortex5/Chimera-X-26B-A4B) for its full description, merge history, intended use, and original credits.

## OpenVINO conversion

- Weight format: asymmetric INT4
- Compression: AWQ, group size 64, ratio 1.0
- MoE routing: router layers excluded from INT4 weight compression
- Graph: multimodal Gemma 4 OpenVINO IR (`VLMPipeline`)
- RoPE lookup-table optimization: 131,072 positions, clamped at index 131,071
- Source model context declaration: 262,144 tokens
- Serving window validated on this build: 16K tokens
- 32K serving is experimental and depends on KV-cache budget and runtime configuration

The 131,072-position RoPE table is the hard ceiling of this particular graph, even though the source configuration declares a larger window.

## Measured Intel Arc performance

Single-request measurements on one Intel Arc Pro B70 using the custom OpenVINO 2026.4 PA/XMX runtime used by Arcanaeum:

| Test | Result |
|---|---:|
| Prompt processing | about 4,967 tokens/s |
| Short decode | about 102.8 tokens/s |
| Decode with about 6.7K tokens of context | about 87.7 tokens/s |
| First token with about 6.7K tokens of context | about 1.45 seconds |

Results are workload- and runtime-dependent. These numbers are measurements of this deployment, not guarantees for every OpenVINO build or Intel GPU.

## Inference with OpenVINO GenAI

Install a recent OpenVINO GenAI build and Hugging Face Hub client:

```bash
pip install -U openvino-genai huggingface-hub
```

Download the full repository and point `VLMPipeline` at the local snapshot:

```python
import openvino_genai as ov_genai
from huggingface_hub import snapshot_download

model_dir = snapshot_download("Wondernutts/Chimera-X-26B-A4B-int4-ov")

pipe = ov_genai.VLMPipeline(
    model_dir,
    "GPU",
    DYNAMIC_QUANTIZATION_GROUP_SIZE=128,
)

config = ov_genai.GenerationConfig()
config.max_new_tokens = 512
config.do_sample = True
config.temperature = 0.9
config.top_p = 0.95
config.repetition_penalty = 1.2
config.apply_chat_template = False

prompt = (
    "<bos>"
    "<|turn>system\nYou are a vivid, consistent roleplay partner.<turn|>\n"
    "<|turn>user\nWrite a short scene in a moonlit inn.<turn|>\n"
    "<|turn>model\n"
    "<|channel>thought\n<channel|>"
)

result = pipe.generate(prompt, generation_config=config)
print(result)
```

The final empty thought channel disables visible reasoning for direct roleplay responses. To use the model's reasoning mode, remove that final pre-closed channel and budget additional generation tokens. Applications should hide internal reasoning from end users.

The model artifact includes vision embeddings. Native image input requires the multimodal `VLMPipeline` API and an OpenVINO GenAI build compatible with this Gemma 4 export. Text inference is the path benchmarked above.

## Notes

- This is not a Transformers checkpoint; use OpenVINO/OpenVINO GenAI rather than `AutoModelForCausalLM`.
- The model is uncensored/creative by design. Deployers remain responsible for prompts, outputs, applicable law, and platform policy.
- License: Apache-2.0, inherited from the source model. Review the upstream card and component licenses before redistribution or commercial deployment.