Instructions to use QuaduxIT/Qwen3.8-27B-Whitehat-W8A16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use QuaduxIT/Qwen3.8-27B-Whitehat-W8A16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="QuaduxIT/Qwen3.8-27B-Whitehat-W8A16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("QuaduxIT/Qwen3.8-27B-Whitehat-W8A16")
model = AutoModelForMultimodalLM.from_pretrained("QuaduxIT/Qwen3.8-27B-Whitehat-W8A16", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use QuaduxIT/Qwen3.8-27B-Whitehat-W8A16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "QuaduxIT/Qwen3.8-27B-Whitehat-W8A16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QuaduxIT/Qwen3.8-27B-Whitehat-W8A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/QuaduxIT/Qwen3.8-27B-Whitehat-W8A16

SGLang

How to use QuaduxIT/Qwen3.8-27B-Whitehat-W8A16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "QuaduxIT/Qwen3.8-27B-Whitehat-W8A16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QuaduxIT/Qwen3.8-27B-Whitehat-W8A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "QuaduxIT/Qwen3.8-27B-Whitehat-W8A16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "QuaduxIT/Qwen3.8-27B-Whitehat-W8A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use QuaduxIT/Qwen3.8-27B-Whitehat-W8A16 with Docker Model Runner:
```
docker model run hf.co/QuaduxIT/Qwen3.8-27B-Whitehat-W8A16
```

WalterHoffmann commited on 11 days ago

Commit

a1b1efc

verified ·

1 Parent(s): b09a537

Upload NOTICE with huggingface_hub

Browse files

Files changed (1) hide show

NOTICE +53 -0

NOTICE ADDED Viewed

	@@ -0,0 +1,53 @@

+Qwen3.8-27B-Whitehat (Quadux) — W8A16
+Copyright 2026 Quadux IT GmbH
+This product includes software and model weights developed by third parties,
+redistributed here in modified form under the Apache License, Version 2.0.
+------------------------------------------------------------------------------
+Base model
+------------------------------------------------------------------------------
+Qwen3.8-27B
+Copyright the Qwen team, Alibaba Cloud.
+Licensed under the Apache License, Version 2.0.
+Source: https://huggingface.co/Qwen/Qwen3.8-27B
+The model weights distributed here are a derivative work of Qwen3.8-27B.
+------------------------------------------------------------------------------
+Modifications by Quadux IT GmbH (Apache-2.0 Section 4(b))
+------------------------------------------------------------------------------
+The following changes were made to the base model to produce this derivative
+work:
+  * Selective behavioural fine-tuning (multimodal LoRA, merged into the
+    weights) that opens computer-/network-security and offensive-security
+    tasks (exploit development, malware and C2 analysis, reverse engineering,
+    phishing-infrastructure and awareness testing, copyright-/DRM- and other
+    legal-restriction bypass for feasibility and security research) while
+    RETAINING refusal of physically harmful content (weapons, explosives,
+    drug synthesis, CBRN, violence) and CSAM. The boundary is enforced
+    language-independently and across the text and vision (image) input paths.
+    This fine-tune is distributed at full precision in the sibling repository
+    QuaduxIT/Qwen3.8-27B-Whitehat (BF16 reference).
+  * Quantization: the language-model linears were quantized to INT8 weight-only (W8A16, 8-bit weights / 16-bit activations) using the compressed-tensors format with Marlin kernels (near-lossless). The quantization covers the language-model linear
+    layers only; the vision tower and multimodal projector are left in FP16.
+    Produced with llm-compressor (https://github.com/vllm-project/llm-compressor).
+    lm_head and embed_tokens are excluded from quantization.
+The vision tower ("visual") was frozen during fine-tuning and is unchanged
+from the base model, and is NOT quantized (kept in FP16).
+------------------------------------------------------------------------------
+Calibration data
+------------------------------------------------------------------------------
+The quantization was calibrated with the publicly available
+"calibration_datav3" text corpus (bartowski). No end-user or customer data
+was used.
+------------------------------------------------------------------------------
+This NOTICE file is provided for informational purposes only and does not
+modify the License. See the LICENSE file for the full Apache License 2.0 terms.
+Use of this model is subject to the Disclaimer and Terms of Use — see DISCLAIMER.md and the model card. / Die Nutzung unterliegt dem Haftungsausschluss — siehe DISCLAIMER.md und die Model-Card.