AdvancedDataIntelligence/adi-qwen2.5-vl-7b-ablit-glm5.2-GGUF

adi-qwen2.5-vl-7b-ablit-glm5.2

Part of the ADI (Advanced Data Intelligence) model line — ADI Qwen series.

An uncensored, vision-capable, fully local model that reasons and answers like a frontier teacher. Built by distilling glm-5.2 general-knowledge responses into an abliterated Qwen2.5-VL-7B student with a light 4-bit QLoRA fine-tune, then merged, converted, and quantized to GGUF. Only the language layers were tuned — the base's vision tower is preserved and shipped as a companion projector — and the abliterated base keeps its minimal-refusal behavior, with the fine-tune kept light specifically to avoid re-aligning it.

Capabilities

Size	Context	Input	Output	Tools
4.68 GB	128K	🅣🖼️ Text + Image	Text	✅

Base model	huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated (abliterated Qwen2.5-VL-7B-Instruct)
Teacher	glm-5.2 (responses distilled, thinking disabled)
Method	Light 4-bit QLoRA SFT (rank 16, 2 epochs, language layers only) → merge → GGUF
Quantization	Q4_K_M (~4.68 GB text) + vision projector (mmproj, ~1.3 GB)
License	Apache-2.0 (inherited from Qwen2.5-VL-7B)
Context	128K (inherited from base)
Vision	Supported — multimodal (image + text → text)

Run it

Pull directly into Ollama:

ollama run hf.co/AdvancedDataIntelligence/adi-qwen2.5-vl-7b-ablit-glm5.2-GGUF:Q4_K_M

It's multimodal — pass an image to have it describe or reason over it:

ollama run adi-qwen2.5-vl-7b-ablit-glm5.2 "What's in this image? /path/to/photo.jpg"

Or download the .gguf (text) + mmproj-*.gguf (vision projector) and point any llama.cpp-based runtime at them. Both files are required for vision.

What this model is

This is a knowledge distillation: a strong teacher (glm-5.2) generated high-quality answers across a clean general-knowledge prompt set, and the abliterated Qwen2.5-VL-7B student was fine-tuned to imitate them. The result reasons and responds more like its teacher on general topics, keeps the base's uncensored character, and retains native image understanding — all while running on a single consumer GPU.

What distillation does — and doesn't do. It transfers the teacher's reasoning style and answer quality, not net-new facts. For raw factual recall, retrieval-augmented generation (RAG) is the right tool, not fine-tuning. What you get here is a 7B that structures and explains like a larger model on topics it already partly knows — without the refusal behavior of an aligned model.

Uncensored behavior — please read

This model is built on an abliterated base: the refusal direction has been suppressed, so it will attempt most requests rather than declining them. The fine-tune was intentionally kept light (2 epochs, benign-only data) to avoid re-introducing refusals. You are responsible for using it lawfully and ethically; it has weaker built-in safety guardrails than stock Qwen2.5-VL-7B-Instruct.

Training

Metric	Value
Training pairs	2,000 (deterministic subset of a 4,982-pair clean set)
Epochs	2 (kept light to preserve abliteration)
Steps	500
Final train loss	1.2618
LoRA rank / alpha	16 / 16
Trainable params	40.4M (language layers only; vision tower frozen)
Precision	4-bit QLoRA (nf4)
Peak VRAM	8.14 GB
Hardware	single RTX 5060 Ti (16 GB)
Training time	1.44 h (~10 s/step)

The seed prompts were drawn from the human-written Databricks Dolly-15k dataset (filtered to remove items requiring an attached context passage, then deduplicated). The teacher was queried with thinking disabled so the student learns clean final answers rather than chain-of-thought.

Notes for re-builders

• Distilling onto an abliterated base is a balancing act. Any SFT can nudge an abliterated model back toward refusals. Two choices kept the behavior intact: benign-only training data (the GLM-5.2 set has zero refusals to re-learn) and a light touch (LoRA rank 16, 2 epochs). Spot-check refusals before/after.
• Vision base = train language only. Load with Unsloth FastVisionModel (load_in_4bit=True) and get_peft_model(finetune_vision_layers=False, finetune_language_layers=True, ...). The vision tower rides through unchanged, so the base's mmproj is the final vision projector — reuse it, don't regenerate.
• Free the GPU before loading. An Ollama model left resident in VRAM makes the 4-bit VL load spill to CPU (ValueError: Some modules are dispatched on the CPU); ollama stop <model> first.
• GGUF conversion via streaming LoRA merge (language keys map model.language_model.* → model.*) → f16 GGUF → Q4_K_M with llama.cpp (Qwen2_5_VLForConditionalGeneration).

Serving note (Ollama vision)

On some Ollama builds the Qwen2.5-VL vision runner can degrade to blank/garbled output after several requests in a session (text is unaffected). If that happens, reload the model — ollama stop adi-qwen2.5-vl-7b-ablit-glm5.2 then re-run, or set keep_alive — for a clean vision pass. The GGUF itself is correct; this is a runtime quirk likely resolved by newer Ollama versions.

Intended use

General-purpose local assistant with image understanding for users who want a capable, private, offline-capable model with minimal refusal behavior: explanations, reasoning, visual Q&A, and creative writing. Not intended as a source of authoritative facts without retrieval, and not a substitute for your own safety review.

License

Apache-2.0, inherited from the Qwen2.5-VL-7B lineage via the abliterated base model. You are free to use, modify, and redistribute under the terms of that license. Distilled training data was generated using glm-5.2; users should review the teacher model's terms for their own use case.

---

Built at theLAB — Learning. Algorithms. Breakthroughs.