library_name: mlx
tags:
- mlx
- quantized
- mixed-precision
- qwen3.5
- moe
license: other
license_name: polyform-noncommercial
base_model: Qwen/Qwen3.5-35B-A3B
base_model_relation: quantized
pipeline_tag: text-generation
Qwen3.5-35B-A3B — 12.5GB (MLX)
Mixed-precision quantized version of Qwen/Qwen3.5-35B-A3B optimised by baa.ai using a proprietary Black Sheep AI method.
Per-tensor bit-width allocation via advanced sensitivity analysis and budget-constrained optimisation — no calibration data required.
Metrics
| Metric | Value |
|---|---|
| Size on disk | 12.54 GB |
| Average bits (LLM body) | 2.35 |
| Bit distribution (LLM body) | 78% 2-bit, 18% 3-bit, 0% 4-bit, 2% 5-bit, 2% 6-bit, 1% 8-bit, 0% 16-bit |
| Embeddings + lm_head | 4-bit |
| WikiText-2 PPL (median, 256×2048, seed 42) | 7.355 |
| WikiText-2 PPL (mean) | 7.308 |
Usage
from mlx_lm import load, generate
model, tokenizer = load("baa-ai/Qwen3.5-35B-A3B-RAM-12.5GB-MLX")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
print(response)
For chat applications, apply the chat template:
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "Write a Python function that reverses a string."}],
tokenize=False,
add_generation_prompt=True,
enable_thinking=False,
)
response = generate(model, tokenizer, prompt=prompt, max_tokens=2048)
Hardware
Apple Silicon Mac with ~16 GB unified memory recommended for inference.
Variants
Other size points in this collection:
Quantized by baa.ai
Black Sheep AI Products
Shepherd — Private AI deployment platform that shrinks frontier models by 50-60% through RAM compression, enabling enterprises to run sophisticated AI on single GPU instances or Apple Silicon hardware. Deploy in your VPC with zero data leaving your infrastructure. Includes CI/CD pipeline integration, fleet deployment across Apple Silicon clusters, air-gapped and sovereign deployment support, and multi-format export (MLX, GGUF). Annual cloud costs from ~$2,700 — or run on a Mac Studio for electricity only.
Watchman — Capability audit and governance platform for compressed AI models. Know exactly what your quantized model can do before it goes live. Watchman predicts which capabilities survive compression in minutes — replacing weeks of benchmarking. Includes compliance-ready reporting for regulated industries, quality valley warnings for counterproductive memory allocations, instant regression diagnosis tracing issues to specific tensors, and 22 adversarial security probes scanning for injection, leakage, hallucination, and code vulnerabilities.
Learn more at baa.ai — Sovereign AI.