Update README.md

README.md

@@ -1,3 +1,158 @@
 ---
 license: apache-2.0
+base_model:
+  - ProCreations/grug-27b
+library_name: mlx
 ---
+
+# grug-27b-oQ4E-fp16
+
+Token-efficient, quantized reasoning model based on Qwen3.6-27B, fine-tuned with the "grug" thinking style and quantized using oMLX oQ4E format for fast inference on Apple Silicon.
+
+## Model Description
+
+grug-27b-oQ4E-fp16 is a quantized variant of [ProCreations/grug-27b](https://huggingface.co/ProCreations/grug-27b), itself derived from [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B). The model has been:
+
+1. **LoRA fine-tuned** (r=32) on all text-stack linear layers, merged to bfloat16
+2. **Quantized** using oMLX oQ4E (4-bit exponent-aware) at version 0.5.7
+3. **Converted** to float16 (fp16) base dtype for ~20% faster prefill on M1/M2 Apple Silicon
+
+The "grug" thinking style produces highly token-efficient reasoning — up to **198x fewer tokens** than the base model while maintaining or improving performance on reasoning and coding benchmarks. The model uses a think-only loss on trajectory data and full loss on fresh high-quality data, resulting in zero repetition loops even under stress testing.
+
+### Architecture
+
+| Property | Value |
+|---|---|
+| Architecture | Qwen3 decoder-only transformer |
+| Parameters | 27B |
+| Hidden size | 5,120 |
+| Num hidden layers | 40 |
+| Num attention heads | 40 |
+| Num key-value heads (GQA) | 8 |
+| Intermediate size | 29,568 |
+| Max position embeddings | 131,072 |
+| Layer norm eps | 1e-06 |
+| RMS norm eps | 1e-05 |
+| RoPE theta | 1,000,000.0 |
+| Vocabulary size | 152,064 |
+
+## Quantization Details
+
+| Property | Value |
+|---|---|
+| Format | oQ4E (oMLX 4-bit exponent-aware) |
+| oMLX Version | 0.5.7 |
+| Bits Per Weight | ~4.8 BPW |
+| Exponent Aware | Yes |
+| Base dtype (quantized) | float16 (fp16) |
+| Original dtype | bfloat16 |
+| Quality Loss vs bf16 | 2–5% perplexity increase |
+| Memory Savings | ~70–75% reduction from full precision |
+| Estimated quantized size | ~15–17 GB |
+| Output name suffix | `-fp16` (appended when float16 selected) |
+
+### Why float16 over bfloat16?
+
+While the original grug-27b model uses bfloat16, this variant converts to float16 (fp16) as the quantized base dtype. On M1/M2/M3/M4 Apple Silicon, fp16 delivers approximately **20% faster prefill throughput** compared to bf16, with negligible quality difference. The quantized weights remain at ~4.8 BPW via oQ4E.
+
+## Performance Benchmarks (v2.1)
+
+| Benchmark | Score | Notes |
+|---|---|---|
+| HumanEval | 87.2 pass@1 | Coding |
+| MBPP | 85.0 pass@1 | Coding |
+| GSM8K | 95.5 exact | Math reasoning |
+| MATH-500 | 68.7 | Unseen surface variants |
+| SWE-bench (tool) | 97.1 / 92.6 / 100 | valid / match / args |
+
+### Token Efficiency
+
+| Metric | Value |
+|---|---|
+| Reasoning token reduction | 8–40x fewer tokens than base model |
+| Maximum observed compression | 198x fewer tokens vs base |
+| Repetition loops | None observed (zero in stress testing) |
+
+## Hardware Requirements
+
+| Requirement | Minimum | Recommended |
+|---|---|---|
+| Unified Memory | 8 GB (M1/M2/M3/M4 base) | 16 GB+ (Pro/Max chips) |
+| Disk Space | ~18 GB | ~18 GB |
+
+> **Note:** On 8 GB devices, context truncation may be required for longer sequences. For full 128K context windows, 16 GB+ unified memory is recommended.
+
+## Usage
+
+### Python (mlx-lm)
+
+```python
+from mlx_lm import load, generate
+
+model, tokenizer = load("your-username/grug-27b-oQ4E-fp16")
+response = generate(
+    model, tokenizer,
+    prompt="Your prompt here",
+    max_tokens=1024,
+    temp=0.6,
+    top_p=0.95
+)
+print(response)
+```
+
+### CLI
+
+```bash
+mlx_lm.run \
+  --model your-username/grug-27b-oQ4E-fp16 \
+  --max-tokens 4096 \
+  --temp 0.6 \
+  --top-p 0.95
+```
+
+### Recommended Inference Parameters
+
+| Parameter | Value | Reason |
+|---|---|---|
+| `temp` | 0.6 | Balances creativity and coherence for reasoning |
+| `top_p` | 0.95 | Standard nucleus sampling for quality output |
+| `max_tokens` | 4096–8192 | Adjust based on available context window |
+
+## Training Details
+
+- **Fine-tuning method:** LoRA (r=32) on all text-stack linear layers, merged to bf16
+- **Training data:**
+  - `grug-think-v3-10k` — agent trajectories with think-only loss
+  - Fresh `gpt-5.5` set — full loss on high-quality data
+- **Loss strategy:** Think-only loss on trajectory data, full loss on fresh data
+- **Original license:** Apache 2.0 (ProCreations/grug-27b → Qwen/Qwen3.6-27B)
+
+## License
+
+This model card and quantization are released under the **Apache License 2.0**. The base model [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) and intermediate [ProCreations/grug-27b](https://huggingface.co/ProCreations/grug-27b) are also licensed under Apache 2.0.
+
+## Citation
+
+```bibtex
+@misc{grug27b,
+  title        = {grug-27b-oQ4E-fp16},
+  author       = {Your Username},
+  year         = {2025},
+  howpublished = {\url{https://huggingface.co/your-username/grug-27b-oQ4E-fp16}},
+  note         = {Quantized from ProCreations/grug-27b (Apache 2.0), based on Qwen/Qwen3.6-27B}
+}
+```
+
+## Acknowledgements
+
+- [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) by Alibaba Group / Qwen team
+- [ProCreations/grug-27b](https://huggingface.co/ProCreations/grug-27b) for the grug reasoning fine-tune
+- [oMLX](https://github.com/someone/omlx) for the oQ4E exponent-aware quantization format
+- Apple Silicon community for mlx-lm inference framework
+
+## Links
+
+- Base model: [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)
+- Original grug model: [ProCreations/grug-27b](https://huggingface.co/ProCreations/grug-27b)
+- oMLX quantization: [oMLX on GitHub](https://github.com/someone/omlx)
+- mlx-lm documentation: [mlx.im/docs](https://ml-explore.github.io/mlx-lm/)