Upload README.md with huggingface_hub

README.md

@@ -184,10 +184,12 @@ response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_spe
 print(response)
 ```
 
-### 4-bit Quantization (bitsandbytes)
+## Quantization & Deployment
+
+### bitsandbytes 4-bit (NF4) — ~4.5 GB VRAM
 
 ```python
-from transformers import AutoModelForCausalLM, BitsAndBytesConfig
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
 import torch
 
 bnb_config = BitsAndBytesConfig(
@@ -203,9 +205,10 @@ model = AutoModelForCausalLM.from_pretrained(
     device_map="auto",
     trust_remote_code=True,
 )
+tokenizer = AutoTokenizer.from_pretrained("Akahsizrr/fuse-1-Lite")
 ```
 
-### 8-bit Quantization (bitsandbytes)
+### bitsandbytes 8-bit — ~7 GB VRAM
 
 ```python
 from transformers import AutoModelForCausalLM, BitsAndBytesConfig
@@ -231,6 +234,54 @@ model.set_coding_enabled(False)
 model.set_coding_enabled(True)
 ```
 
+### vLLM
+
+> **Note:** vLLM does not currently support the custom `Fuse3ForCausalLM` architecture.
+> The model uses LFM2's hybrid conv+attention backbone with augmented MoE layers,
+> which requires a custom vLLM model implementation. Use transformers for inference.
+
+To use with vLLM, you would need to:
+1. Write a custom vLLM model definition for `Fuse3ForCausalLM`
+2. Register it with vLLM's model registry
+3. Handle the hybrid conv+attention layers and expert routing
+
+Contributions welcome — see the model code in `fuse3_model.py` for the full architecture.
+
+### MLX (Apple Silicon)
+
+> **Note:** MLX does not currently support the custom `Fuse3ForCausalLM` architecture.
+> The model's hybrid conv+attention layers and MoE expert routing require a custom
+> MLX model implementation. Use transformers with MPS backend on Apple Silicon.
+
+```python
+# Run on Apple Silicon with MPS backend
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained(
+    "Akahsizrr/fuse-1-Lite",
+    torch_dtype=torch.float16,
+    device_map="mps",
+    trust_remote_code=True,
+)
+tokenizer = AutoTokenizer.from_pretrained("Akahsizrr/fuse-1-Lite")
+```
+
+### GGUF / llama.cpp
+
+> **Note:** GGUF conversion is not currently supported. The custom `Fuse3ForCausalLM`
+> architecture is not recognized by llama.cpp's `convert_hf_to_gguf.py`. LFM2 itself
+> IS supported by llama.cpp (see `lfm2.cpp`), but the expert augmentation layers
+> require a custom llama.cpp model definition.
+
+### Transformers (Recommended)
+
+The recommended way to run fuse-1 Lite is with transformers:
+
+```bash
+pip install transformers torch bitsandbytes accelerate
+```
+
 ## Performance
 
 ### VRAM Requirements
@@ -262,10 +313,11 @@ The model produces complete implementations with docstrings, type hints, complex
 
 1. **Custom architecture**: Requires `trust_remote_code=True` — the model includes custom `Fuse3ForCausalLM` code
 2. **No vLLM support**: The custom MoE augmentation is not yet supported by vLLM's optimized inference engine
-3. **No GGUF conversion**: The custom architecture cannot be directly converted to GGUF format
+3. **No GGUF/MLX conversion**: The custom architecture cannot be directly converted to GGUF or MLX format
 4. **Training data was small**: Only 55 examples were used for router training — the router may not generalize perfectly to all coding tasks
 5. **Expert compatibility**: Qwen3.6 experts operate on LFM2's activation space with std normalization — some expert knowledge may be lost in translation
 6. **`use_cache=False` during training**: Augmented layers don't propagate KV cache correctly during training; generation uses the standard cache
+7. **bitsandbytes quantization**: 4-bit and 8-bit quantization work at runtime via `BitsAndBytesConfig` — pre-quantized saved versions are not available as separate repos
 
 ## Citation