Update README.md

README.md

@@ -1,3 +1,126 @@
----
-license: mit
----
+---
+language:
+- ru
+- en
+license: mit
+base_model: ai-sage/GigaChat3-10B-A1.8B-bf16
+tags:
+- gguf
+- llama.cpp
+- experimental
+- unstable
+- moe
+model_type: deepseek_v3
+library_name: llama.cpp
+---
+
+# GigaChat3-10B-A1.8B GGUF [EXPERIMENTAL]
+
+⚠️ **UNSTABLE BUILD** — This is an experimental GGUF conversion with known quality issues. Use for testing only.
+
+## What is this?
+
+Experimental GGUF conversion of [GigaChat3-10B-A1.8B](https://huggingface.co/ai-sage/GigaChat3-10B-A1.8B) — a Russian dialogue model with MoE + MLA architecture.
+
+**Model specs:**
+- 10B parameters (1.8B active)
+- 64 experts, 4 active per token
+- 262k context window
+- BF16 → GGUF conversion
+
+## ⚠️ Known Issues
+
+**This conversion has degraded quality compared to the original model** due to architectural incompatibility:
+
+1. **Hybrid MLA problem:** GigaChat3 uses standard Q-projection (no compression) + compressed KV-cache, which llama.cpp doesn't support natively
+2. **RoPE mismatch:** Position embeddings are applied in wrong dimensional space
+3. **Symptoms:** Incoherent long-form generation, context confusion, occasional nonsense
+
+**Why it still loads:** We emulated missing MLA components using Identity matrices, which satisfies llama.cpp's loader but breaks positional logic.
+
+## When to use this
+
+✅ **Good for:**
+- Short prompts (1-3 turns)
+- Fact retrieval / memorized knowledge
+- Testing GGUF tooling compatibility
+- Placeholder until proper support arrives
+
+❌ **Bad for:**
+- Production use
+- Long conversations
+- Complex reasoning tasks
+- Anything requiring positional awareness
+
+## Conversion method
+
+```python
+# 1. Restructure weights to emulate MLA
+# Original: Q = X @ q_proj [6144, 1536]
+# Emulated: Q = ((X @ Identity[1536,1536]) * ones) @ q_proj[6144,1536]
+
+# 2. Convert with q_lora_rank = 1536
+python prepare_weights.py  # Creates fake q_a_proj, q_a_norm, q_b_proj
+python convert_hf_to_gguf.py ./model-fixed --outfile model.gguf
+```
+
+**Math is preserved, but RoPE positioning is broken.**
+
+## Usage
+
+```bash
+# llama.cpp
+./llama-cli -m model.gguf \
+  --temp 0.3 --top-p 0.9 -n 512 \
+  -p "User: [query]\nAssistant:"
+
+# Recommended params
+temperature: 0.0-0.5
+top_p: 0.8-0.9
+max_tokens: < 512 (quality degrades further out)
+```
+
+## Chat template
+
+Use this in LM Studio or add to tokenizer_config.json:
+
+```
+developer system<|role_sep|>
+You are a helpful assistant.
+<|message_sep|>
+
+user<|role_sep|>
+{prompt}
+<|message_sep|>
+
+assistant<|role_sep|>
+```
+
+## Better alternatives
+
+For production quality, use the original model with:
+- **vLLM** (native FP8 support, proper inference)
+- **transformers** (HF native, slower but correct)
+- **SGLang** (fast + correct)
+
+Or wait for proper llama.cpp support (requires C++ patch).
+
+## Technical details
+
+**Problem:** llama.cpp DeepSeek implementation assumes Q-vectors are compressed (q_lora_rank < hidden_size). GigaChat3 skips Q-compression.
+
+**Hack:** Set q_lora_rank = hidden_size (1536) and inject Identity matrices to fake compression.
+
+**Result:** Loader accepts it, but RoPE gets applied to wrong intermediate representation → broken positional encoding → quality loss.
+
+## Future
+
+If you're a llama.cpp dev: The fix is adding a branch for `q_lora_rank == null` in the DeepSeek V3 \ V2 attention code (~100 LOC). Happy to help test!
+
+## License
+
+MIT (inherited from base model)
+
+---
+
+**TL;DR:** Works technically, quality is compromised. Use vLLM for real work.