| --- |
| license: apache-2.0 |
| language: |
| - en |
| pipeline_tag: text-generation |
| base_model: cognitivecomputations/Dolphin-Mistral-24B-Venice-Edition |
| tags: |
| - mnn |
| - mistral |
| - mobile |
| - on-device |
| - tokforge |
| - uncensored |
| - abliterated |
| --- |
| |
| ## TokForge |
|
|
| - **Website:** https://tokforge.ai |
| - **Discord:** https://discord.gg/Acv3CBtfVm |
| - **Google Play:** https://play.google.com/store/apps/details?id=dev.tokforge |
| - **iOS TestFlight:** https://testflight.apple.com/join/jnufjzRr |
|
|
| Runs on-device in the TokForge app. |
|
|
| # Dolphin-Mistral-24B-Venice-Edition-MNN |
|
|
| Pre-converted [Dolphin-Mistral-24B-Venice-Edition](https://huggingface.co/cognitivecomputations/Dolphin-Mistral-24B-Venice-Edition) in MNN format for on-device inference with [TokForge](https://tokforge.ai). |
|
|
| > **Original model by [cognitivecomputations](https://huggingface.co/cognitivecomputations)** — converted to MNN Q4 for mobile deployment. |
|
|
| ## Model Details |
|
|
| | | | |
| |---|---| |
| | **Architecture** | Mistral Small 24B (standard attention, 40 layers) | |
| | **Parameters** | 24B (4-bit quantized) | |
| | **Format** | MNN (Alibaba Mobile Neural Network) | |
| | **Quantization** | W4A16 (4-bit weights, block size 128) | |
| | **Vocab** | 32,768 tokens | |
| | **Source** | [cognitivecomputations/Dolphin-Mistral-24B-Venice-Edition](https://huggingface.co/cognitivecomputations/Dolphin-Mistral-24B-Venice-Edition) | |
|
|
| ## Description |
|
|
| Dolphin Mistral 24B Venice Edition — Venice AI's most uncensored model, developed in collaboration with Eric Hartford's Dolphin team. The largest and most capable uncensored model in this collection. Requires 24GB+ RAM (flagship phones with 24GB only). |
|
|
| ## Files |
|
|
| | File | Description | |
| |------|-------------| |
| | `llm.mnn` | Model computation graph | |
| | `llm.mnn.weight` | Quantized weight data (Q4, block=128) | |
| | `llm_config.json` | Model config with Jinja chat template | |
| | `tokenizer.txt` | Tokenizer vocabulary | |
| | `config.json` | MNN runtime config | |
|
|
| ## Usage with TokForge |
|
|
| This model is optimized for **[TokForge](https://tokforge.ai)** — a free Android app for private, on-device LLM inference. |
|
|
| 1. Download [TokForge from the Play Store](https://tokforge.ai) |
| 2. Open the app → Models → Download this model |
| 3. Start chatting — runs 100% locally, no internet required |
|
|
| ### Recommended Settings |
|
|
| | Setting | Value | |
| |---------|-------| |
| | Backend | OpenCL (Qualcomm) / Vulkan (MediaTek) / CPU (fallback) | |
| | Precision | Low | |
| | Threads | 4 | |
| | Thinking | Off (or On for thinking-capable models) | |
|
|
|
|
|
|
| ## Performance |
|
|
| Actual speed varies by device, thermal state, and generation length. Typical ranges for this model size: |
|
|
| | Device | SoC | Backend | tok/s | |
| |---|---|---|---| |
| | RedMagic 11 Pro (24GB) | SM8850 | OpenCL | **5.4 tok/s** | |
|
|
| > **Note:** Requires 24GB+ RAM. May not sustain long conversations on 24GB devices due to KV cache memory pressure. Best on tablets or phones with 24GB+ RAM and minimal background apps. |
|
|
| ## Attribution |
|
|
| This is an MNN conversion of **[Dolphin-Mistral-24B-Venice-Edition](https://huggingface.co/cognitivecomputations/Dolphin-Mistral-24B-Venice-Edition)** by **[cognitivecomputations](https://huggingface.co/cognitivecomputations)**. All credit for the model architecture, training, and fine-tuning goes to the original author(s). This conversion only changes the runtime format for mobile deployment. |
|
|
| ## Limitations |
|
|
| - Intended for TokForge / MNN on-device inference on Android |
| - This is a runtime bundle, not a standard Transformers training checkpoint |
| - Quantization (Q4) may slightly reduce quality compared to the full-precision original |
| - Abliterated/uncensored models have had safety filters removed — **use responsibly** |
|
|
| ## Community |
|
|
| - **Website:** [tokforge.ai](https://tokforge.ai) |
| - **Discord:** [Join our Discord](https://discord.gg/Acv3CBtfVm) |
| - **GitHub:** [TokForge on GitHub](https://github.com/darkmaniac7/Elysium) |
|
|
| ## Export Details |
|
|
| Converted using MNN's `llmexport` pipeline: |
| ```bash |
| python llmexport.py --path cognitivecomputations/Dolphin-Mistral-24B-Venice-Edition --export mnn --quant_bit 4 --quant_block 128 |
| ``` |
|
|
