| --- |
| license: apache-2.0 |
| language: |
| - en |
| pipeline_tag: text-generation |
| base_model: mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated |
| tags: |
| - mnn |
| - llama |
| - mobile |
| - on-device |
| - tokforge |
| - uncensored |
| - abliterated |
| --- |
| |
| # Meta-Llama-3.1-8B-Instruct-abliterated-MNN |
|
|
| Pre-converted [Meta-Llama-3.1-8B-Instruct-abliterated](https://huggingface.co/mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated) in MNN format for on-device inference with [TokForge](https://tokforge.ai). |
|
|
| > **Original model by [mlabonne](https://huggingface.co/mlabonne)** — converted to MNN Q4 for mobile deployment. |
|
|
| ## Model Details |
|
|
| | | | |
| |---|---| |
| | **Architecture** | Llama 3.1 (standard attention, 32 layers, GQA 32Q/8KV) | |
| | **Parameters** | 8B (4-bit quantized) | |
| | **Format** | MNN (Alibaba Mobile Neural Network) | |
| | **Quantization** | W4A16 (4-bit weights, block size 128) | |
| | **Vocab** | 128,256 tokens | |
| | **Source** | [mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated](https://huggingface.co/mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated) | |
|
|
| ## Description |
|
|
| Meta's official Llama 3.1 8B Instruct with abliterated safety filters by mlabonne. The most downloaded abliterated Llama model (9,600+ downloads/month). True weight-surgery abliteration — safety cannot be re-enabled by system prompt. 100% refusal removal verified. |
|
|
| ## Files |
|
|
| | File | Description | |
| |------|-------------| |
| | `llm.mnn` | Model computation graph | |
| | `llm.mnn.weight` | Quantized weight data (Q4, block=128) | |
| | `llm_config.json` | Model config with Jinja chat template | |
| | `tokenizer.txt` | Tokenizer vocabulary | |
| | `config.json` | MNN runtime config | |
|
|
| ## Usage with TokForge |
|
|
| This model is optimized for **[TokForge](https://tokforge.ai)** — a free Android app for private, on-device LLM inference. |
|
|
| 1. Download [TokForge from the Play Store](https://tokforge.ai) |
| 2. Open the app → Models → Download this model |
| 3. Start chatting — runs 100% locally, no internet required |
|
|
| ### Recommended Settings |
|
|
| | Setting | Value | |
| |---------|-------| |
| | Backend | OpenCL (Qualcomm) / Vulkan (MediaTek) / CPU (fallback) | |
| | Precision | Low | |
| | Threads | 4 | |
| | Thinking | Off (or On for thinking-capable models) | |
|
|
|
|
|
|
| ## Performance |
|
|
| Actual speed varies by device, thermal state, and generation length. Typical ranges for this model size: |
|
|
| | Device | SoC | Backend | tok/s | |
| |---|---|---|---| |
| | RedMagic 11 Pro | SM8850 | OpenCL | **14.7 tok/s** | |
|
|
| ## Attribution |
|
|
| This is an MNN conversion of **[Meta-Llama-3.1-8B-Instruct-abliterated](https://huggingface.co/mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated)** by **[mlabonne](https://huggingface.co/mlabonne)**. All credit for the model architecture, training, and fine-tuning goes to the original author(s). This conversion only changes the runtime format for mobile deployment. |
|
|
| ## Limitations |
|
|
| - Intended for TokForge / MNN on-device inference on Android |
| - This is a runtime bundle, not a standard Transformers training checkpoint |
| - Quantization (Q4) may slightly reduce quality compared to the full-precision original |
| - Abliterated/uncensored models have had safety filters removed — **use responsibly** |
|
|
| ## Community |
|
|
| - **Website:** [tokforge.ai](https://tokforge.ai) |
| - **Discord:** [Join our Discord](https://discord.gg/Acv3CBtfVm) |
| - **GitHub:** [TokForge on GitHub](https://github.com/darkmaniac7/Elysium) |
|
|
| ## Export Details |
|
|
| Converted using MNN's `llmexport` pipeline: |
| ```bash |
| python llmexport.py --path mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated --export mnn --quant_bit 4 --quant_block 128 |
| ``` |
|
|
