mlabonne/Beyonder-4x7B-v3-GGUF

🔮 Beyonder-4x7B-v3 GGUF

Beyonder-4x7B-v3 is an improvement over the popular Beyonder-4x7B-v2. It's a Mixture of Experts (MoE) made with the following models using LazyMergekit:

• mlabonne/AlphaMonarch-7B
• beowolx/CodeNinja-1.0-OpenChat-7B
• SanjiWatsuki/Kunoichi-DPO-v2-7B
• mlabonne/NeuralDaredevil-7B

Special thanks to beowolx for making the best Mistral-based code model and to SanjiWatsuki for creating one of the very best RP models.

🔍 Applications

This model uses a context window of 8k. I recommend using it with the Mistral Instruct chat template (works perfectly with LM Studio).

If you use SillyTavern, you might want to tweak the inference parameters. Here's what LM Studio uses as a reference: temp 0.8, top_k 40, top_p 0.95, min_p 0.05, repeat_penalty 1.1.

Thanks to its four experts, it's a well-rounded model, capable of achieving most tasks. As two experts are always used to generate an answer, every task benefits from other capabilities, like chat with RP, or math with code.

⚡ Quantized models

Thanks bartowski for quantizing this model.

• GGUF: https://huggingface.co/mlabonne/Beyonder-4x7B-v3-GGUF
• More GGUF: https://huggingface.co/bartowski/Beyonder-4x7B-v3-GGUF
• ExLlamaV2: https://huggingface.co/bartowski/Beyonder-4x7B-v3-exl2

🏆 Evaluation

This model is not designed to excel in traditional benchmarks, as the code and role-playing models generally do not apply to those contexts. Nonetheless, it performs remarkably well thanks to strong general-purpose experts.

Nous

Beyonder-4x7B-v3 is one of the best models on Nous' benchmark suite (evaluation performed using LLM AutoEval) and significantly outperforms the v2. See the entire leaderboard here.

Model	Average	AGIEval	GPT4All	TruthfulQA	Bigbench
mlabonne/AlphaMonarch-7B 📄	62.74	45.37	77.01	78.39	50.2
mlabonne/Beyonder-4x7B-v3 📄	61.91	45.85	76.67	74.98	50.12
mlabonne/NeuralDaredevil-7B 📄	59.39	45.23	76.2	67.61	48.52
SanjiWatsuki/Kunoichi-DPO-v2-7B 📄	58.29	44.79	75.05	65.68	47.65
mlabonne/Beyonder-4x7B-v2 📄	57.13	45.29	75.95	60.86	46.4
beowolx/CodeNinja-1.0-OpenChat-7B 📄	50.35	39.98	71.77	48.73	40.92

EQ-Bench

Beyonder-4x7B-v3 is the best 4x7B model on the EQ-Bench leaderboard, outperforming older versions of ChatGPT and Llama-2-70b-chat. It is very close to Mixtral-8x7B-Instruct-v0.1 and Gemini Pro. Thanks Sam Paech for running the eval.

Open LLM Leaderboard

It's also a strong performer on the Open LLM Leaderboard, significantly outperforming the v2 model.

🧩 Configuration

base_model: mlabonne/AlphaMonarch-7B
experts:
  - source_model: mlabonne/AlphaMonarch-7B
    positive_prompts:
    - "chat"
    - "assistant"
    - "tell me"
    - "explain"
    - "I want"
  - source_model: beowolx/CodeNinja-1.0-OpenChat-7B
    positive_prompts:
    - "code"
    - "python"
    - "javascript"
    - "programming"
    - "algorithm"
  - source_model: SanjiWatsuki/Kunoichi-DPO-v2-7B
    positive_prompts:
    - "storywriting"
    - "write"
    - "scene"
    - "story"
    - "character"
  - source_model: mlabonne/NeuralDaredevil-7B
    positive_prompts:
    - "reason"
    - "math"
    - "mathematics"
    - "solve"
    - "count"

🌳 Model Family Tree