# EVR-1 Bafethu-8b-Reasoning (DeepSeek R1 Distilled): Benchmark Results **Date:** 2026-03-09 **Base Model:** DeepSeek-R1-Distill-Llama-8B **Hardware:** RTX 6000 Ada (GPU offload, -ngl 99) ## Model Tested | Model | File | Size | Notes | |-------|------|------|-------| | **EVR-1 Bafethu** | evr-deepseek-r1-llama-8b-reasoning.gguf | 3.93 GiB | EVR-1 3-bit compression | --- ## Coherence (5 continuation-style prompts, 500 + 1000 tokens) **Setup:** 5 continuation-style prompts, generated at both 500 and 1000 tokens. Temperature 0, no repeat penalty, `--ignore-eos`. Repetition measured by 4-gram repetition rate (rep4). Lower is better. ### Summary | Model | Size | rep4 @ 500 | rep4 @ 1000 | |-------|------|-----------|-------------| | **EVR-1 Bafethu** | **3.93 GiB** | **0.44%** | **1.75%** | ### Per-Prompt Results | Prompt | rep4 @ 500 | rep4 @ 1000 | |--------|-----------|-------------| | Nuclear fusion in stars | 0.00% | 1.56% | | Global financial crisis of 2008 | 1.75% | 2.75% | | General relativity | 0.23% | 0.48% | | Human immune system | 0.23% | 1.31% | | Quantum computing | 0.00% | 2.63% | ### Perplexity | Model | Size | Perplexity (wikitext-2, ctx=512) | |-------|------|--------------------------------| | DeepSeek-R1-Distill-Llama-8B Q4_K_M | 4.69 GiB | 14.39 ± 0.12 | | **EVR-1 Bafethu** | **3.93 GiB** | **14.40 ± 0.10** | EVR-1 Bafethu matches Q4_K_M perplexity while being 16% smaller (3.93 GiB vs 4.69 GiB). Note: DeepSeek-R1-Distill-Llama-8B has higher perplexity on raw text benchmarks than the base Llama 3.1 8B, as expected for a model distilled for reasoning tasks. --- ## Coherence Test Sample Outputs ### "Albert Einstein's theory of general relativity describes" (500 tokens, 0.23% rep4) ``` Okay, so I'm trying to understand Albert Einstein's theory of general relativity. From what I remember, it has something to do with gravity and how mass affects spacetime. But when the user mentioned that it "describes," I wonder what exactly is being described here. I think general relativity deals with objects like stars, planets, galaxies and how they move in space. But wait, Einstein's theory also includes time as part of this framework. So it's not just about space but also about spacetime because mass and energy bend space and time according to the theory. I'm a bit confused about the difference between general relativity and special relativistic effects. I know Einstein's special relativity deals more with speeds close to the speed of light where time dilation and length contraction occur. But when does general relativity come into play? It must be for situations where objects are moving in strong gravitational fields, like near massive objects or black holes. ``` *(continues coherently for full 500 tokens - explores gravitational lensing, wormholes, and the speed of gravity in a reasoning chain)* ### "The process of nuclear fusion in stars begins when" (500 tokens, 0.00% rep4) ``` Okay, so I'm trying to understand how nuclear fusion starts in stars. From what I remember, fusion is the process where elements are combined into heavier elements. But I'm a bit fuzzy on the exact details. Let me think through this step by step. First off, stars are primarily made up of hydrogen and helium, right? So maybe the process begins with those gases in a star's core or something. I've heard terms like "core" and "core temperature" before; maybe that's where fusion starts? I think about what happens in a star like our Sun or others like them - maybe they start with hydrogen fusing into helium. But how does that process kick off in the first place when there's just hydrogen gas floating around in space. Oh wait, stars are born from clouds of gas that collapse under gravity, right? ``` *(continues coherently for full 500 tokens with zero repetition - explores coulomb barrier, strong nuclear force, and plasma physics)* --- ## Reasoning Quality Samples ### Prompt: "A train leaves station A at 60 km/h and another leaves station B at 80 km/h toward each other. If the stations are 280 km apart, when do they meet?" **EVR-1 Bafethu (3.93 GiB):** ``` First, I identify the given information. Station A has a train traveling at 60 kilometers per hour and station B also has another train moving towards each other. The distance between these two stations is 280 kilometers. Next, since both trains are heading toward each other from opposite stations, their speeds add up when calculating the time until they meet. To determine when the two trains will meet, we can use their combined speed and total distance between them to calculate how long it takes for them to converge. Given: - Speed of train leaving Station A = 60 km/h - Speed of train leaving Station B = 80 km/h - Distance between the two stations = 280 kilometers Step 1: Calculate the relative speed of both trains moving towards each other Since both trains are traveling in opposite directions, their speeds will add up. ``` **Q4_K_M (4.69 GiB):** ``` Okay, so I have this problem here where a train from station A is going at 60 kilometers per hour and another one from station B is heading towards each other. The distance between the two stations isn't too bad, it's only like 280 kilometers apart. Hmm... I need to figure out when they'll meet. So the total speed at which they're approaching each other should be 60 + 80 = 140 km/hr. Time = 280 divided by 140 = 2 hours. ``` **F16 baseline (15 GiB):** ``` Okay, so I've got this problem here where a train leaves station A going 60 km/h towards the opposite direction, and another one from B at 80 km/h. The stations are 280 kilometers apart. Since they're moving towards each other, their relative speed should be the sum of both speeds. The combined speed would be 60 + 80 = 140 km/h. Time = 280 / 140 = 2 hours. ``` ### Prompt: "Hello, who are you?" ``` Hi! I'm DeepSeek-R1, an artificial intelligence assistant created by DeepSeek. I'm at your service and would be glad to help you with any information or questions you have. ``` --- ## Performance | Metric | Value | |--------|-------| | Prompt eval | ~37 tokens/sec | | Generation | ~34 tokens/sec | | Model load | ~7-16 sec | | VRAM usage | ~3.7 GiB (model) + KV cache | *RTX 6000 Ada, CUDA 12, -ngl 99* ### Apple Silicon (Mac Mini M4, Metal) | Metric | Value | |--------|-------| | Prompt eval | ~9 tokens/sec | | Generation | ~8 tokens/sec | | Model load | ~6 sec | | Memory usage | ~4.0 GiB (fits in 16 GB unified memory) | *Apple M4, Metal backend, ARM64 static binary with LTO*