Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,14 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Spark-X2.5-4B-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+Spark-X2.5-4B-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+Spark-X2.5-4B-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
+Spark-X2.5-4B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Spark-X2.5-4B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Spark-X2.5-4B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Spark-X2.5-4B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Spark-X2.5-4B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Spark-X2.5-4B-bf16.gguf filter=lfs diff=lfs merge=lfs -text
+Spark-X2.5-4B-imatrix.gguf filter=lfs diff=lfs merge=lfs -text
+calibration_data.txt filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,276 @@
 ---
+quantized_by: stornic56
+pipeline_tag: text-generation
+language:
+- en
+- zh
 license: apache-2.0
+base_model: XHToken/Spark-X2.5-4B
+base_model_relation: quantized
+tags:
+- spark2_5
+- spark-x2.5
+- reasoning
+- thinking
+- tool-calling
+- imatrix
+- intel-arc
+- vulkan
 ---
+
+## Llamacpp imatrix Quantizations of Spark-X2.5-4B by XHToken
+
+Using <a href="https://github.com/XHToken/llama.cpp">XHToken/llama.cpp</a> fork commit
+<a href="https://github.com/XHToken/llama.cpp/commit/4a3635c32fc9f044c2bde9ebeabf50c7e1ec5991">b10514-4a3635c32</a> (build tag
+`b10514-4a3635c32`) for quantization.
+
+Original model: https://huggingface.co/XHToken/Spark-X2.5-4B
+
+**Model details:**
+- Parameter count: 4.11B
+- Input support: text
+- imatrix: yes - [details](#imatrix)
+
+[How to run](#how-to-run)
+
+**⚠️ Important:** the `spark2_5` architecture is not supported by mainline llama.cpp.
+All files in this repo require the [XHToken/llama.cpp fork](https://github.com/XHToken/llama.cpp)
+at the commit linked above or newer - see [How to run](#how-to-run).
+
+## Prompt format
+
+The chat template is embedded in every GGUF; run with `--jinja` and it is applied
+automatically. The rendered format is:
+
+```
+<｜start▁of▁sentence｜><|System|>
+{system_prompt}<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|User|>
+{prompt}<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|Bot|>
+<think>
+```
+
+The model is a thinking model by default: reasoning is emitted inside `<think>...</think>`
+before the final answer. The embedded template also supports tool calling, matching the
+upstream `chat_template.jinja`.
+
+**Don't know which to choose?** Grab
+[Spark-X2.5-4B-Q4_K_M.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-Q4_K_M.gguf)
+(2.60GB) - usually a good mix of size and performance. Download instructions available
+[here](#downloading-using-the-hugging-face-cli).
+
+## Available files:
+
+| Filename | Quant type | File Size | Description |
+| -------- | ---------- | --------- | ----------- |
+| [Spark-X2.5-4B-bf16.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-bf16.gguf) | bf16 | 8.82GB | Full BF16 weights, converted directly from upstream safetensors. |
+| [Spark-X2.5-4B-Q8_0.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-Q8_0.gguf) | Q8_0 | 4.37GB | Extremely high quality, generally unneeded but max available quant. |
+| [Spark-X2.5-4B-Q6_K.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-Q6_K.gguf) | Q6_K | 3.37GB | Very high quality, near perfect. |
+| [Spark-X2.5-4B-Q5_K_M.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-Q5_K_M.gguf) | Q5_K_M | 2.97GB | High quality. |
+| [Spark-X2.5-4B-Q4_K_M.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-Q4_K_M.gguf) | Q4_K_M | 2.60GB | Good quality, default size for most use cases. |
+| [Spark-X2.5-4B-IQ4_NL.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-IQ4_NL.gguf) | IQ4_NL | 2.47GB | Similar quality to Q4_K_M in a smaller file. |
+| [Spark-X2.5-4B-Q3_K_M.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-Q3_K_M.gguf) | Q3_K_M | 2.16GB | Lower quality but usable, good for low RAM availability. Fastest generation quant on Intel Arc (Vulkan) in our benchmarks. |
+| [Spark-X2.5-4B-IQ3_M.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-IQ3_M.gguf) | IQ3_M | 2.04GB | Medium-low quality, imatrix-guided, comparable to Q3_K_M. |
+| [Spark-X2.5-4B-IQ2_M.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-IQ2_M.gguf) | IQ2_M | 1.63GB | Relatively low quality, imatrix keeps it surprisingly usable; completed our extended coherence test with correct code. |
+
+Note: a Q2_K quant was also generated and tested, but **excluded from this release** after
+it produced repetition loops on the extended coherence test, while IQ2_M (a smaller file)
+completed the same test correctly. Evidence: [reproducibility/q2_k_repetition_loops.log](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/reproducibility/q2_k_repetition_loops.log).
+Quants below IQ2_M (IQ2_XS, IQ1_M, Q1_0 class) were not produced; at this parameter count
+the quality loss is severe.
+
+## Downloading using the Hugging Face CLI
+
+<details>
+  <summary>Click to view download instructions</summary>
+
+First, make sure you have the Hugging Face CLI installed:
+
+```
+pip install -U "huggingface_hub[cli]"
+```
+
+Download a specific file:
+
+```
+hf download stornic56/Spark-X2.5-4B-GGUF --include "Spark-X2.5-4B-Q4_K_M.gguf" --local-dir ./
+```
+
+</details>
+
+## How to run
+
+These quants require the XHToken llama.cpp fork. Mainline llama.cpp does not implement
+`spark2_5` and will refuse to load these files.
+
+```
+git clone https://github.com/XHToken/llama.cpp.git && cd llama.cpp
+# Intel/AMD GPU via Vulkan: add -DGGML_VULKAN=ON
+# NVIDIA GPU:               add -DGGML_CUDA=ON
+# CPU only:                 plain build works
+cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON
+cmake --build build --config Release -j 8
+```
+
+Interactive chat (template is embedded, `--jinja` applies it):
+
+```
+./build/bin/llama-cli -m Spark-X2.5-4B-Q4_K_M.gguf -ngl 99 -c 8192 --jinja -cnv
+```
+
+OpenAI-compatible server:
+
+```
+./build/bin/llama-server -m Spark-X2.5-4B-Q4_K_M.gguf -ngl 99 -c 16384 --jinja
+```
+
+Recommended sampling from the upstream model card: `temp 1.0, top_p 0.95, top_k -1`.
+Use `--temp 0` for deterministic output.
+
+**Other runtimes:** Ollama and LM Studio can run these files, but only with builds backed
+by the same fork - see the "Ollama" and "LM Studio" sections of the
+[official model card](https://huggingface.co/XHToken/Spark-X2.5-4B) for the build steps.
+
+## Flash Attention
+
+The fork enables FlashAttention automatically for the sliding-window layers of `spark2_5`.
+No extra flag is needed.
+
+## imatrix
+
+All quants below bf16 were made using the imatrix option, computed with `llama-imatrix`
+from this fork. The calibration corpus is the `wikitext-2-raw-v1` train split
+([Salesforce/wikitext](https://huggingface.co/datasets/Salesforce/wikitext)), written to
+disk verbatim and in order by [reproducibility/calibration.py](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/reproducibility/calibration.py).
+The corpus file is included in this repo:
+[calibration_data.txt](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/calibration_data.txt).
+
+The imatrix is available here:
+[Spark-X2.5-4B-imatrix.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-imatrix.gguf).
+
+<details>
+<summary>Calibration details</summary>
+
+```
+{
+  "dataset": "Salesforce/wikitext",
+  "config": "wikitext-2-raw-v1",
+  "split": "train",
+  "rendering": "raw text, no chat template, no special tokens",
+  "chunk_size": 512,
+  "chunks": 128,
+  "tokens_consumed": 65536,
+  "consumption_order": "sequential from file start (deterministic)",
+  "threads": 8,
+  "final_perplexity": "49.3694 +/- 1.09206",
+  "imatrix_format": "GGUF (fork default)",
+  "notes": "corpus is plain prose; a future revision may render tool-calling and reasoning conversations through the chat template, as done by other maintainers"
+}
+```
+
+Reproducibility check: two independent imatrix runs over the same file with the same
+parameters produced identical final perplexity (49.3694 ± 1.09206). Run log:
+[reproducibility/imatrix-run.log](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/reproducibility/imatrix-run.log).
+
+</details>
+
+## Which file should I choose?
+
+<details>
+  <summary>Click here for details</summary>
+
+The first thing to figure out is how big a model you can run. If you want the model
+running as fast as possible, fit the whole thing in VRAM: aim for a file 1-2GB smaller
+than your GPU's total VRAM, leaving room for the KV cache (this model's hybrid attention
+keeps the cache small, which helps). If you want maximum quality and can tolerate partial
+CPU offload, add your system RAM and VRAM together and pick a file 1-2GB smaller than
+that total.
+
+Next, K-quant or I-quant? On NVIDIA (CUDA) and Apple silicon, I-quants (IQX_X) generally
+offer better quality per bit below Q4. On **Intel Arc via Vulkan (Mesa)**, our measurements
+show the opposite for generation speed: K-quants are faster at equal bit-width, while
+I-quants win on file size - see the benchmarks below. Either way, both families were
+validated for coherence in this release.
+
+</details>
+
+## Benchmarks
+
+Measured with `llama-bench` from the same fork commit, Intel Arc B580 (Battlemage,
+Mesa Vulkan driver), `-ngl 99 -t 4`, 3 runs. Raw log:
+[reproducibility/benchmarks_gpu.txt](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/reproducibility/benchmarks_gpu.txt).
+
+| Quant | pp512 t/s | tg128 t/s |
+| ----- | -------: | -------: |
+| Q8_0  | 2039 | 69.5 |
+| Q6_K  | 1844 | 70.5 |
+| Q5_K_M | 1949 | 79.8 |
+| Q4_K_M | 2004 | 93.3 |
+| IQ4_NL | 2079 | 53.9 |
+| Q3_K_M | 1894 | 95.3 |
+| IQ3_M  | 1969 | 76.4 |
+| IQ2_M  | 2048 | 46.4 |
+
+Prompt processing is roughly flat across quants (~2k t/s, compute-bound on the GPU's
+matrix cores). Long-context reading, Q4_K_M with the whole model in VRAM:
+`pp32768 = 309 t/s` (~6GB VRAM total including KV cache). The upstream 1M-token context
+was not validated in this release.
+
+CPU reference readings (i3-12100F, 8 threads, single runs, not formal benchmarks):
+BF16 ≈ 4.8 t/s, Q4_K_M ≈ 13.4 t/s, Q3_K_M ≈ 15.6 t/s, IQ2_M ≈ 10.9 t/s generation.
+
+## Validation
+
+- `test-llama-archs -a spark2_5` from the fork: OK on CPU (NMSE 0.00e+00), Vulkan
+  (8.58e-08) and meta buffers. `Roundtrip: SKIP` is expected for this architecture.
+- Greedy decoding (`--temp 0`) produced token-identical output on CPU and Vulkan GPU,
+  verified on Q4_K_M.
+- Every published quant completed a 700-token coding coherence test at `--temp 0`;
+  Q2_K failed with repetition loops and was excluded (log linked in the files table).
+
+## Reproducibility
+
+Everything needed to rebuild these files bit-for-bit is in the repo:
+
+| File | Content |
+| ---- | ------- |
+| [SHA256SUMS.txt](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/SHA256SUMS.txt) | Checksums of all GGUFs, imatrix and calibration corpus |
+| [Spark-X2.5-4B-imatrix.gguf](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/Spark-X2.5-4B-imatrix.gguf) | The importance matrix itself |
+| [calibration_data.txt](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/calibration_data.txt) | Calibration corpus, verbatim |
+| [reproducibility/calibration.py](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/reproducibility/calibration.py) | Exact corpus generation script |
+| [reproducibility/fork_commit.txt](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/reproducibility/fork_commit.txt) | Fork commit used for conversion, imatrix and quantization |
+| [reproducibility/base_model_revision.txt](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/reproducibility/base_model_revision.txt) | Upstream safetensors revision |
+| [reproducibility/imatrix-run.log](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/reproducibility/imatrix-run.log) | Imatrix run output (tail of final clean run; identical PPL reproduced across runs) |
+| [reproducibility/benchmarks_gpu.txt](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/reproducibility/benchmarks_gpu.txt) | Raw llama-bench output |
+| [reproducibility/q2_k_repetition_loops.log](https://huggingface.co/stornic56/Spark-X2.5-4B-GGUF/blob/main/reproducibility/q2_k_repetition_loops.log) | Evidence for the Q2_K exclusion |
+
+Exact commands:
+
+```
+python convert_hf_to_gguf.py Spark-X2.5-4B --outfile Spark-X2.5-4B-bf16.gguf --outtype bf16
+
+./build/bin/llama-imatrix \
+  -m Spark-X2.5-4B-bf16.gguf \
+  -f calibration_data.txt \
+  -o Spark-X2.5-4B-imatrix.gguf \
+  -c 512 --chunks 128 -t 8
+
+./build/bin/llama-quantize --imatrix Spark-X2.5-4B-imatrix.gguf \
+  Spark-X2.5-4B-bf16.gguf Spark-X2.5-4B-Q4_K_M.gguf Q4_K_M
+```
+
+## ARM/AVX information
+
+llama.cpp automatically repacks weights into an interleaved layout at load time for
+faster inference on ARM and AVX machines, covering Q4_0, IQ4_NL and most K-quants.
+No special quant choice is needed for CPU inference.
+
+## Credits
+
+Thanks to the XHToken/SparkLLM team for releasing the model and maintaining the llama.cpp
+fork, and to bartowski for the quantization card format this repo follows.
+
+## License & attribution
+
+The upstream model is released under Apache License 2.0; these derivative quantizations
+inherit it. Preserve upstream attribution when redistributing. This is an unofficial
+community release, not endorsed by XHToken/SparkLLM.

SHA256SUMS.txt

@@ -0,0 +1,11 @@
+2ff41881527d095dbc02fe0c9b8e6ecd221dfe28b242d1f03dc5592d1b39fbb2  Spark-X2.5-4B-bf16.gguf
+ec53c7b34670406e293675002a16fce012ceb31987ec062da6e97ebb43da93db  Spark-X2.5-4B-imatrix.gguf
+1d5ebeb9baa9101a4e4096055c08414051cf9accaf5ed105dd65e8b0632fd53e  Spark-X2.5-4B-IQ2_M.gguf
+46933a5cfb07b7c0652cdfe920c10790e47a35413d5ab48de16795ad59026f2b  Spark-X2.5-4B-IQ3_M.gguf
+aae5f3748fa3c8d37cee4d969ea283c15d616123b759667f7261bbdfad2a8e3d  Spark-X2.5-4B-IQ4_NL.gguf
+43937c350708aad690184b5ab1256e7ea55865a101e7e7e8ff52bc5e5e31c826  Spark-X2.5-4B-Q3_K_M.gguf
+dc08c21953fbdf797d77fbe7cdecb353d8dc6cb3517c2f911e209bb37633726a  Spark-X2.5-4B-Q4_K_M.gguf
+ae851906d46fb622aa4cebea6fa9ca78b213d4258c85b60e943d2dace5a15f6f  Spark-X2.5-4B-Q5_K_M.gguf
+8c96b615b376c1178e2e284a389a9fca24bc4e40430ed735b9c88cef921453b7  Spark-X2.5-4B-Q6_K.gguf
+092a263df8c891cdddd98b14b9ed71e44bb84643049fbfe656fb682b71d316c6  Spark-X2.5-4B-Q8_0.gguf
+fc1dd2874bf42e945db9bd38e856ae18010f5c088e7446c706c5ba2f39310ee3  calibration_data.txt

Spark-X2.5-4B-IQ2_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d5ebeb9baa9101a4e4096055c08414051cf9accaf5ed105dd65e8b0632fd53e
+size 1636324096

Spark-X2.5-4B-IQ3_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46933a5cfb07b7c0652cdfe920c10790e47a35413d5ab48de16795ad59026f2b
+size 2043300096

Spark-X2.5-4B-IQ4_NL.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aae5f3748fa3c8d37cee4d969ea283c15d616123b759667f7261bbdfad2a8e3d
+size 2476361216

Spark-X2.5-4B-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43937c350708aad690184b5ab1256e7ea55865a101e7e7e8ff52bc5e5e31c826
+size 2165197056

Spark-X2.5-4B-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc08c21953fbdf797d77fbe7cdecb353d8dc6cb3517c2f911e209bb37633726a
+size 2600224256

Spark-X2.5-4B-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae851906d46fb622aa4cebea6fa9ca78b213d4258c85b60e943d2dace5a15f6f
+size 2977895936

Spark-X2.5-4B-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c96b615b376c1178e2e284a389a9fca24bc4e40430ed735b9c88cef921453b7
+size 3379172096

Spark-X2.5-4B-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:092a263df8c891cdddd98b14b9ed71e44bb84643049fbfe656fb682b71d316c6
+size 4375021056

Spark-X2.5-4B-bf16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ff41881527d095dbc02fe0c9b8e6ecd221dfe28b242d1f03dc5592d1b39fbb2
+size 8229920032

Spark-X2.5-4B-imatrix.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec53c7b34670406e293675002a16fce012ceb31987ec062da6e97ebb43da93db
+size 3572672

calibration_data.txt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc1dd2874bf42e945db9bd38e856ae18010f5c088e7446c706c5ba2f39310ee3
+size 10914845

reproducibility/base_model_revision.txt

@@ -0,0 +1 @@
+5e10fcc0286756aebf7c41dc52c1e42d95c70281

reproducibility/benchmarks_gpu.txt

@@ -0,0 +1,56 @@
+═══ Q8_0 ═══
+| model                          |       size |     params | backend    | ngl |            test |                  t/s |
+| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
+| spark2_5 ?B Q8_0               |   4.07 GiB |     4.11 B | Vulkan     |  99 |           pp512 |      2039.45 ± 11.89 |
+| spark2_5 ?B Q8_0               |   4.07 GiB |     4.11 B | Vulkan     |  99 |           tg128 |         69.49 ± 0.11 |
+
+build: 4a3635c32 (10514)
+═══ Q6_K ═══
+| model                          |       size |     params | backend    | ngl |            test |                  t/s |
+| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
+| spark2_5 ?B Q6_K               |   3.14 GiB |     4.11 B | Vulkan     |  99 |           pp512 |       1843.87 ± 5.20 |
+| spark2_5 ?B Q6_K               |   3.14 GiB |     4.11 B | Vulkan     |  99 |           tg128 |         70.48 ± 0.02 |
+
+build: 4a3635c32 (10514)
+═══ Q5_K_M ═══
+| model                          |       size |     params | backend    | ngl |            test |                  t/s |
+| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
+| spark2_5 ?B Q5_K - Medium      |   2.77 GiB |     4.11 B | Vulkan     |  99 |           pp512 |       1949.32 ± 9.95 |
+| spark2_5 ?B Q5_K - Medium      |   2.77 GiB |     4.11 B | Vulkan     |  99 |           tg128 |         79.81 ± 0.64 |
+
+build: 4a3635c32 (10514)
+═══ Q4_K_M ═══
+| model                          |       size |     params | backend    | ngl |            test |                  t/s |
+| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
+| spark2_5 ?B Q4_K - Medium      |   2.42 GiB |     4.11 B | Vulkan     |  99 |           pp512 |       2003.62 ± 1.34 |
+| spark2_5 ?B Q4_K - Medium      |   2.42 GiB |     4.11 B | Vulkan     |  99 |           tg128 |         93.31 ± 0.53 |
+
+build: 4a3635c32 (10514)
+═══ IQ4_NL ═══
+| model                          |       size |     params | backend    | ngl |            test |                  t/s |
+| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
+| spark2_5 ?B IQ4_NL - 4.5 bpw   |   2.30 GiB |     4.11 B | Vulkan     |  99 |           pp512 |       2079.09 ± 2.51 |
+| spark2_5 ?B IQ4_NL - 4.5 bpw   |   2.30 GiB |     4.11 B | Vulkan     |  99 |           tg128 |         53.87 ± 0.04 |
+
+build: 4a3635c32 (10514)
+═══ Q3_K_M ═══
+| model                          |       size |     params | backend    | ngl |            test |                  t/s |
+| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
+| spark2_5 ?B Q3_K - Medium      |   2.01 GiB |     4.11 B | Vulkan     |  99 |           pp512 |      1893.78 ± 41.34 |
+| spark2_5 ?B Q3_K - Medium      |   2.01 GiB |     4.11 B | Vulkan     |  99 |           tg128 |         95.29 ± 0.10 |
+
+build: 4a3635c32 (10514)
+═══ IQ3_M ═══
+| model                          |       size |     params | backend    | ngl |            test |                  t/s |
+| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
+| spark2_5 ?B IQ3_S mix - 3.66 bpw |   1.90 GiB |     4.11 B | Vulkan     |  99 |           pp512 |       1969.10 ± 7.31 |
+| spark2_5 ?B IQ3_S mix - 3.66 bpw |   1.90 GiB |     4.11 B | Vulkan     |  99 |           tg128 |         76.39 ± 0.48 |
+
+build: 4a3635c32 (10514)
+═══ IQ2_M ═══
+| model                          |       size |     params | backend    | ngl |            test |                  t/s |
+| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
+| spark2_5 ?B IQ2_M - 2.7 bpw    |   1.52 GiB |     4.11 B | Vulkan     |  99 |           pp512 |       2047.56 ± 7.56 |
+| spark2_5 ?B IQ2_M - 2.7 bpw    |   1.52 GiB |     4.11 B | Vulkan     |  99 |           tg128 |         46.36 ± 0.01 |
+
+build: 4a3635c32 (10514)

reproducibility/calibration.py

@@ -0,0 +1,5 @@
+from datasets import load_dataset
+with open("calibration_data.txt", "w", encoding="utf-8") as f:
+    ds = load_dataset("Salesforce/wikitext", "wikitext-2-raw-v1", split="train")
+    for ex in ds:
+        f.write(ex["text"])

reproducibility/fork_commit.txt

@@ -0,0 +1 @@
+4a3635c32fc9f044c2bde9ebeabf50c7e1ec5991

reproducibility/imatrix-run.log

@@ -0,0 +1,21 @@
+# Command: ./build/bin/llama-imatrix -m Spark-X2.5-4B-BF16.gguf -f calibration_data.txt -o Spark-X2.5-4B-imatrix.gguf -c 512 --chunks 128 -t 8
+# Note: terminal-visible tail of the final (clean) run; early chunks scrolled off.
+# A previous full run produced an identical final PPL (49.3694 +/- 1.09206), confirming determinism.
+# --- run output below ---
+[89]51.5243,[90]51.4395,[91]51.7192,[92]51.8162,[93]52.3294,[94]52.1363,[95]51.7138,[96]51.5093,23.32.001.298 W
+save_imatrix: saving imatrix using GGUF format with a different suffix than .gguf
+23.32.001.316 W save_imatrix: if you want the previous imatrix format, use --output-format dat
+
+[97]51.4544,[98]52.8739,[99]52.7433,[100]52.5876,[101]53.3877,[102]52.9241,[103]52.1672,[104]51.4715,[105]50.7402,[106]51.1758,[107]51.6608,[108]50.9367,25.54.279.108 W
+save_imatrix: saving imatrix using GGUF format with a different suffix than .gguf
+25.54.279.127 W save_imatrix: if you want the previous imatrix format, use --output-format dat
+
+[109]51.5952,[110]52.0192,[111]51.6026,[112]51.1949,[113]50.7800,[114]51.0304,[115]50.4734,[116]51.0291,28.15.994.809 W
+save_imatrix: saving imatrix using GGUF format with a different suffix than .gguf
+28.15.994.812 W save_imatrix: if you want the previous imatrix format, use --output-format dat
+
+[117]51.2028,[118]51.4231,[119]50.3916,[120]50.9860,[121]50.5352,[122]50.0576,[123]49.6900,[124]49.4488,[125]49.1098,[126]49.0112,[127]48.7618,[128]49.3694,
+Final estimate: PPL = 49.3694 +/- 1.09206
+30.23.999.092 W
+save_imatrix: saving imatrix using GGUF format with a different suffix than .gguf
+30.23.999.093 W save_imatrix: if you want the previous imatrix format, use --output-format dat

reproducibility/q2_k_repetition_loops.log

@@ -0,0 +1,129 @@
+for type in Q2_K IQ2_M; do
+  echo "═════════ $type (test extendido) ═════════"
+  ./build/bin/llama-cli -m ../Spark-X2.5-4B-${type}.gguf \
+    -c 2048 -ngl 0 -t 8 --temp 0 --seed 1 -n 700 \
+    -cnv -st --jinja --simple-io \
+    -p "Write a Python function that checks if a number is prime:" 2>&1 | tail -60
+done
+═════════ Q2_K (test extendido) ═════════
+    return True
+
+But we should also consider that the number might be negative, zero, or 1. So we can add a check at the beginning.
+
+Alternatively, we can use a more efficient approach, but for this problem, a simple loop is fine.
+
+We need to write a function that checks if a number is prime.
+
+Let's write the function with proper handling.
+
+We can also use a helper function to check for divisibility.
+
+We should also consider that the number might be negative, zero, or 1. So we can add a check at the beginning.
+
+We can write:
+
+def is_prime(n):
+    if n < 2:
+        return False
+    if n == 2 or n == 3:
+        return True
+    for i in range(2, int(n**0.5) + 1):
+        if n % i == 0:
+            return False
+    return True
+
+But we should also consider that the number might be negative, zero, or 1. So we can add a check at the beginning.
+
+We can also use a more efficient approach, but for this problem, a simple loop is fine.
+
+We should also consider that the number might be negative, zero, or 1. So we can add a check at the beginning.
+
+We can write:
+
+def is_prime(n):
+    if n < 2:
+        return False
+    if n == 2 or n == 3:
+        return True
+    for i in range(2, int(n**0.5) + 1):
+        if n % i == 0:
+            return False
+    return True
+
+But we should also consider that the number might be negative, zero, or 1. So we can add a check at the beginning.
+
+We can also use a more efficient approach, but for this problem, a simple loop is fine.
+
+We should also consider that the number might be negative, zero, or 1. So we can add a check at the beginning.
+
+We can write:
+
+def is_prime(n):
+    if n < 2:
+        return False
+
+[ Prompt: 36,6 t/s | Generation: 18,2 t/s ]
+
+
+Exiting...
+═════════ IQ2_M (test extendido) ═════════
+  /clear              clear the chat history
+  /read <file>        add a text file
+  /glob <pattern>     add text files using globbing pattern
+
+
+
+> Write a Python function that checks if a number is prime:
+
+[Start thinking]
+
+We are going to write a function that checks if a number is prime.
+ Steps:
+ 1. If the number is less than 2, it is not prime. Return False.
+ 2. Check for divisors from 2 to the square root of the number (inclusive). If we find a divisor, return False.
+ 3. If no divisors are found, return True.
+
+ We can optimize by checking only odd numbers after 2, but note that we must check 2 and then odd numbers.
+
+ Alternatively, we can write a loop that checks from 2 to n-1, but we can break early when the divisor squared exceeds n.
+
+ Let's write the function with these steps.
+[End thinking]
+
+```python
+def is_prime(n):
+    """
+    Check if a given number is prime.
+
+    Parameters:
+    n (int): The number to check for primality.
+
+    Returns:
+    bool: True if n is prime, False otherwise.
+    """
+    if n < 2:  # 0 and 1 are not prime
+        return False
+    if n == 2:  # 2 is prime
+        return True
+    if n % 2 == 0:  # Even numbers other than 2 are not prime
+        return False
+
+    # Check odd numbers from 3 to the square root of n (inclusive)
+    i = 3
+    while i * i <= n:
+        if n % i == 0:
+            return False
+        i += 2
+    return True
+```
+
+**Explanation:**
+1. **Base cases:** If `n` is less than 2, it's not prime. If `n` is 2, it's prime.
+2. **Even numbers:** If `n` is even and not 2, it's not prime.
+3. **Check odd divisors:** We check odd numbers starting from 3 up to the square root of `n` (inclusive). If `n` is divisible by any odd number, it's not prime.
+4. **Efficiency:** By checking only odd numbers and stopping at the square root, the function avoids unnecessary checks and improves performance.
+
+[ Prompt: 14,8 t/s | Generation: 10,8 t/s ]
+
+
+Exiting...