Text Generation
GGUF
llama.cpp
qlora
agentic
coding
reasoning
smollm3
conversational
AnkitAI commited on
Commit
fe20e86
Β·
verified Β·
1 Parent(s): 53d9f68

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Parable-SmolLM3-3B-Claude-Fable-5-GGUF-F16.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Parable-SmolLM3-3B-Claude-Fable-5-GGUF-F16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:86930a18b35020d0e90e945c1a44f6704972fc32943ed190c121ce18953d57e4
3
+ size 6158339584
Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb88b166fe36aa59771db25bdae54e4b2fa756b165b4a32209f92c59825433b6
3
+ size 1915305472
Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20e863ea3b278932a1713a7b176c5d6215db2cc6f0b7fc50e5064a577e3a5c96
3
+ size 2213756416
Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e3c97b0daf8ebbe335b54fa09160d74fc1b4b9b33a93b8f507daa17dc0109013
3
+ size 2530860544
Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ac4277c463ad245c880545480ef0983cd416ad24de78e9affe1a8e9e149e51d
3
+ size 3275574784
README.md ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: HuggingFaceTB/SmolLM3-3B
4
+ datasets:
5
+ - Glint-Research/Fable-5-traces
6
+ - Roman1111111/gpt5.5-terminal
7
+ pipeline_tag: text-generation
8
+ library_name: llama.cpp
9
+ tags:
10
+ - gguf
11
+ - qlora
12
+ - agentic
13
+ - coding
14
+ - reasoning
15
+ - smollm3
16
+ ---
17
+
18
+ # Parable-SmolLM3-3B-Claude-Fable-5-GGUF
19
+
20
+ ![SmolLM3-3B](banner.svg)
21
+
22
+ Part of the **Parable** series: small local LLMs fine-tuned on genuine agent
23
+ traces. This is HuggingFaceTB/SmolLM3-3B tuned on real Claude Fable 5 agent
24
+ transcripts so its step-by-step reasoning voice carries into local use.
25
+
26
+ Full-precision weights: [Parable-SmolLM3-3B-Claude-Fable-5](https://huggingface.co/AnkitAI/Parable-SmolLM3-3B-Claude-Fable-5)
27
+
28
+ ## Files
29
+
30
+ | File | Quant | Size | |
31
+ |---|---|---|---|
32
+ | Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf | Q4_K_M | 1.9 GB | **recommended default** |
33
+ | Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q5_K_M.gguf | Q5_K_M | 2.2 GB | |
34
+ | Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q6_K.gguf | Q6_K | 2.5 GB | |
35
+ | Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q8_0.gguf | Q8_0 | 3.3 GB | |
36
+ | Parable-SmolLM3-3B-Claude-Fable-5-GGUF-F16.gguf | F16 | 6.2 GB | for re-quantizing |
37
+
38
+ ## Usage
39
+
40
+ ```bash
41
+ llama-cli -m Parable-SmolLM3-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf \
42
+ -c 4096 -p "Write a python function that reverses a string." --temp 0.6
43
+ ```
44
+
45
+ SmolLM3 is supported by current llama.cpp releases (brew, Ollama, LM Studio
46
+ builds included); no special build is required.
47
+
48
+ Output begins with a `<think>...</think>` reasoning block, then the answer.
49
+ If you are building on top of this model, parse and strip the think block
50
+ before showing text to end users. The chat template identifies the model as
51
+ "Parable, a coding assistant that reasons before it answers."
52
+
53
+ ## Model details
54
+
55
+ - **Base:** [HuggingFaceTB/SmolLM3-3B](https://huggingface.co/HuggingFaceTB/SmolLM3-3B) (3B, Apache-2.0, 64k context)
56
+ - **Method:** MLX QLoRA on a 4-bit quantized base, 16 layers adapted,
57
+ 6.7M trainable parameters (0.218%)
58
+ - **Data:** 4,076 training rows from real Claude Fable 5 agent-session
59
+ traces plus gpt5.5-terminal transcripts, prepared at a 4,096-token window
60
+ (268 over-length rows dropped; 226/226 rows held out for validation/test)
61
+ - **Schedule:** 1,200-iteration budget across a paused-and-resumed run;
62
+ best checkpoint selected on validation loss (iteration 200 of the final
63
+ segment, val 1.154)
64
+
65
+ ## Evaluation
66
+
67
+ | | Held-out trace test loss |
68
+ |---|---|
69
+ | SmolLM3-3B base | 1.889 |
70
+ | **This model** | **1.115** |
71
+
72
+ The tuned model fits the Fable-5 reasoning distribution 41% better by
73
+ held-out loss on a 226-row test split never seen in training. That is the
74
+ honest headline for what this fine-tune does; we do not claim general
75
+ benchmark gains.
76
+
77
+ This lane trains on trace data without a replay mix, so impact on general
78
+ coding benchmarks is unmeasured here. The series' technical report
79
+ (DOI: [10.5281/zenodo.21676407](https://doi.org/10.5281/zenodo.21676407))
80
+ documents why that matters and what replay does about it.
81
+
82
+ ## Limitations
83
+
84
+ - Training ran on a 4-bit quantized base (16 GB M1 constraint); the F16
85
+ merge and higher quants cannot exceed 4-bit-base quality.
86
+ - Modest scale: one seed, loss-based evaluation, no external benchmark run
87
+ for this model yet.
88
+ - Not trained for: multi-file repo navigation, vision, non-English.
89
+ - Inherits SmolLM3-3B's knowledge cutoff. Treat generated commands as
90
+ drafts to review.
91
+
92
+ ## Quantization
93
+
94
+ Quantized with llama.cpp `llama-quantize` from the F16 merge.
95
+
96
+ ## Provenance & licensing
97
+
98
+ Fine-tuned from HuggingFaceTB/SmolLM3-3B (Apache-2.0). Training data:
99
+ [Glint-Research/Fable-5-traces](https://huggingface.co/datasets/Glint-Research/Fable-5-traces)
100
+ (AGPL-3.0) and
101
+ [Roman1111111/gpt5.5-terminal](https://huggingface.co/datasets/Roman1111111/gpt5.5-terminal)
102
+ (MIT). Because those traces originate from third-party assistants, the
103
+ providers' terms may apply to downstream training and distillation. If you
104
+ plan to build on this model commercially, confirm your use aligns with those
105
+ terms.
106
+
107
+ ## Citation
108
+
109
+ ```bibtex
110
+ @misc{aglawe2026parable,
111
+ author = {Aglawe, Ankit},
112
+ title = {Agent-Trace Fine-Tuning of Small Language Models under Constrained Compute},
113
+ year = {2026},
114
+ doi = {10.5281/zenodo.21676407},
115
+ url = {https://doi.org/10.5281/zenodo.21676407}
116
+ }
117
+ ```
118
+
119
+ ## Acknowledgements
120
+
121
+ The SmolLM3 team at Hugging Face for the base model; Glint-Research and
122
+ Roman1111111 for the trace datasets; empero-ai for the recipe this series
123
+ iterates on.
banner.svg ADDED