textilelabs commited on
Commit
63d069c
·
verified ·
1 Parent(s): 60a131c

Upload 16 files

Browse files
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ banner.jpg filter=lfs diff=lfs merge=lfs -text
37
+ logo.jpg filter=lfs diff=lfs merge=lfs -text
38
+ loom-crucible-preview-f16.gguf filter=lfs diff=lfs merge=lfs -text
ATTRIBUTION.md ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Attribution
2
+
3
+ Loom Crucible Preview was trained from scratch (random-init lineage — grown from Textile Labs'
4
+ own Loom Tapestry 3, which was grown from Loom Weave 3; no third-party checkpoint).
5
+ Grounded-reading rows embed real encyclopedic prose. Training data, all permissively licensed:
6
+
7
+ | slice | source | licence |
8
+ |---|---|---|
9
+ | grounded reading + "the answer isn't in this passage" | SQuAD 2.0 | CC BY-SA 4.0 |
10
+ | multi-hop grounded reading | HotpotQA | CC BY-SA 4.0 |
11
+ | trivia reading (answer sentence from the entity's article) | TriviaQA | Apache 2.0 |
12
+ | when to reach for a tool | MASSIVE / CLINC150 | CC BY 4.0 / CC BY 3.0 |
13
+ | instruction following | databricks-dolly-15k | CC BY-SA 3.0 |
14
+ | multi-turn dialogue | OpenAssistant OASST1 | Apache 2.0 |
15
+ | tokenizer coverage of real prose | HuggingFaceFW/fineweb-edu | ODC-By 1.0 |
16
+ | encyclopedic passages | Wikipedia | CC BY-SA |
17
+ | identity, limits, warmth, attribution | Textile Labs | — |
18
+
19
+ OASST conversations in which the original assistant named itself were removed. No language model
20
+ wrote any training query. No real user data was used. Model weights: MIT.
LICENSE ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Textile Labs
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy of this model
6
+ and associated files (the "Model"), to deal in the Model without restriction, including
7
+ without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
8
+ and/or sell copies of the Model, and to permit persons to whom the Model is furnished to do
9
+ so, subject to the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be included in all copies or
12
+ substantial portions of the Model.
13
+
14
+ THE MODEL IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
15
+ BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
16
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
18
+ CONNECTION WITH THE MODEL OR THE USE OR OTHER DEALINGS IN THE MODEL.
Modelfile ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ FROM ./loom-crucible-preview-f16.gguf
2
+ TEMPLATE "<tools:off>\n<user>\n{{ .Prompt }}\n<|eot|>\n<loom>\n"
3
+ PARAMETER stop "<|eot|>"
4
+ PARAMETER stop "<user>"
5
+ PARAMETER stop "<result>"
6
+ PARAMETER temperature 0.7
7
+ PARAMETER top_k 40
8
+ PARAMETER repeat_penalty 1.0
9
+ PARAMETER num_predict 96
README.md CHANGED
@@ -1,3 +1,209 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ language: en
4
+ library_name: transformers
5
+ pipeline_tag: text-generation
6
+ tags:
7
+ - tiny-model
8
+ - llama
9
+ - from-scratch
10
+ - model-growth
11
+ - conversational
12
+ - multi-turn
13
+ - tool-use
14
+ - agent-harness
15
+ - retrieval-augmented
16
+ - question-answering
17
+ - unanswerable-questions
18
+ - attribution
19
+ - humble-ai
20
+ - small-language-model
21
+ - muon
22
+ - gguf
23
+ - text-generation-inference
24
+ widget:
25
+ - text: "<tools:off>\n<user>\nwho are you\n<|eot|>\n<loom>\n"
26
+ example_title: "Identity"
27
+ - text: "<tools:on>\n<user>\nwho wrote dracula\n<|eot|>\n<loom>\n"
28
+ example_title: "Reads real prose"
29
+ - text: "<tools:off>\n<user>\nwhat did i eat yesterday\n<|eot|>\n<loom>\n"
30
+ example_title: "Limits"
31
  ---
32
+
33
+ <div align="center">
34
+ <img src="banner.jpg" alt="Loom Crucible Preview" width="520">
35
+ </div>
36
+
37
+ # Loom Crucible Preview
38
+
39
+ <img src="logo.jpg" alt="" width="20" height="20" style="border-radius:4px;vertical-align:middle;margin-right:6px;"> **155.0M parameters · 52 layers · 1024 context · Textile Labs**
40
+
41
+ The biggest Loom by far, and a **preview** of how far our growth method can push a small,
42
+ disciplined model. We took [Loom Tapestry 3](https://huggingface.co/textilelabs/Loom-Tapestry-3)
43
+ (69.2M) and **grew it in both directions at once**, wider (384 → 512) and deeper (40 → 52 layers).
44
+ The growth is function-preserving: the 155M model started out producing exactly what Tapestry 3
45
+ produced, then trained for 12 hours on a rebuilt corpus.
46
+
47
+ It calls itself **Loom Crucible**. It reads what a search brings back and answers from it, says
48
+ when it looked something up, says when it can't know something about you, and stays a Loom:
49
+ plain, brief, and not a story generator.
50
+
51
+ Trained from-scratch lineage (no third-party checkpoint), on a Kaggle dual-T4. Runs offline via
52
+ Ollama; llama.cpp-compatible.
53
+
54
+ ```
55
+ you who wrote dracula
56
+ Loom Crucible <lookup>wrote dracula</lookup>
57
+ harness ← Dracula is an 1897 Gothic horror novel by Irish author Bram Stoker.
58
+ Loom Crucible Bram Stoker. I had to look that up.
59
+ ```
60
+
61
+ ## What "Preview" means
62
+ Crucible clearly beats every earlier Loom on its core skills: **reading a passage correctly
63
+ (9/10 vs Tapestry 3's 4/10)** and the **acceptance battery (125/133)**. It is **not yet better
64
+ end to end on live Wikipedia (10/20 held-out vs Tapestry 3's 12/20)**. On that test the limit is
65
+ now the search step, not the reading step: when the search found the answer, both models got it
66
+ right about 85% of the time. A better harness is the next release, and it helps every Loom.
67
+
68
+ ## How it was grown
69
+ Cold-training a big model on little data fails. Instead we grow a trained small one:
70
+
71
+ - **Wider, 384 → 512.** New channels start at zero so they cannot disturb the old ones. Two
72
+ details made this exact rather than approximate. First, the 2 new attention heads per layer
73
+ are placed *inside* their key/value groups, because appending them silently re-pairs the old
74
+ heads with the wrong keys. Second, every RMSNorm weight is scaled by √(384/512) and ε by
75
+ 384/512, because zero-padding changes the norm.
76
+ - **Deeper, 40 → 52.** 12 new layers are inserted as no-ops (zeroed output projections).
77
+
78
+ At step 0 the 155M model matched Tapestry 3 to 3e-4 in the logits and 1e-6 in the loss. All
79
+ twelve hours went into getting better, not starting over.
80
+
81
+ ## Measured behaviour
82
+
83
+ **The family, same harness, 2026-09:**
84
+
85
+ | model | params | battery /133 | live search (held-out) | reads real prose |
86
+ |---|---:|---:|---:|---|
87
+ | Loom Tapestry 2 | 22.8M | 107 (80.5%) | — | curated only |
88
+ | Loom Tapestry 3 Flash | 7.18M | 112 (84.2%) | 3/20 | curated only |
89
+ | Loom Spark 3 Flash | 7.18M | 119 (89.5%) | 5/20 | curated only |
90
+ | Loom Spark 3 | 12.2M | 120 (90.2%) | 7/20 | curated only |
91
+ | Loom Weave 3 | 31.5M | 120 (90.2%) | 6/20 | yes |
92
+ | Loom Tapestry 3 | 69.2M | 123 (92.5%) | **12/20** | yes + multi-hop |
93
+ | **Loom Crucible Preview** | **155.0M** | **125 (94.0%)†** | 10/20 | **yes, best reader** |
94
+
95
+ † Crucible's score is the first on a fully clean held-out battery. While building its corpus we
96
+ found that some battery prompts had leaked into the training data used since Tapestry 3. We
97
+ removed them for Crucible, so earlier scores are somewhat optimistic by comparison.
98
+
99
+ **The acceptance battery, row by row:**
100
+
101
+ | row | Tapestry 3 | **Crucible** |
102
+ |---|---:|---:|
103
+ | A · says its own name | 12/12 | **12/12** |
104
+ | B · its own name under rough typing | 11/12 | 11/12 |
105
+ | C · 5-turn conversation stays on thread | 5/5 | 5/5 |
106
+ | D · answers from a search result | 5/5 | 4/5 |
107
+ | E · follow-up from the same result | 2/5 | **5/5** |
108
+ | F · says it looked, after a lookup | 5/5 | 4/5 |
109
+ | G · **never** claims a lookup it didn't make | 16/16 | **16/16** |
110
+ | H · admits what it can't know about you | 8/8 | 6/8 |
111
+ | I · says when a result doesn't contain the answer | 2/5 | **3/5** |
112
+ | J · never leaks a search tag with tools off | 28/28 | 28/28 |
113
+ | K · stops on its own | 12/12 | 12/12 |
114
+ | L · searches when it should, not for your private things | 17/20 | **19/20** |
115
+ | **total** | **123/133** | **125/133** |
116
+
117
+ **Held-out reading bench** (hand-written passages with distractors, e.g. a country's capital
118
+ next to its largest city, a start year next to an end year; none in any training data):
119
+
120
+ | | Tapestry 3 | **Crucible** |
121
+ |---|---:|---:|
122
+ | answers correctly from the passage | 4/10 | **9/10** |
123
+ | says so when an on-topic passage lacks the answer | 0/10 | **3/10** |
124
+ | says so when the passage is off-topic | 0/8 | **6/8** |
125
+ | refuses a passage that *does* hold the answer | 0 | **0** |
126
+ | with tools off, won't answer a famous fact from memory | 3/8 | 3/8 |
127
+
128
+ **End to end**, everyday questions it had never seen, live Wikipedia, model writing its own
129
+ query, scored on the final answer: **tuning 11/20, held-out 10/20** (Tapestry 3: 12/20, 12/20).
130
+ When the search found the answer, Crucible got it right 10 times out of 12.
131
+
132
+ ## Read this before you use it
133
+ Every point measured.
134
+ - **It is not better than Tapestry 3 end to end yet.** It reads better, but live results are
135
+ capped by the search step. Run `harness.py --show` to see the sentence it read.
136
+ - **When a search comes back without the answer, it usually answers anyway.** Given an
137
+ on-topic passage that lacks the fact, it says so only 3 times in 10. If the passage holds
138
+ something of the right *type* (a number for "how tall", a year for "when did it end"), it
139
+ tends to grab that. Trust the sentence shown by `--show`, not just the reply.
140
+ - **With tools off, it sometimes answers famous facts from half-memory** (3/8 held). Those
141
+ answers are often wrong. Keep tools on for facts.
142
+ - **It once refused a passage that did hold the answer** (1 of 5 battery reading rows).
143
+ - **It is not a calculator.** Arithmetic is out of scope; use a tool.
144
+ - **Harness search is Wikipedia only**: no time, weather, news or prices.
145
+ - **Plain, brief voice by design.** It is a small reader and assistant, not a chat stylist.
146
+
147
+ What it does reliably: knows what it is, **stops on its own**, holds a conversation, **answers
148
+ follow-ups from the same result (5/5)**, **never claims a lookup it didn't make (16/16)**, and
149
+ **reads a supplied passage better than any earlier Loom**.
150
+
151
+ ## Usage — the harness
152
+ ```bash
153
+ python3 harness.py "who wrote dracula"
154
+ python3 harness.py --show "how tall is ben nevis"
155
+ python3 harness.py --no-tools "who are you"
156
+ ```
157
+ Stdlib + `certifi`. Wikipedia needs no API key.
158
+
159
+ ## Usage — Ollama
160
+ ```bash
161
+ ollama run hf.co/textilelabs/Loom-Crucible-Preview "who are you"
162
+ ```
163
+ `template` and `params` are read automatically.
164
+
165
+ ## Usage — transformers
166
+ ```python
167
+ import torch
168
+ from transformers import AutoTokenizer, AutoModelForCausalLM
169
+ tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Crucible-Preview")
170
+ model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Crucible-Preview").eval()
171
+ eot = tok.convert_tokens_to_ids("<|eot|>")
172
+ def ask(message, tools=False):
173
+ p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
174
+ ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
175
+ with torch.no_grad():
176
+ out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
177
+ pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
178
+ return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>","").strip()
179
+ ```
180
+ Prompt format is exact: `<tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n`.
181
+
182
+ ## How it was built
183
+ | | |
184
+ |---|---|
185
+ | architecture | Llama — 52 layers × 512d, GQA (8 heads / 2 KV), SwiGLU, RoPE, tied embeddings |
186
+ | parameters | 154,980,864 |
187
+ | grown from | Loom Tapestry 3 (40 × 384) by function-preserving width + depth growth |
188
+ | context | 1,024 |
189
+ | vocabulary | 16,384 custom BPE |
190
+ | optimiser | Muon (LR 0.025) on 2D hidden matrices, AdamW on embeddings and norms |
191
+ | loss | masked to the reply tokens; whole conversations packed per block |
192
+ | corpus | 275,788 conversations: Loom curriculum + SQuAD 2.0 (incl. unanswerable) + HotpotQA + TriviaQA |
193
+ | training | 6 h grown from Tapestry 3, then 6 h continued on the rebuilt corpus · best-validation checkpoint |
194
+ | hardware | Kaggle dual T4 (GPU) |
195
+ | decoding | ship with `repeat_penalty 1.0`: higher values corrupt copying from the passage |
196
+
197
+ ## Files
198
+ ```
199
+ config.json / model.safetensors the model
200
+ tokenizer.json / tokenizer_config.json custom BPE tokenizer, 16,384 tokens
201
+ loom-crucible-preview-f16.gguf for Ollama / llama.cpp (lookup tags USER_DEFINED)
202
+ harness.py runnable search harness
203
+ template / params read automatically by `ollama run hf.co/...`
204
+ Modelfile for building locally
205
+ ATTRIBUTION.md required credits for the training corpora
206
+ ```
207
+
208
+ ## License
209
+ Model: MIT. Training data retains its original licences and attribution.
banner.jpg ADDED

Git LFS Details

  • SHA256: 3972b3698ad39827c8127ab5f75a90bab6374dfa573715c42555d5110ce26ff3
  • Pointer size: 132 Bytes
  • Size of remote file: 3.08 MB
config.json ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "LlamaForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": null,
8
+ "dtype": "float32",
9
+ "eos_token_id": 0,
10
+ "head_dim": 64,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 512,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 1408,
15
+ "max_position_embeddings": 1024,
16
+ "mlp_bias": false,
17
+ "model_type": "llama",
18
+ "num_attention_heads": 8,
19
+ "num_hidden_layers": 52,
20
+ "num_key_value_heads": 2,
21
+ "pad_token_id": 1,
22
+ "pretraining_tp": 1,
23
+ "rms_norm_eps": 7.500000000000001e-06,
24
+ "rope_parameters": {
25
+ "rope_theta": 10000.0,
26
+ "rope_type": "default"
27
+ },
28
+ "tie_word_embeddings": true,
29
+ "transformers_version": "5.0.0",
30
+ "use_cache": true,
31
+ "vocab_size": 16384,
32
+ "torch_dtype": "float32"
33
+ }
generation_config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "eos_token_id": 0,
4
+ "output_attentions": false,
5
+ "output_hidden_states": false,
6
+ "pad_token_id": 1,
7
+ "transformers_version": "5.0.0",
8
+ "use_cache": false
9
+ }
harness.py ADDED
@@ -0,0 +1,216 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Loom harness — the search half of Loom Crucible Preview.
3
+
4
+ The model never searches. It decides a lookup is needed and writes the query:
5
+
6
+ <lookup>france</lookup>
7
+
8
+ This script does the rest: searches Wikipedia, finds the ONE sentence most likely to
9
+ hold the answer, hands it back as a <result>, and lets the model answer from it.
10
+
11
+ python3 harness.py "what's the capital of france"
12
+ python3 harness.py # interactive
13
+ python3 harness.py --no-tools "who are you"
14
+ python3 harness.py --show "who wrote hamlet" # print what was searched and read
15
+
16
+ How it finds the answer, and why each step exists (all measured on live questions):
17
+ * searches the model's query AND the subject it can see in your question —
18
+ "whats the capital of france" searched as-is returns "Capital city" and "Das Kapital"
19
+ * prefers the real article over lists, films, albums and disambiguation pages
20
+ * reads the article's intro first, and further only when the intro has no answer of
21
+ the right kind (a height with a unit, a year, a number, a name)
22
+ * strips brackets and pronunciation guides, so real text looks like training text
23
+ * hands back ONE sentence. A 340-character window found the answer more often but the
24
+ model misread it four times in five; one sentence doubled the final score (15% -> 30%)
25
+
26
+ Swap search() for anything you like — the contract is text in, one sentence out.
27
+ Wikipedia needs no API key. Stdlib only.
28
+ """
29
+ from __future__ import annotations
30
+
31
+ import argparse, json, re, ssl, sys, time, urllib.error, urllib.parse, urllib.request
32
+
33
+ try: # macOS system Python often lacks a CA bundle
34
+ import certifi
35
+ SSL_CTX = ssl.create_default_context(cafile=certifi.where())
36
+ except Exception:
37
+ SSL_CTX = ssl.create_default_context()
38
+
39
+ OLLAMA = "http://localhost:11434/api/generate"
40
+ MODEL = "hf.co/textilelabs/Loom-Crucible-Preview"
41
+ API = "https://en.wikipedia.org/w/api.php?"
42
+ # Wikipedia returns 403 without a descriptive User-Agent.
43
+ UA = {"User-Agent": "LoomHarness/3.1 (Textile Labs; https://huggingface.co/textilelabs)"}
44
+ LOOKUP = re.compile(r"<lookup>(.*?)</lookup>", re.S)
45
+ _cache: dict = {}
46
+
47
+ # ------------------------------------------------------------------- the model
48
+ def loom(prompt: str, n: int = 64) -> str:
49
+ body = json.dumps({"model": MODEL, "prompt": prompt, "raw": True, "stream": False,
50
+ "options": {"temperature": 0, "repeat_penalty": 1.0, "num_predict": n,
51
+ "stop": ["<|eot|>", "<user>", "<result>"]}}).encode()
52
+ req = urllib.request.Request(OLLAMA, data=body, headers={"Content-Type": "application/json"})
53
+ with urllib.request.urlopen(req, timeout=120) as r:
54
+ return json.load(r)["response"].strip()
55
+
56
+ # ------------------------------------------------------------------ wikipedia
57
+ def _get(params: dict) -> dict:
58
+ key = json.dumps(params, sort_keys=True)
59
+ if key in _cache:
60
+ return _cache[key]
61
+ for attempt in range(3):
62
+ try:
63
+ with urllib.request.urlopen(urllib.request.Request(
64
+ API + urllib.parse.urlencode(params), headers=UA),
65
+ context=SSL_CTX, timeout=20) as r:
66
+ _cache[key] = json.load(r)
67
+ return _cache[key]
68
+ except urllib.error.HTTPError as e:
69
+ if e.code == 429:
70
+ time.sleep(3 * (attempt + 1)); continue
71
+ raise
72
+ raise RuntimeError("Wikipedia rate limit")
73
+
74
+ def search(q: str, n: int = 3) -> list:
75
+ return [h["title"] for h in _get({"action": "query", "list": "search", "srsearch": q,
76
+ "format": "json", "srlimit": n})["query"]["search"]]
77
+
78
+ def _extract(title: str, intro: bool) -> str:
79
+ p = {"action": "query", "prop": "extracts", "explaintext": 1, "titles": title,
80
+ "format": "json", "redirects": 1}
81
+ if intro:
82
+ p["exintro"] = 1
83
+ return next(iter(_get(p)["query"]["pages"].values())).get("extract", "") or ""
84
+
85
+ # ------------------------------------------------------------------ the finder
86
+ SENT = re.compile(r"(?<=[.!?])\s+(?=[A-Z0-9])")
87
+ PAREN = re.compile(r"\s*\([^()]*\)")
88
+ HEADING = re.compile(r"^\s*=+[^=]+=+\s*$", re.M)
89
+ STOP = set(("what whats who whos whom whose when where which why how is are was were be the a an "
90
+ "of in on to for does did do by from with as at and or that this it its there tell me "
91
+ "please can you many much").split())
92
+ ATTR = set(("capital city height tall high elevation population largest biggest smallest longest "
93
+ "shortest tallest highest deepest first last symbol chemical language languages spoken "
94
+ "legs year date end ended sink sank invented inventor discovered discovery developed "
95
+ "wrote written author painted painter president founded born died age old size area "
96
+ "distance speed").split())
97
+ JUNK = re.compile(r"^(lists? of|outline of|index of|timeline of)\b|\((film|album|song|band|"
98
+ r"novel|play|tv series|musical|opera|video game|book|composition|poem)\)|"
99
+ r"\bdisambiguation\b", re.I)
100
+
101
+ def keywords(t: str) -> list:
102
+ return [w for w in re.findall(r"[^\W_]+", t.lower()) if w not in STOP]
103
+
104
+ def subject(question: str) -> str:
105
+ kw = keywords(question)
106
+ return " ".join(k for k in kw if k not in ATTR) or " ".join(kw)
107
+
108
+ def clean(t: str) -> str:
109
+ prev = None
110
+ while prev != t:
111
+ prev, t = t, PAREN.sub("", t)
112
+ return re.sub(r"\s+", " ", t.replace(" ,", ",")).strip()
113
+
114
+ def _hard(q: str, s: str) -> float:
115
+ """The answer is of the right KIND: a height with a unit, a year, a number, a name."""
116
+ b = 0.0
117
+ if re.search(r"\b(how tall|how high|height|elevation)\b", q):
118
+ b += 2.0 if re.search(r"\d[\d,.]*\s*(m|metres|meters|ft|feet|km)\b", s) else 0
119
+ if re.search(r"\b(when|what year|which year|what date)\b", q):
120
+ b += 2.0 if re.search(r"\b(1\d{3}|20\d{2})\b", s) else 0
121
+ if re.search(r"\b(how many|how much|population|number of)\b", q):
122
+ b += 1.5 if re.search(r"\d", s) else 0
123
+ if re.search(r"\bwho\b", q):
124
+ b += 1.5 if re.search(r"\b[A-Z][a-z]+ [A-Z][a-z]+", s) else 0
125
+ if re.search(r"\bsymbol\b", q):
126
+ b += 2.0 if re.search(r"\bsymbol\b", s, re.I) else 0
127
+ if re.search(r"\bcapital\b", q):
128
+ b += 2.0 if re.search(r"\bcapital\b", s, re.I) else 0
129
+ return b
130
+
131
+ def _kind(q: str, s: str) -> float:
132
+ b, sl = _hard(q, s), s.lower()
133
+ if re.search(r"\b(how tall|how high|height|elevation)\b", q):
134
+ b += 1.5 if re.search(r"\b(summit|elevation|height|above sea level|highest|stands)\b", sl) else -0.5
135
+ if re.search(r"\b(end|ended|finish|finished)\b", q):
136
+ b += 1.5 if re.search(r"\b(ended|end of|surrender|surrendered|concluded|finished)\b", sl) else -0.5
137
+ if re.search(r"\bpopulation\b", q):
138
+ b += 2.0 if re.search(r"\d{1,3}(,\d{3})+|\d+(\.\d+)?\s*(million|billion)", s) else -1.0
139
+ if re.search(r"\b(invent|invented|inventor|discovered|wrote|painted|composed|founded)\b", q):
140
+ b += 1.0 if re.search(r"\b[A-Z][a-z]+ (?:[A-Z][a-z]+ )?[A-Z][a-z]+\b", s) else 0.0
141
+ return b
142
+
143
+ def find(query: str, question: str) -> tuple:
144
+ """One sentence most likely to hold the answer, and the article it came from."""
145
+ q = question.lower()
146
+ subj = subject(question)
147
+ pool = {}
148
+ for tq in dict.fromkeys(x for x in (query.strip(), subj, " ".join(keywords(question))) if x):
149
+ for rank, t in enumerate(search(tq, 3)):
150
+ tl = t.lower()
151
+ s = (4.0 if tl in (subj, query.strip().lower()) else 2.0 if subj and tl.startswith(subj) else 0.0)
152
+ s += -4.0 if JUNK.search(t) else 0.0
153
+ pool[t] = max(pool.get(t, -1e9), s - 0.3 * rank)
154
+ qk = list(dict.fromkeys(keywords(question) + keywords(query)))
155
+ top = sorted(pool.items(), key=lambda x: -x[1])[:3]
156
+ typed = bool(re.search(r"\b(how tall|how high|height|elevation|when|what year|which year|"
157
+ r"how many|how much|population|who|symbol|capital)\b", q))
158
+ best = (-1e9, "", "")
159
+ for intro in (True, False):
160
+ found_kind = False
161
+ for title, ps in top:
162
+ raw = _extract(title, intro)
163
+ if re.search(r"\b(may|can) refer to\b", raw[:400]):
164
+ continue
165
+ body = clean(HEADING.sub(" ", raw))
166
+ sents = [s.strip() for s in SENT.split(body) if 20 < len(s.strip()) < 600]
167
+ for i, s in enumerate(sents[: 14 if intro else 90]):
168
+ sc = ps + sum(1.0 for k in qk if k in s.lower()) + _kind(q, s) + (0.5 if i < 3 else 0.0)
169
+ if sc > best[0]:
170
+ best = (sc, s, title)
171
+ found_kind = _hard(q, s) > 0
172
+ if best[1] and (not typed or found_kind):
173
+ break # the intro held an answer of the right kind
174
+ return best[1], best[2]
175
+
176
+ # ------------------------------------------------------------------ the loop
177
+ def ask(message: str, tools: bool = True, show: bool = False) -> str:
178
+ convo = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message.strip()}\n<|eot|>\n<loom>\n"
179
+ first = loom(convo)
180
+ m = LOOKUP.search(first)
181
+ if not m:
182
+ return first
183
+ query = m.group(1).strip()
184
+ try:
185
+ result, source = find(query, message)
186
+ except Exception as e:
187
+ # Never feed an error in as if it were a result — the model will answer from it.
188
+ return f"[harness] lookup failed for {query!r}: {e}"
189
+ if not result:
190
+ return f"[harness] nothing found for {query!r}"
191
+ if show:
192
+ print(f" [searched: {query!r}]\n [read from {source}: {result[:150]}]")
193
+ return loom(convo + first + f"<|eot|>\n<result>\n{result}\n<|eot|>\n<loom>\n", n=48)
194
+
195
+ def main() -> int:
196
+ global MODEL
197
+ ap = argparse.ArgumentParser(description="Loom Spark 3 harness")
198
+ ap.add_argument("message", nargs="*")
199
+ ap.add_argument("--no-tools", action="store_true", help="chat only, no lookups")
200
+ ap.add_argument("--show", action="store_true", help="print the query and the sentence read")
201
+ ap.add_argument("--model", default=MODEL)
202
+ a = ap.parse_args()
203
+ MODEL = a.model
204
+ if a.message:
205
+ print(ask(" ".join(a.message), not a.no_tools, a.show)); return 0
206
+ print(f"Loom harness — {MODEL} (tools {'off' if a.no_tools else 'on'}, ctrl-c to quit)\n")
207
+ while True:
208
+ try:
209
+ msg = input("you > ").strip()
210
+ except (EOFError, KeyboardInterrupt):
211
+ print(); return 0
212
+ if msg:
213
+ print(f"loom > {ask(msg, not a.no_tools, a.show)}\n")
214
+
215
+ if __name__ == "__main__":
216
+ sys.exit(main())
logo.jpg ADDED

Git LFS Details

  • SHA256: 4dff6148e17f0eecdacc3ea606132423e48418a212c393faa9a0cbdc79c9181f
  • Pointer size: 132 Bytes
  • Size of remote file: 2.93 MB
loom-crucible-preview-f16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e29441661d59351e073a27c4ceb562ff761fdce901563dcd30fa28788daaea59
3
+ size 310662304
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51ee9d9726c2c64c32a524b8570a31ca7eeac66031f13337a26fae6d79266f2f
3
+ size 619975896
params ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "stop": ["<|eot|>", "<user>", "<result>"],
3
+ "temperature": 0.7,
4
+ "top_k": 40,
5
+ "repeat_penalty": 1.0,
6
+ "num_predict": 96
7
+ }
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"eos_token":"<|eot|>","pad_token":"<|pad|>","additional_special_tokens":["<tools:on>","<tools:off>","<user>","<loom>","<result>","<lookup>","</lookup>"]}
template ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ <tools:off>
2
+ <user>
3
+ {{ .Prompt }}
4
+ <|eot|>
5
+ <loom>
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"tokenizer_class":"PreTrainedTokenizerFast","model_max_length":1024,"eos_token":"<|eot|>","pad_token":"<|pad|>","additional_special_tokens":["<tools:on>","<tools:off>","<user>","<loom>","<result>","<lookup>","</lookup>"],"clean_up_tokenization_spaces":false}