textilelabs/Loom-Crucible-Preview

Loom Crucible Preview

155.0M parameters · 52 layers · 1024 context · Textile Labs

The biggest Loom by far, and a preview of how far our growth method can push a small, disciplined model. We took Loom Tapestry 3 (69.2M) and grew it in both directions at once, wider (384 → 512) and deeper (40 → 52 layers). The growth is function-preserving: the 155M model started out producing exactly what Tapestry 3 produced, then trained for 12 hours on a rebuilt corpus.

It calls itself Loom Crucible. It reads what a search brings back and answers from it, says when it looked something up, says when it can't know something about you, and stays a Loom: plain, brief, and not a story generator.

Trained from-scratch lineage (no third-party checkpoint), on a Kaggle dual-T4. Runs offline via Ollama; llama.cpp-compatible.

you            who wrote dracula
Loom Crucible  <lookup>wrote dracula</lookup>
harness        ← Dracula is an 1897 Gothic horror novel by Irish author Bram Stoker.
Loom Crucible  Bram Stoker. I had to look that up.

What "Preview" means

Crucible clearly beats every earlier Loom on its core skills: reading a passage correctly (9/10 vs Tapestry 3's 4/10) and the acceptance battery (125/133). It is not yet better end to end on live Wikipedia (10/20 held-out vs Tapestry 3's 12/20). On that test the limit is now the search step, not the reading step: when the search found the answer, both models got it right about 85% of the time. A better harness is the next release, and it helps every Loom.

How it was grown

Cold-training a big model on little data fails. Instead we grow a trained small one:

• Wider, 384 → 512. New channels start at zero so they cannot disturb the old ones. Two details made this exact rather than approximate. First, the 2 new attention heads per layer are placed inside their key/value groups, because appending them silently re-pairs the old heads with the wrong keys. Second, every RMSNorm weight is scaled by √(384/512) and ε by 384/512, because zero-padding changes the norm.
• Deeper, 40 → 52. 12 new layers are inserted as no-ops (zeroed output projections).

At step 0 the 155M model matched Tapestry 3 to 3e-4 in the logits and 1e-6 in the loss. All twelve hours went into getting better, not starting over.

Measured behaviour

The family, same harness, 2026-09:

model	params	battery /133	live search (held-out)	reads real prose
Loom Tapestry 2	22.8M	107 (80.5%)	—	curated only
Loom Tapestry 3 Flash	7.18M	112 (84.2%)	3/20	curated only
Loom Spark 3 Flash	7.18M	119 (89.5%)	5/20	curated only
Loom Spark 3	12.2M	120 (90.2%)	7/20	curated only
Loom Weave 3	31.5M	120 (90.2%)	6/20	yes
Loom Tapestry 3	69.2M	123 (92.5%)	12/20	yes + multi-hop
Loom Crucible Preview	155.0M	125 (94.0%)†	10/20	yes, best reader

† Crucible's score is the first on a fully clean held-out battery. While building its corpus we found that some battery prompts had leaked into the training data used since Tapestry 3. We removed them for Crucible, so earlier scores are somewhat optimistic by comparison.

The acceptance battery, row by row:

row	Tapestry 3	Crucible
A · says its own name	12/12	12/12
B · its own name under rough typing	11/12	11/12
C · 5-turn conversation stays on thread	5/5	5/5
D · answers from a search result	5/5	4/5
E · follow-up from the same result	2/5	5/5
F · says it looked, after a lookup	5/5	4/5
G · never claims a lookup it didn't make	16/16	16/16
H · admits what it can't know about you	8/8	6/8
I · says when a result doesn't contain the answer	2/5	3/5
J · never leaks a search tag with tools off	28/28	28/28
K · stops on its own	12/12	12/12
L · searches when it should, not for your private things	17/20	19/20
total	123/133	125/133

Held-out reading bench (hand-written passages with distractors, e.g. a country's capital next to its largest city, a start year next to an end year; none in any training data):

	Tapestry 3	Crucible
answers correctly from the passage	4/10	9/10
says so when an on-topic passage lacks the answer	0/10	3/10
says so when the passage is off-topic	0/8	6/8
refuses a passage that does hold the answer	0	0
with tools off, won't answer a famous fact from memory	3/8	3/8

End to end, everyday questions it had never seen, live Wikipedia, model writing its own query, scored on the final answer: tuning 11/20, held-out 10/20 (Tapestry 3: 12/20, 12/20). When the search found the answer, Crucible got it right 10 times out of 12.

Read this before you use it

Every point measured.

• It is not better than Tapestry 3 end to end yet. It reads better, but live results are capped by the search step. Run harness.py --show to see the sentence it read.
• When a search comes back without the answer, it usually answers anyway. Given an on-topic passage that lacks the fact, it says so only 3 times in 10. If the passage holds something of the right type (a number for "how tall", a year for "when did it end"), it tends to grab that. Trust the sentence shown by --show, not just the reply.
• With tools off, it sometimes answers famous facts from half-memory (3/8 held). Those answers are often wrong. Keep tools on for facts.
• It once refused a passage that did hold the answer (1 of 5 battery reading rows).
• It is not a calculator. Arithmetic is out of scope; use a tool.
• Harness search is Wikipedia only: no time, weather, news or prices.
• Plain, brief voice by design. It is a small reader and assistant, not a chat stylist.

What it does reliably: knows what it is, stops on its own, holds a conversation, answers follow-ups from the same result (5/5), never claims a lookup it didn't make (16/16), and reads a supplied passage better than any earlier Loom.

Usage — the harness

python3 harness.py "who wrote dracula"
python3 harness.py --show "how tall is ben nevis"
python3 harness.py --no-tools "who are you"

Stdlib + certifi. Wikipedia needs no API key.

Usage — Ollama

ollama run hf.co/textilelabs/Loom-Crucible-Preview "who are you"

template and params are read automatically.

Usage — transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Crucible-Preview")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Crucible-Preview").eval()
eot = tok.convert_tokens_to_ids("<|eot|>")
def ask(message, tools=False):
    p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
    ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
    with torch.no_grad():
        out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
                             pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
    return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>","").strip()

Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n.

How it was built

architecture	Llama — 52 layers × 512d, GQA (8 heads / 2 KV), SwiGLU, RoPE, tied embeddings
parameters	154,980,864
grown from	Loom Tapestry 3 (40 × 384) by function-preserving width + depth growth
context	1,024
vocabulary	16,384 custom BPE
optimiser	Muon (LR 0.025) on 2D hidden matrices, AdamW on embeddings and norms
loss	masked to the reply tokens; whole conversations packed per block
corpus	275,788 conversations: Loom curriculum + SQuAD 2.0 (incl. unanswerable) + HotpotQA + TriviaQA
training	6 h grown from Tapestry 3, then 6 h continued on the rebuilt corpus · best-validation checkpoint
hardware	Kaggle dual T4 (GPU)
decoding	ship with repeat_penalty 1.0: higher values corrupt copying from the passage

Files

config.json / model.safetensors           the model
tokenizer.json / tokenizer_config.json    custom BPE tokenizer, 16,384 tokens
loom-crucible-preview-f16.gguf            for Ollama / llama.cpp (lookup tags USER_DEFINED)
harness.py                                 runnable search harness
template / params                          read automatically by `ollama run hf.co/...`
Modelfile                                  for building locally
ATTRIBUTION.md                             required credits for the training corpora

License

Model: MIT. Training data retains its original licences and attribution.