textilelabs/Loom-Tapestry-3

Loom Tapestry 3

69.2M parameters · 40 layers · 1024 context · Textile Labs

The best Loom yet, and the first one grown. We took our 31.5M Loom Weave 3, stacked its 16 layers into 40 (function-preserving, so the bigger model started exactly where Weave 3 left off), and continue-trained it for two hours on a richer corpus. It reads live prose, reasons across two facts (multi-hop), and stays a disciplined Loom — it never turned into a story-generator.

Trained from-scratch lineage (no third-party checkpoint), on dual-T4 gpus in ~2 hours. Runs offline via Ollama; llama.cpp-compatible.

you             which magazine was started first, Arthur's or First for Women?
Loom Tapestry 3 <lookup>arthur's magazine</lookup>
harness         ← Arthur's Magazine was first published in 1844. First for Women started in 1989.
Loom Tapestry 3 Arthur's Magazine. I had to look that up.

How it was grown

Cold-training a big model on little data fails (undertrained). Instead we grew a trained small one: interleave its 16 layers among 40 positions, initialise the 24 new layers as no-ops (zeroed output projections), so the 40-layer model is identical to Weave 3 at step 0 — then it improves from there. Init loss was 2.2, not 9.7. All two hours went into getting better, not starting over.

Measured behaviour vs the family (same harness, 2026-09)

model	params	battery /133	live search (held-out)	reads real prose
Loom Tapestry 2	22.8M	107 (80.5%)	—	curated only
Loom Spark 3 Flash	7.18M	119 (89.5%)	5/20	curated only
Loom Spark 3	12.2M	120 (90.2%)	7/20	curated only
Loom Weave 3	31.5M	120 (90.2%)	6/20	yes
Loom Tapestry 3	69.2M	123 (92.5%)	12/20	yes + multi-hop

The acceptance battery, row by row:

row	Loom Tapestry 3
A · says its own name	12/12
B · its own name under rough typing	11/12
C · 5-turn conversation stays on thread	5/5
D · answers from a search result	5/5
E · follow-up from the same result	2/5
F · says it looked, after a lookup	5/5
G · never claims a lookup it didn't make	16/16
H · admits what it can't know about you	8/8
I · says when a result doesn't contain the answer	2/5
J · never leaks a search tag with tools off	28/28
K · stops on its own	12/12
L · searches when it should, not for your private things	17/20
total	123/133

End to end, everyday questions it had never seen, live Wikipedia, model writing its own query, scored on the final answer: tuning 12/20 (60%), held-out 12/20 (60%) — double Weave 3's held-out reading.

Read this before you use it

Every point measured.

• "I looked that up" means it searched — not that it read perfectly. Held-out reading is ~60%; run harness.py --show and trust the sentence it read.
• Follow-up questions from the same result are weak (~40%). The family's oldest gap.
• It is not a calculator. Arithmetic is out of scope; use a tool.
• It rarely asks a clarifying question on an ambiguous request.
• Harness search is Wikipedia only — no time, weather, news or prices.
• It is a small reader/assistant, not a chat stylist. Voice is plain and brief by design.

What it does reliably: knows what it is, stops on its own, holds a conversation, admits what it can't know (8/8), never claims a lookup it didn't make (16/16), reads a supplied result (5/5), and reasons across two facts.

Usage — the harness

python3 harness.py "who wrote dracula"
python3 harness.py --show "how tall is mount everest"
python3 harness.py --no-tools "who are you"

Stdlib + certifi. Wikipedia needs no API key.

Usage — Ollama

ollama run hf.co/textilelabs/Loom-Tapestry-3 "who are you"

template and params are read automatically.

Usage — transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Tapestry-3")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Tapestry-3").eval()
eot = tok.convert_tokens_to_ids("<|eot|>")
def ask(message, tools=False):
    p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
    ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
    with torch.no_grad():
        out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
                             pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
    return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>","").strip()

Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n.

How it was built

architecture	Llama — 40 layers × 384d, GQA (6 heads / 2 KV), SwiGLU, RoPE, tied embeddings
parameters	69,237,120
grown from	Loom Weave 3 (16 layers) via function-preserving layer stacking
context	1,024
vocabulary	16,384 custom BPE
optimiser	Muon (LR 0.025) on 2D hidden matrices, AdamW on embeddings and norms
loss	masked to the reply tokens; whole conversations packed per block (FFD)
corpus	Loom curriculum + HotpotQA multi-hop + unanswerable rows (real prose in results)
training	~500 steps · ~67M tokens · best-validation checkpoint · from random-init lineage
hardware	Kaggle dual T4 (GPU) · ~2 hours

Files

config.json / model.safetensors           the model
tokenizer.json / tokenizer_config.json    custom BPE tokenizer, 16,384 tokens
loom-tapestry-3-f16.gguf                   for Ollama / llama.cpp (lookup tags USER_DEFINED)
harness.py                                 runnable search harness
template / params                          read automatically by `ollama run hf.co/...`
Modelfile                                  for building locally
ATTRIBUTION.md                             required credits for the training corpora

License

Model: MIT. Training data retains its original licences and attribution.