Upload 3 files

Modelfile

@@ -0,0 +1,14 @@
+FROM ./loom-tapestry-2-f16.gguf
+TEMPLATE """<tools:off>
+<user>
+{{ .Prompt }}
+<|eot|>
+<loom>
+"""
+PARAMETER stop "<|eot|>"
+PARAMETER stop "<user>"
+PARAMETER stop "<result>"
+PARAMETER temperature 0.7
+PARAMETER top_k 40
+PARAMETER repeat_penalty 1.0
+PARAMETER num_predict 96

README.md

@@ -0,0 +1,247 @@
+---
+license: mit
+language: en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- tiny-model
+- llama
+- from-scratch
+- conversational
+- tool-use
+- agent-harness
+- retrieval-augmented
+- attribution
+- calibrated-honesty
+- humble-ai
+- philosophy-of-mind
+- small-language-model
+- cpu-trained
+- muon
+- gguf
+- text-generation-inference
+widget:
+- text: "<tools:off>\n<user>\nwho are you\n<|eot|>\n<loom>\n"
+  example_title: "Identity"
+- text: "<tools:off>\n<user>\nwhat is my sisters name\n<|eot|>\n<loom>\n"
+  example_title: "Limits"
+- text: "<tools:on>\n<user>\nwhat is the capital of peru\n<|eot|>\n<loom>\n"
+  example_title: "Lookup"
+---
+
+<div align="center">
+<img src="banner.jpg" alt="Loom Tapestry 2" width="520">
+</div>
+
+# Loom Tapestry 2
+
+<img src="logo.jpg" alt="" width="20" height="20" style="border-radius:4px;vertical-align:middle;margin-right:6px;"> **22.8M parameters · 20 layers · 768 context · Textile Labs**
+
+The first model in the **Tapestry** tier. Loom tiers run **Flash → Spark → Weave → Tapestry**.
+
+Trained from scratch on a 2013 desktop CPU — randomly initialised weights, nothing
+fine-tuned from a pretrained base.
+
+**It looks things up, and it tells you when it did.**
+
+```
+who are you              →  Loom Tapestry 2, a small model by Textile Labs.
+what is my sisters name  →  I don't know that about you.
+
+what is the capital of peru
+  →  <lookup>what is the capital of peru</lookup>
+  ←  Lima is the capital and largest city of Peru…
+  →  Lima is the capital and largest city of Peru. I had to look that up.
+how many people live there
+  →  9.7 million.          ← same result, no second lookup
+```
+
+## Why "I looked that up" matters
+
+Most small models make you guess which of their answers to trust. Tapestry has three
+honest registers, and you can tell them apart **by reading**:
+
+| situation | what it does |
+|---|---|
+| answered from a retrieved `<result>` | **says it looked it up** |
+| answered from training | answers plainly |
+| cannot be known | *"I don't know that about you."* |
+
+It never claims a lookup it didn't make — **16/16** on that check below. A false
+attribution would be worse than none, so that is the one number held to 100%.
+
+## Measured
+
+Full acceptance battery, hand-written prompts held out of the training generator,
+scored on **content** rather than shape. Every failure is listed rather than summarised.
+
+| | score | |
+|---|---:|---|
+| no false attribution | **16/16** | never claims a lookup it didn't make |
+| no `<lookup>` leak with tools off | **28/28** | |
+| self-terminates without a Modelfile | **12/12** | |
+| answers from a supplied `<result>` | **5/5** | |
+| identity — names Tapestry | **11/12** | |
+| attribution present after a real lookup | **4/5** | |
+| identity under CAPS / typos / "?" | **10/12** | |
+| 5-turn conversation stays on thread | **4/5** | |
+| admits an unknowable | **4/8** | |
+| follow-up answered from the same result | **2/5** | |
+| says the result doesn't contain it | **1/5** | first Loom to score above zero |
+| tool decision with tools **on** | **10/20** | 10/10 correct when a lookup *is* needed; **0/10** when it is not — see below |
+| **overall** | **107/133 · 80.5%** | |
+
+### Against the previous generation
+
+Identical corpus family, same evaluation method.
+
+| | params | val loss | val accuracy |
+|---|---:|---:|---:|
+| Loom Spark 2 | 19.9M | 2.692 | 0.536 |
+| Loom Weave 2 Flash | 19.9M | 2.254 | 0.580 |
+| **Loom Tapestry 2** | **22.8M** | **1.963** | **0.622** |
+
+13% lower loss and +4.2 accuracy points over the previous best.
+
+## Read this before you use it
+
+**Keep tools OFF for conversation.** The persona was trained entirely under `tools:off`.
+With tools **on**, identity and personal questions get turned into a lookup — measured
+**0/10** on that case. The shipped Ollama template defaults to `tools:off`; switch to
+`tools:on` only for the retrieval loop.
+
+**Validate what it tells you from a result.** It answers from a `<result>` whether or not
+the answer is actually in there — "says the result doesn't contain it" is 1/5. Treat the
+retrieved text as the trustworthy part and the model's summary of it as unreliable.
+Extraction picks the wrong span roughly a third of the time.
+
+**It is a lookup assistant, not a chat companion.** At 22.8M parameters it does not
+improvise, explain in its own words, or hold a free-ranging conversation. What it does
+reliably is decide a lookup is needed, write the query, read the answer back, and say
+where the answer came from.
+
+**It has almost no world knowledge.** With tools off it declines factual questions. That is
+the intended behaviour, not a fault.
+
+## Two modes
+
+**`<tools:off>` (default)** — conversational. Identity, limits, warmth, brevity.
+
+**`<tools:on>`** — emits `<lookup>query</lookup>` and stops. Your harness runs the lookup
+and continues with a `<result>` block:
+
+```
+<tools:on>
+<user>
+what is the capital of peru
+<|eot|>
+<loom>
+<lookup>what is the capital of peru</lookup><|eot|>
+<result>
+Lima is the capital and largest city of Peru.
+<|eot|>
+<loom>
+```
+
+## Usage — the harness
+
+`harness.py` in this repo runs the lookup and feeds the result back. Wikipedia is used
+because it is free and needs no key — swap the `search()` function for anything else; the
+contract is text in, text out.
+
+```bash
+python3 harness.py "who wrote dracula"      # with lookups
+python3 harness.py                          # interactive
+python3 harness.py --no-tools "who are you" # chat only
+```
+
+Three things any harness for this model needs:
+
+- **Never feed a failed lookup back as a `<result>`.** It will earnestly answer from the
+  error text. Fail loudly instead — `harness.py` does.
+- Wikipedia returns **403** without a descriptive `User-Agent`.
+- macOS system Python often needs **certifi** for TLS.
+
+## Usage — Ollama
+
+```bash
+ollama run hf.co/textilelabs/Loom-Tapestry-2 "who are you"
+```
+
+The `template` and `params` files in this repo are read automatically. To build locally:
+`ollama create loom-tapestry-2 -f Modelfile`.
+
+**Do not add a repetition penalty.** This model answers by quoting from the `<result>` you
+give it, so penalising repeated tokens penalises the correct answer. Measured at
+`repeat_penalty 1.15` it changed "1,345 metres" into "2,345 metres" — silently wrong rather
+than merely worse. `params` ships it at 1.0 for that reason. The trade-off is that on a
+question it cannot handle it will occasionally loop on a short phrase until it hits
+`num_predict`; that is the safer failure.
+
+## Usage — transformers
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Tapestry-2")
+model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Tapestry-2").eval()
+eot = tok.convert_tokens_to_ids("<|eot|>")
+
+def ask(message, tools=False):
+    p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
+    ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
+    with torch.no_grad():
+        out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
+                             pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
+    return tok.decode(out[ids.shape[1]:], skip_special_tokens=True).strip()
+
+ask("who are you")   # -> 'Loom Tapestry 2, a small model by Textile Labs.'
+```
+
+Prompt format is exact: `<tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n`.
+No trailing space after `<loom>`.
+
+## How it was built
+
+| | |
+|---|---|
+| architecture | Llama — 20 layers × 320d, GQA, SwiGLU, RoPE, tied embeddings |
+| context | 768 |
+| vocabulary | 4,096 custom BPE |
+| optimiser | **Muon** on all 140 hidden matrices, AdamW on embeddings and norms |
+| schedule | warmup → stable → decay (WSD) |
+| corpus | 130,741 conversations · 17.8M tokens · **56% multi-turn** |
+| training | 2,058 steps from random initialisation |
+
+Depth was chosen over width deliberately: an earlier ladder study on this family found
+that narrowing the hidden size cost about 3 points while removing a layer cost ten.
+
+## Files
+
+```
+config.json / model.safetensors           the model
+tokenizer.json / tokenizer_config.json    custom BPE tokenizer, 4,096 tokens
+loom-tapestry-2-f16.gguf                  44MB, for Ollama / llama.cpp
+harness.py                                runnable harness — runs lookups, feeds results back
+template / params                         read automatically by `ollama run hf.co/...`
+Modelfile                                 for building locally
+ATTRIBUTION.md                            required credits for the training corpora
+```
+
+## Training data
+
+Openly licensed corpora of real human text, plus a persona curriculum written for Loom.
+See `ATTRIBUTION.md` — several of these licences require credit.
+
+| slice | source |
+|---|---|
+| grounded reading, and "the result doesn't say" | **SQuAD 2.0** (CC BY-SA 4.0) |
+| when to reach for a tool | **MASSIVE** (CC BY 4.0) · **CLINC150** (CC BY 3.0) |
+| instruction following | **databricks-dolly-15k** (CC BY-SA 3.0) |
+| multi-turn dialogue structure | **OpenAssistant OASST1** (Apache 2.0) |
+| identity, limits, warmth, attribution | Textile Labs — written for Loom |
+
+## License
+
+Model: MIT. Training data retains its original licences and attribution.

params

@@ -0,0 +1,7 @@
+{
+  "stop": ["<|eot|>", "<user>", "<result>"],
+  "temperature": 0.7,
+  "top_k": 40,
+  "repeat_penalty": 1.0,
+  "num_predict": 96
+}