Text Generation
Transformers
Safetensors
GGUF
English
llama
tiny-model
from-scratch
model-growth
conversational
multi-turn
tool-use
agent-harness
retrieval-augmented
multi-hop-qa
question-answering
attribution
humble-ai
small-language-model
muon
text-generation-inference
Instructions to use textilelabs/Loom-Tapestry-3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use textilelabs/Loom-Tapestry-3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="textilelabs/Loom-Tapestry-3") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("textilelabs/Loom-Tapestry-3") model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Tapestry-3", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use textilelabs/Loom-Tapestry-3 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf textilelabs/Loom-Tapestry-3:F16 # Run inference directly in the terminal: llama cli -hf textilelabs/Loom-Tapestry-3:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf textilelabs/Loom-Tapestry-3:F16 # Run inference directly in the terminal: llama cli -hf textilelabs/Loom-Tapestry-3:F16
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf textilelabs/Loom-Tapestry-3:F16 # Run inference directly in the terminal: ./llama-cli -hf textilelabs/Loom-Tapestry-3:F16
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf textilelabs/Loom-Tapestry-3:F16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf textilelabs/Loom-Tapestry-3:F16
Use Docker
docker model run hf.co/textilelabs/Loom-Tapestry-3:F16
- LM Studio
- Jan
- vLLM
How to use textilelabs/Loom-Tapestry-3 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "textilelabs/Loom-Tapestry-3" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "textilelabs/Loom-Tapestry-3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/textilelabs/Loom-Tapestry-3:F16
- SGLang
How to use textilelabs/Loom-Tapestry-3 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "textilelabs/Loom-Tapestry-3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "textilelabs/Loom-Tapestry-3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "textilelabs/Loom-Tapestry-3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "textilelabs/Loom-Tapestry-3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use textilelabs/Loom-Tapestry-3 with Ollama:
ollama run hf.co/textilelabs/Loom-Tapestry-3:F16
- Unsloth Desktop
- Docker Model Runner
How to use textilelabs/Loom-Tapestry-3 with Docker Model Runner:
docker model run hf.co/textilelabs/Loom-Tapestry-3:F16
- Lemonade
How to use textilelabs/Loom-Tapestry-3 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull textilelabs/Loom-Tapestry-3:F16
Run and chat with the model
lemonade run user.Loom-Tapestry-3-F16
List all available models
lemonade list
- Atomic Chat
Upload 16 files
Browse files- .gitattributes +3 -0
- ATTRIBUTION.md +18 -0
- LICENSE +18 -0
- Modelfile +9 -0
- README.md +164 -0
- banner.jpg +3 -0
- config.json +32 -0
- generation_config.json +9 -0
- harness.py +216 -0
- logo.jpg +3 -0
- loom-tapestry-3-f16.gguf +3 -0
- model.safetensors +3 -0
- params +7 -0
- special_tokens_map.json +1 -0
- template +5 -0
- tokenizer.json +0 -0
- tokenizer_config.json +1 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
banner.jpg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
logo.jpg filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
loom-tapestry-3-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
ATTRIBUTION.md
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Attribution
|
| 2 |
+
|
| 3 |
+
Loom Tapestry 3 was trained from scratch (random-init lineage — grown from Textile Labs'
|
| 4 |
+
own Loom Weave 3, no third-party checkpoint). Grounded-reading rows embed real encyclopedic
|
| 5 |
+
prose. Training data, all permissively licensed:
|
| 6 |
+
|
| 7 |
+
| slice | source | licence |
|
| 8 |
+
|---|---|---|
|
| 9 |
+
| grounded reading | SQuAD 2.0 | CC BY-SA 4.0 |
|
| 10 |
+
| multi-hop grounded reading | HotpotQA | CC BY-SA 4.0 |
|
| 11 |
+
| when to reach for a tool | MASSIVE / CLINC150 | CC BY 4.0 / CC BY 3.0 |
|
| 12 |
+
| instruction following | databricks-dolly-15k | CC BY-SA 3.0 |
|
| 13 |
+
| multi-turn dialogue | OpenAssistant OASST1 | Apache 2.0 |
|
| 14 |
+
| tokenizer coverage of real prose | HuggingFaceFW/fineweb-edu | ODC-By 1.0 |
|
| 15 |
+
| encyclopedic passages | Wikipedia | CC BY-SA |
|
| 16 |
+
| identity, limits, warmth, attribution | Textile Labs | — |
|
| 17 |
+
|
| 18 |
+
No language model wrote any training query. No real user data was used. Model weights: MIT.
|
LICENSE
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2026 Textile Labs
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy of this model
|
| 6 |
+
and associated files (the "Model"), to deal in the Model without restriction, including
|
| 7 |
+
without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
|
| 8 |
+
and/or sell copies of the Model, and to permit persons to whom the Model is furnished to do
|
| 9 |
+
so, subject to the following conditions:
|
| 10 |
+
|
| 11 |
+
The above copyright notice and this permission notice shall be included in all copies or
|
| 12 |
+
substantial portions of the Model.
|
| 13 |
+
|
| 14 |
+
THE MODEL IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
|
| 15 |
+
BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
| 16 |
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 17 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
| 18 |
+
CONNECTION WITH THE MODEL OR THE USE OR OTHER DEALINGS IN THE MODEL.
|
Modelfile
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
FROM ./loom-tapestry-3-f16.gguf
|
| 2 |
+
TEMPLATE "<tools:off>\n<user>\n{{ .Prompt }}\n<|eot|>\n<loom>\n"
|
| 3 |
+
PARAMETER stop "<|eot|>"
|
| 4 |
+
PARAMETER stop "<user>"
|
| 5 |
+
PARAMETER stop "<result>"
|
| 6 |
+
PARAMETER temperature 0.7
|
| 7 |
+
PARAMETER top_k 40
|
| 8 |
+
PARAMETER repeat_penalty 1.0
|
| 9 |
+
PARAMETER num_predict 96
|
README.md
CHANGED
|
@@ -1,3 +1,167 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
language: en
|
| 4 |
+
library_name: transformers
|
| 5 |
+
pipeline_tag: text-generation
|
| 6 |
+
tags:
|
| 7 |
+
- tiny-model
|
| 8 |
+
- llama
|
| 9 |
+
- from-scratch
|
| 10 |
+
- model-growth
|
| 11 |
+
- conversational
|
| 12 |
+
- multi-turn
|
| 13 |
+
- tool-use
|
| 14 |
+
- agent-harness
|
| 15 |
+
- retrieval-augmented
|
| 16 |
+
- multi-hop-qa
|
| 17 |
+
- question-answering
|
| 18 |
+
- attribution
|
| 19 |
+
- humble-ai
|
| 20 |
+
- small-language-model
|
| 21 |
+
- muon
|
| 22 |
+
- gguf
|
| 23 |
+
- text-generation-inference
|
| 24 |
+
widget:
|
| 25 |
+
- text: "<tools:off>\n<user>\nwho are you\n<|eot|>\n<loom>\n"
|
| 26 |
+
example_title: "Identity"
|
| 27 |
+
- text: "<tools:on>\n<user>\nwho wrote dracula\n<|eot|>\n<loom>\n"
|
| 28 |
+
example_title: "Reads real prose"
|
| 29 |
+
- text: "<tools:off>\n<user>\nwhat did i eat yesterday\n<|eot|>\n<loom>\n"
|
| 30 |
+
example_title: "Limits"
|
| 31 |
---
|
| 32 |
+
|
| 33 |
+
<div align="center">
|
| 34 |
+
<img src="banner.jpg" alt="Loom Tapestry 3" width="520">
|
| 35 |
+
</div>
|
| 36 |
+
|
| 37 |
+
# Loom Tapestry 3
|
| 38 |
+
|
| 39 |
+
<img src="logo.jpg" alt="" width="20" height="20" style="border-radius:4px;vertical-align:middle;margin-right:6px;"> **69.2M parameters · 40 layers · 1024 context · Textile Labs**
|
| 40 |
+
|
| 41 |
+
The best Loom yet, and the first one **grown**. We took our 31.5M
|
| 42 |
+
[Loom Weave 3](https://huggingface.co/textilelabs/Loom-Weave-3), **stacked its 16 layers into
|
| 43 |
+
40** (function-preserving, so the bigger model started exactly where Weave 3 left off), and
|
| 44 |
+
continue-trained it for two hours on a richer corpus. It reads live prose, reasons across two
|
| 45 |
+
facts (multi-hop), and stays a disciplined Loom — it never turned into a story-generator.
|
| 46 |
+
|
| 47 |
+
Trained from-scratch lineage (no third-party checkpoint), on a Kaggle dual-T4 in ~2 hours.
|
| 48 |
+
Runs offline via Ollama; llama.cpp-compatible.
|
| 49 |
+
|
| 50 |
+
```
|
| 51 |
+
you which magazine was started first, Arthur's or First for Women?
|
| 52 |
+
Loom Tapestry 3 <lookup>arthur's magazine</lookup>
|
| 53 |
+
harness ← Arthur's Magazine was first published in 1844. First for Women started in 1989.
|
| 54 |
+
Loom Tapestry 3 Arthur's Magazine. I had to look that up.
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
## How it was grown
|
| 58 |
+
Cold-training a big model on little data fails (undertrained). Instead we **grew** a trained
|
| 59 |
+
small one: interleave its 16 layers among 40 positions, initialise the 24 new layers as
|
| 60 |
+
no-ops (zeroed output projections), so the 40-layer model is **identical** to Weave 3 at
|
| 61 |
+
step 0 — then it improves from there. Init loss was 2.2, not 9.7. All two hours went into
|
| 62 |
+
getting better, not starting over.
|
| 63 |
+
|
| 64 |
+
## Measured behaviour vs the family (same harness, 2026-09)
|
| 65 |
+
|
| 66 |
+
| model | params | battery /133 | live search (held-out) | reads real prose |
|
| 67 |
+
|---|---:|---:|---:|---|
|
| 68 |
+
| Loom Tapestry 2 | 22.8M | 107 (80.5%) | — | curated only |
|
| 69 |
+
| Loom Spark 3 Flash | 7.18M | 119 (89.5%) | 5/20 | curated only |
|
| 70 |
+
| Loom Spark 3 | 12.2M | 120 (90.2%) | 7/20 | curated only |
|
| 71 |
+
| Loom Weave 3 | 31.5M | 120 (90.2%) | 6/20 | yes |
|
| 72 |
+
| **Loom Tapestry 3** | **69.2M** | **123 (92.5%)** | **12/20** | **yes + multi-hop** |
|
| 73 |
+
|
| 74 |
+
**The acceptance battery, row by row:**
|
| 75 |
+
|
| 76 |
+
| row | Loom Tapestry 3 |
|
| 77 |
+
|---|---:|
|
| 78 |
+
| A · says its own name | **12/12** |
|
| 79 |
+
| B · its own name under rough typing | 11/12 |
|
| 80 |
+
| C · 5-turn conversation stays on thread | 5/5 |
|
| 81 |
+
| D · answers from a search result | **5/5** |
|
| 82 |
+
| E · follow-up from the same result | 2/5 |
|
| 83 |
+
| F · says it looked, after a lookup | 5/5 |
|
| 84 |
+
| G · **never** claims a lookup it didn't make | **16/16** |
|
| 85 |
+
| H · admits what it can't know about you | **8/8** |
|
| 86 |
+
| I · says when a result doesn't contain the answer | 2/5 |
|
| 87 |
+
| J · never leaks a search tag with tools off | 28/28 |
|
| 88 |
+
| K · stops on its own | 12/12 |
|
| 89 |
+
| L · searches when it should, not for your private things | 17/20 |
|
| 90 |
+
| **total** | **123/133** |
|
| 91 |
+
|
| 92 |
+
**End to end**, everyday questions it had never seen, live Wikipedia, model writing its own
|
| 93 |
+
query, scored on the final answer: **tuning 12/20 (60%), held-out 12/20 (60%)** — double
|
| 94 |
+
Weave 3's held-out reading.
|
| 95 |
+
|
| 96 |
+
## Read this before you use it
|
| 97 |
+
Every point measured.
|
| 98 |
+
- **"I looked that up" means it searched — not that it read perfectly.** Held-out reading is
|
| 99 |
+
~60%; run `harness.py --show` and trust the sentence it read.
|
| 100 |
+
- **Follow-up questions from the same result are weak (~40%).** The family's oldest gap.
|
| 101 |
+
- **It is not a calculator.** Arithmetic is out of scope; use a tool.
|
| 102 |
+
- **It rarely asks a clarifying question** on an ambiguous request.
|
| 103 |
+
- **Harness search is Wikipedia only** — no time, weather, news or prices.
|
| 104 |
+
- **It is a small reader/assistant, not a chat stylist.** Voice is plain and brief by design.
|
| 105 |
+
|
| 106 |
+
What it does reliably: knows what it is, **stops on its own**, holds a conversation,
|
| 107 |
+
**admits what it can't know (8/8)**, **never claims a lookup it didn't make (16/16)**, reads
|
| 108 |
+
a supplied result (5/5), and reasons across two facts.
|
| 109 |
+
|
| 110 |
+
## Usage — the harness
|
| 111 |
+
```bash
|
| 112 |
+
python3 harness.py "who wrote dracula"
|
| 113 |
+
python3 harness.py --show "how tall is mount everest"
|
| 114 |
+
python3 harness.py --no-tools "who are you"
|
| 115 |
+
```
|
| 116 |
+
Stdlib + `certifi`. Wikipedia needs no API key.
|
| 117 |
+
|
| 118 |
+
## Usage — Ollama
|
| 119 |
+
```bash
|
| 120 |
+
ollama run hf.co/textilelabs/Loom-Tapestry-3 "who are you"
|
| 121 |
+
```
|
| 122 |
+
`template` and `params` are read automatically.
|
| 123 |
+
|
| 124 |
+
## Usage — transformers
|
| 125 |
+
```python
|
| 126 |
+
import torch
|
| 127 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 128 |
+
tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Tapestry-3")
|
| 129 |
+
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Tapestry-3").eval()
|
| 130 |
+
eot = tok.convert_tokens_to_ids("<|eot|>")
|
| 131 |
+
def ask(message, tools=False):
|
| 132 |
+
p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
|
| 133 |
+
ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
|
| 134 |
+
with torch.no_grad():
|
| 135 |
+
out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
|
| 136 |
+
pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
|
| 137 |
+
return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>","").strip()
|
| 138 |
+
```
|
| 139 |
+
Prompt format is exact: `<tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n`.
|
| 140 |
+
|
| 141 |
+
## How it was built
|
| 142 |
+
| | |
|
| 143 |
+
|---|---|
|
| 144 |
+
| architecture | Llama — 40 layers × 384d, GQA (6 heads / 2 KV), SwiGLU, RoPE, tied embeddings |
|
| 145 |
+
| parameters | 69,237,120 |
|
| 146 |
+
| grown from | Loom Weave 3 (16 layers) via function-preserving layer stacking |
|
| 147 |
+
| context | 1,024 |
|
| 148 |
+
| vocabulary | 16,384 custom BPE |
|
| 149 |
+
| optimiser | Muon (LR 0.025) on 2D hidden matrices, AdamW on embeddings and norms |
|
| 150 |
+
| loss | masked to the reply tokens; whole conversations packed per block (FFD) |
|
| 151 |
+
| corpus | Loom curriculum + HotpotQA multi-hop + unanswerable rows (real prose in results) |
|
| 152 |
+
| training | ~500 steps · ~67M tokens · best-validation checkpoint · from random-init lineage |
|
| 153 |
+
| hardware | Kaggle dual T4 (GPU) · ~2 hours |
|
| 154 |
+
|
| 155 |
+
## Files
|
| 156 |
+
```
|
| 157 |
+
config.json / model.safetensors the model
|
| 158 |
+
tokenizer.json / tokenizer_config.json custom BPE tokenizer, 16,384 tokens
|
| 159 |
+
loom-tapestry-3-f16.gguf for Ollama / llama.cpp (lookup tags USER_DEFINED)
|
| 160 |
+
harness.py runnable search harness
|
| 161 |
+
template / params read automatically by `ollama run hf.co/...`
|
| 162 |
+
Modelfile for building locally
|
| 163 |
+
ATTRIBUTION.md required credits for the training corpora
|
| 164 |
+
```
|
| 165 |
+
|
| 166 |
+
## License
|
| 167 |
+
Model: MIT. Training data retains its original licences and attribution.
|
banner.jpg
ADDED
|
Git LFS Details
|
config.json
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"LlamaForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"attention_bias": false,
|
| 6 |
+
"attention_dropout": 0.0,
|
| 7 |
+
"bos_token_id": null,
|
| 8 |
+
"dtype": "float32",
|
| 9 |
+
"eos_token_id": 0,
|
| 10 |
+
"head_dim": 64,
|
| 11 |
+
"hidden_act": "silu",
|
| 12 |
+
"hidden_size": 384,
|
| 13 |
+
"initializer_range": 0.02,
|
| 14 |
+
"intermediate_size": 1024,
|
| 15 |
+
"max_position_embeddings": 1024,
|
| 16 |
+
"mlp_bias": false,
|
| 17 |
+
"model_type": "llama",
|
| 18 |
+
"num_attention_heads": 6,
|
| 19 |
+
"num_hidden_layers": 40,
|
| 20 |
+
"num_key_value_heads": 2,
|
| 21 |
+
"pad_token_id": 1,
|
| 22 |
+
"pretraining_tp": 1,
|
| 23 |
+
"rms_norm_eps": 1e-05,
|
| 24 |
+
"rope_parameters": {
|
| 25 |
+
"rope_theta": 10000.0,
|
| 26 |
+
"rope_type": "default"
|
| 27 |
+
},
|
| 28 |
+
"tie_word_embeddings": true,
|
| 29 |
+
"transformers_version": "5.0.0",
|
| 30 |
+
"use_cache": true,
|
| 31 |
+
"vocab_size": 16384
|
| 32 |
+
}
|
generation_config.json
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_from_model_config": true,
|
| 3 |
+
"eos_token_id": 0,
|
| 4 |
+
"output_attentions": false,
|
| 5 |
+
"output_hidden_states": false,
|
| 6 |
+
"pad_token_id": 1,
|
| 7 |
+
"transformers_version": "5.0.0",
|
| 8 |
+
"use_cache": false
|
| 9 |
+
}
|
harness.py
ADDED
|
@@ -0,0 +1,216 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Loom harness — the search half of Loom Spark 3.
|
| 3 |
+
|
| 4 |
+
The model never searches. It decides a lookup is needed and writes the query:
|
| 5 |
+
|
| 6 |
+
<lookup>france</lookup>
|
| 7 |
+
|
| 8 |
+
This script does the rest: searches Wikipedia, finds the ONE sentence most likely to
|
| 9 |
+
hold the answer, hands it back as a <result>, and lets the model answer from it.
|
| 10 |
+
|
| 11 |
+
python3 harness.py "what's the capital of france"
|
| 12 |
+
python3 harness.py # interactive
|
| 13 |
+
python3 harness.py --no-tools "who are you"
|
| 14 |
+
python3 harness.py --show "who wrote hamlet" # print what was searched and read
|
| 15 |
+
|
| 16 |
+
How it finds the answer, and why each step exists (all measured on live questions):
|
| 17 |
+
* searches the model's query AND the subject it can see in your question —
|
| 18 |
+
"whats the capital of france" searched as-is returns "Capital city" and "Das Kapital"
|
| 19 |
+
* prefers the real article over lists, films, albums and disambiguation pages
|
| 20 |
+
* reads the article's intro first, and further only when the intro has no answer of
|
| 21 |
+
the right kind (a height with a unit, a year, a number, a name)
|
| 22 |
+
* strips brackets and pronunciation guides, so real text looks like training text
|
| 23 |
+
* hands back ONE sentence. A 340-character window found the answer more often but the
|
| 24 |
+
model misread it four times in five; one sentence doubled the final score (15% -> 30%)
|
| 25 |
+
|
| 26 |
+
Swap search() for anything you like — the contract is text in, one sentence out.
|
| 27 |
+
Wikipedia needs no API key. Stdlib only.
|
| 28 |
+
"""
|
| 29 |
+
from __future__ import annotations
|
| 30 |
+
|
| 31 |
+
import argparse, json, re, ssl, sys, time, urllib.error, urllib.parse, urllib.request
|
| 32 |
+
|
| 33 |
+
try: # macOS system Python often lacks a CA bundle
|
| 34 |
+
import certifi
|
| 35 |
+
SSL_CTX = ssl.create_default_context(cafile=certifi.where())
|
| 36 |
+
except Exception:
|
| 37 |
+
SSL_CTX = ssl.create_default_context()
|
| 38 |
+
|
| 39 |
+
OLLAMA = "http://localhost:11434/api/generate"
|
| 40 |
+
MODEL = "hf.co/textilelabs/Loom-Tapestry-3"
|
| 41 |
+
API = "https://en.wikipedia.org/w/api.php?"
|
| 42 |
+
# Wikipedia returns 403 without a descriptive User-Agent.
|
| 43 |
+
UA = {"User-Agent": "LoomHarness/3.0 (Textile Labs; https://huggingface.co/textilelabs)"}
|
| 44 |
+
LOOKUP = re.compile(r"<lookup>(.*?)</lookup>", re.S)
|
| 45 |
+
_cache: dict = {}
|
| 46 |
+
|
| 47 |
+
# ------------------------------------------------------------------- the model
|
| 48 |
+
def loom(prompt: str, n: int = 64) -> str:
|
| 49 |
+
body = json.dumps({"model": MODEL, "prompt": prompt, "raw": True, "stream": False,
|
| 50 |
+
"options": {"temperature": 0, "num_predict": n,
|
| 51 |
+
"stop": ["<|eot|>", "<user>", "<result>"]}}).encode()
|
| 52 |
+
req = urllib.request.Request(OLLAMA, data=body, headers={"Content-Type": "application/json"})
|
| 53 |
+
with urllib.request.urlopen(req, timeout=120) as r:
|
| 54 |
+
return json.load(r)["response"].strip()
|
| 55 |
+
|
| 56 |
+
# ------------------------------------------------------------------ wikipedia
|
| 57 |
+
def _get(params: dict) -> dict:
|
| 58 |
+
key = json.dumps(params, sort_keys=True)
|
| 59 |
+
if key in _cache:
|
| 60 |
+
return _cache[key]
|
| 61 |
+
for attempt in range(3):
|
| 62 |
+
try:
|
| 63 |
+
with urllib.request.urlopen(urllib.request.Request(
|
| 64 |
+
API + urllib.parse.urlencode(params), headers=UA),
|
| 65 |
+
context=SSL_CTX, timeout=20) as r:
|
| 66 |
+
_cache[key] = json.load(r)
|
| 67 |
+
return _cache[key]
|
| 68 |
+
except urllib.error.HTTPError as e:
|
| 69 |
+
if e.code == 429:
|
| 70 |
+
time.sleep(3 * (attempt + 1)); continue
|
| 71 |
+
raise
|
| 72 |
+
raise RuntimeError("Wikipedia rate limit")
|
| 73 |
+
|
| 74 |
+
def search(q: str, n: int = 3) -> list:
|
| 75 |
+
return [h["title"] for h in _get({"action": "query", "list": "search", "srsearch": q,
|
| 76 |
+
"format": "json", "srlimit": n})["query"]["search"]]
|
| 77 |
+
|
| 78 |
+
def _extract(title: str, intro: bool) -> str:
|
| 79 |
+
p = {"action": "query", "prop": "extracts", "explaintext": 1, "titles": title,
|
| 80 |
+
"format": "json", "redirects": 1}
|
| 81 |
+
if intro:
|
| 82 |
+
p["exintro"] = 1
|
| 83 |
+
return next(iter(_get(p)["query"]["pages"].values())).get("extract", "") or ""
|
| 84 |
+
|
| 85 |
+
# ------------------------------------------------------------------ the finder
|
| 86 |
+
SENT = re.compile(r"(?<=[.!?])\s+(?=[A-Z0-9])")
|
| 87 |
+
PAREN = re.compile(r"\s*\([^()]*\)")
|
| 88 |
+
HEADING = re.compile(r"^\s*=+[^=]+=+\s*$", re.M)
|
| 89 |
+
STOP = set(("what whats who whos whom whose when where which why how is are was were be the a an "
|
| 90 |
+
"of in on to for does did do by from with as at and or that this it its there tell me "
|
| 91 |
+
"please can you many much").split())
|
| 92 |
+
ATTR = set(("capital city height tall high elevation population largest biggest smallest longest "
|
| 93 |
+
"shortest tallest highest deepest first last symbol chemical language languages spoken "
|
| 94 |
+
"legs year date end ended sink sank invented inventor discovered discovery developed "
|
| 95 |
+
"wrote written author painted painter president founded born died age old size area "
|
| 96 |
+
"distance speed").split())
|
| 97 |
+
JUNK = re.compile(r"^(lists? of|outline of|index of|timeline of)\b|\((film|album|song|band|"
|
| 98 |
+
r"novel|play|tv series|musical|opera|video game|book|composition|poem)\)|"
|
| 99 |
+
r"\bdisambiguation\b", re.I)
|
| 100 |
+
|
| 101 |
+
def keywords(t: str) -> list:
|
| 102 |
+
return [w for w in re.findall(r"[^\W_]+", t.lower()) if w not in STOP]
|
| 103 |
+
|
| 104 |
+
def subject(question: str) -> str:
|
| 105 |
+
kw = keywords(question)
|
| 106 |
+
return " ".join(k for k in kw if k not in ATTR) or " ".join(kw)
|
| 107 |
+
|
| 108 |
+
def clean(t: str) -> str:
|
| 109 |
+
prev = None
|
| 110 |
+
while prev != t:
|
| 111 |
+
prev, t = t, PAREN.sub("", t)
|
| 112 |
+
return re.sub(r"\s+", " ", t.replace(" ,", ",")).strip()
|
| 113 |
+
|
| 114 |
+
def _hard(q: str, s: str) -> float:
|
| 115 |
+
"""The answer is of the right KIND: a height with a unit, a year, a number, a name."""
|
| 116 |
+
b = 0.0
|
| 117 |
+
if re.search(r"\b(how tall|how high|height|elevation)\b", q):
|
| 118 |
+
b += 2.0 if re.search(r"\d[\d,.]*\s*(m|metres|meters|ft|feet|km)\b", s) else 0
|
| 119 |
+
if re.search(r"\b(when|what year|which year|what date)\b", q):
|
| 120 |
+
b += 2.0 if re.search(r"\b(1\d{3}|20\d{2})\b", s) else 0
|
| 121 |
+
if re.search(r"\b(how many|how much|population|number of)\b", q):
|
| 122 |
+
b += 1.5 if re.search(r"\d", s) else 0
|
| 123 |
+
if re.search(r"\bwho\b", q):
|
| 124 |
+
b += 1.5 if re.search(r"\b[A-Z][a-z]+ [A-Z][a-z]+", s) else 0
|
| 125 |
+
if re.search(r"\bsymbol\b", q):
|
| 126 |
+
b += 2.0 if re.search(r"\bsymbol\b", s, re.I) else 0
|
| 127 |
+
if re.search(r"\bcapital\b", q):
|
| 128 |
+
b += 2.0 if re.search(r"\bcapital\b", s, re.I) else 0
|
| 129 |
+
return b
|
| 130 |
+
|
| 131 |
+
def _kind(q: str, s: str) -> float:
|
| 132 |
+
b, sl = _hard(q, s), s.lower()
|
| 133 |
+
if re.search(r"\b(how tall|how high|height|elevation)\b", q):
|
| 134 |
+
b += 1.5 if re.search(r"\b(summit|elevation|height|above sea level|highest|stands)\b", sl) else -0.5
|
| 135 |
+
if re.search(r"\b(end|ended|finish|finished)\b", q):
|
| 136 |
+
b += 1.5 if re.search(r"\b(ended|end of|surrender|surrendered|concluded|finished)\b", sl) else -0.5
|
| 137 |
+
if re.search(r"\bpopulation\b", q):
|
| 138 |
+
b += 2.0 if re.search(r"\d{1,3}(,\d{3})+|\d+(\.\d+)?\s*(million|billion)", s) else -1.0
|
| 139 |
+
if re.search(r"\b(invent|invented|inventor|discovered|wrote|painted|composed|founded)\b", q):
|
| 140 |
+
b += 1.0 if re.search(r"\b[A-Z][a-z]+ (?:[A-Z][a-z]+ )?[A-Z][a-z]+\b", s) else 0.0
|
| 141 |
+
return b
|
| 142 |
+
|
| 143 |
+
def find(query: str, question: str) -> tuple:
|
| 144 |
+
"""One sentence most likely to hold the answer, and the article it came from."""
|
| 145 |
+
q = question.lower()
|
| 146 |
+
subj = subject(question)
|
| 147 |
+
pool = {}
|
| 148 |
+
for tq in dict.fromkeys(x for x in (query.strip(), subj, " ".join(keywords(question))) if x):
|
| 149 |
+
for rank, t in enumerate(search(tq, 3)):
|
| 150 |
+
tl = t.lower()
|
| 151 |
+
s = (4.0 if tl in (subj, query.strip().lower()) else 2.0 if subj and tl.startswith(subj) else 0.0)
|
| 152 |
+
s += -4.0 if JUNK.search(t) else 0.0
|
| 153 |
+
pool[t] = max(pool.get(t, -1e9), s - 0.3 * rank)
|
| 154 |
+
qk = list(dict.fromkeys(keywords(question) + keywords(query)))
|
| 155 |
+
top = sorted(pool.items(), key=lambda x: -x[1])[:3]
|
| 156 |
+
typed = bool(re.search(r"\b(how tall|how high|height|elevation|when|what year|which year|"
|
| 157 |
+
r"how many|how much|population|who|symbol|capital)\b", q))
|
| 158 |
+
best = (-1e9, "", "")
|
| 159 |
+
for intro in (True, False):
|
| 160 |
+
found_kind = False
|
| 161 |
+
for title, ps in top:
|
| 162 |
+
raw = _extract(title, intro)
|
| 163 |
+
if re.search(r"\b(may|can) refer to\b", raw[:400]):
|
| 164 |
+
continue
|
| 165 |
+
body = clean(HEADING.sub(" ", raw))
|
| 166 |
+
sents = [s.strip() for s in SENT.split(body) if 20 < len(s.strip()) < 600]
|
| 167 |
+
for i, s in enumerate(sents[: 14 if intro else 90]):
|
| 168 |
+
sc = ps + sum(1.0 for k in qk if k in s.lower()) + _kind(q, s) + (0.5 if i < 3 else 0.0)
|
| 169 |
+
if sc > best[0]:
|
| 170 |
+
best = (sc, s, title)
|
| 171 |
+
found_kind = _hard(q, s) > 0
|
| 172 |
+
if best[1] and (not typed or found_kind):
|
| 173 |
+
break # the intro held an answer of the right kind
|
| 174 |
+
return best[1], best[2]
|
| 175 |
+
|
| 176 |
+
# ------------------------------------------------------------------ the loop
|
| 177 |
+
def ask(message: str, tools: bool = True, show: bool = False) -> str:
|
| 178 |
+
convo = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message.strip()}\n<|eot|>\n<loom>\n"
|
| 179 |
+
first = loom(convo)
|
| 180 |
+
m = LOOKUP.search(first)
|
| 181 |
+
if not m:
|
| 182 |
+
return first
|
| 183 |
+
query = m.group(1).strip()
|
| 184 |
+
try:
|
| 185 |
+
result, source = find(query, message)
|
| 186 |
+
except Exception as e:
|
| 187 |
+
# Never feed an error in as if it were a result — the model will answer from it.
|
| 188 |
+
return f"[harness] lookup failed for {query!r}: {e}"
|
| 189 |
+
if not result:
|
| 190 |
+
return f"[harness] nothing found for {query!r}"
|
| 191 |
+
if show:
|
| 192 |
+
print(f" [searched: {query!r}]\n [read from {source}: {result[:150]}]")
|
| 193 |
+
return loom(convo + first + f"<|eot|>\n<result>\n{result}\n<|eot|>\n<loom>\n", n=48)
|
| 194 |
+
|
| 195 |
+
def main() -> int:
|
| 196 |
+
global MODEL
|
| 197 |
+
ap = argparse.ArgumentParser(description="Loom Spark 3 harness")
|
| 198 |
+
ap.add_argument("message", nargs="*")
|
| 199 |
+
ap.add_argument("--no-tools", action="store_true", help="chat only, no lookups")
|
| 200 |
+
ap.add_argument("--show", action="store_true", help="print the query and the sentence read")
|
| 201 |
+
ap.add_argument("--model", default=MODEL)
|
| 202 |
+
a = ap.parse_args()
|
| 203 |
+
MODEL = a.model
|
| 204 |
+
if a.message:
|
| 205 |
+
print(ask(" ".join(a.message), not a.no_tools, a.show)); return 0
|
| 206 |
+
print(f"Loom harness — {MODEL} (tools {'off' if a.no_tools else 'on'}, ctrl-c to quit)\n")
|
| 207 |
+
while True:
|
| 208 |
+
try:
|
| 209 |
+
msg = input("you > ").strip()
|
| 210 |
+
except (EOFError, KeyboardInterrupt):
|
| 211 |
+
print(); return 0
|
| 212 |
+
if msg:
|
| 213 |
+
print(f"loom > {ask(msg, not a.no_tools, a.show)}\n")
|
| 214 |
+
|
| 215 |
+
if __name__ == "__main__":
|
| 216 |
+
sys.exit(main())
|
logo.jpg
ADDED
|
Git LFS Details
|
loom-tapestry-3-f16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0e9ceacc4d2198f55ffa1b4dd8a4bb1197fa9f30eea13391a9456a322fd49bc9
|
| 3 |
+
size 139123040
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ac14586054b5ac511c24fb518368abbbb367b119af25a7d09d33ae7f722b2376
|
| 3 |
+
size 276988720
|
params
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"stop": ["<|eot|>", "<user>", "<result>"],
|
| 3 |
+
"temperature": 0.7,
|
| 4 |
+
"top_k": 40,
|
| 5 |
+
"repeat_penalty": 1.0,
|
| 6 |
+
"num_predict": 96
|
| 7 |
+
}
|
special_tokens_map.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"eos_token":"<|eot|>","pad_token":"<|pad|>","additional_special_tokens":["<tools:on>","<tools:off>","<user>","<loom>","<result>","<lookup>","</lookup>"]}
|
template
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<tools:off>
|
| 2 |
+
<user>
|
| 3 |
+
{{ .Prompt }}
|
| 4 |
+
<|eot|>
|
| 5 |
+
<loom>
|
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"tokenizer_class":"PreTrainedTokenizerFast","model_max_length":1024,"eos_token":"<|eot|>","pad_token":"<|pad|>","additional_special_tokens":["<tools:on>","<tools:off>","<user>","<loom>","<result>","<lookup>","</lookup>"],"clean_up_tokenization_spaces":false}
|