Instructions to use lumierenoir/klev-0.8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use lumierenoir/klev-0.8b with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Unsloth Desktop
base/: usage notes and the two easy mistakes (temperature 2.2449, and re-derive the delimiter ids)
Browse files- base/README.md +28 -5
base/README.md
CHANGED
|
@@ -66,8 +66,31 @@ kev's *own* decision state already recovers the hate-speech signal with no exter
|
|
| 66 |
which is why there is little left for the stitch to add. That holds across two different
|
| 67 |
external adapters, so it looks like a property of kev's decision representation on this task.
|
| 68 |
|
| 69 |
-
##
|
| 70 |
-
|
| 71 |
-
Identical to the top-level card except
|
| 72 |
-
`
|
| 73 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
which is why there is little left for the stitch to add. That holds across two different
|
| 67 |
external adapters, so it looks like a property of kev's decision representation on this task.
|
| 68 |
|
| 69 |
+
## Using it
|
| 70 |
+
|
| 71 |
+
Identical to the top-level card except the base — see its **Usage** section for the full guide
|
| 72 |
+
(in-process `model.infer.answer()`, the `examples/system_one.py` CLI, and the optional
|
| 73 |
+
`serve/server.py`). For this arm:
|
| 74 |
+
|
| 75 |
+
```python
|
| 76 |
+
from model.infer import load_klev, answer
|
| 77 |
+
import unsloth # noqa: F401 — must precede transformers / peft
|
| 78 |
+
|
| 79 |
+
# point at the parent repo and the base/ prefix; load_klev downloads adapter/* and head.pt,
|
| 80 |
+
# so for this arm copy or symlink them, or pass a local dir with them under adapter/ + head.pt
|
| 81 |
+
model, tokenizer, head = load_klev("path/to/a/dir/containing/base", preset="qwen35-08b-base",
|
| 82 |
+
base="Qwen/Qwen3.5-0.8B-Base")
|
| 83 |
+
answer(model, tokenizer, head, request, temperature=2.2449) # <- 2.2449, not 2.2974
|
| 84 |
+
```
|
| 85 |
+
|
| 86 |
+
Two things differ from the IT arm and both are easy to get wrong:
|
| 87 |
+
|
| 88 |
+
- **serve at temperature 2.2449**, not 2.2974. `head.pt` here also stores 1.0, and the raw
|
| 89 |
+
readout is badly overconfident.
|
| 90 |
+
- **re-derive the delimiter ids** rather than assuming. They happen to be the same on this
|
| 91 |
+
checkpoint (248060..248064) but nothing guarantees it; `load_klev(preset="qwen35-08b-base")`
|
| 92 |
+
does it for you, and using the wrong set yields a row layout with zero delimiters found
|
| 93 |
+
rather than a clean error.
|
| 94 |
+
|
| 95 |
+
If you have both arms, note that the two are alternatives, not one model plus a delta: the
|
| 96 |
+
`base/` weights are a separate training run of the same recipe.
|