RoeiG commited on
Commit
f16b495
·
verified ·
1 Parent(s): ae18f3a

Card: head weights are trained from scratch

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -44,7 +44,8 @@ call time. It is not a chatbot and it does not generate text.
44
 
45
  - **Encoder:** [`dicta-il/neodictabert-bilingual`](https://huggingface.co/dicta-il/neodictabert-bilingual) (NeoBERT, 28
46
  layers, Hebrew + English)
47
- - **Head:** Laya's `DecisionModel` head, with 2 transformer layers and a scorer over the option markers
 
48
  - **Size:** 378M parameters, about 120–230 ms per call on an Apple M1 CPU
49
  - **Input:** up to 1,024 tokens of state, and up to 256 tokens of options
50
  - **Training:** Laya's RLCD objective (proper-scoring-rule rewards plus soft cross-entropy), with a temperature for each
 
44
 
45
  - **Encoder:** [`dicta-il/neodictabert-bilingual`](https://huggingface.co/dicta-il/neodictabert-bilingual) (NeoBERT, 28
46
  layers, Hebrew + English)
47
+ - **Head:** Laya's `DecisionModel` architecture (2 transformer layers and a scorer over the option markers), trained
48
+ from scratch. No weights come from Laya's published checkpoints; the only pretrained weights are the encoder's.
49
  - **Size:** 378M parameters, about 120–230 ms per call on an Apple M1 CPU
50
  - **Input:** up to 1,024 tokens of state, and up to 256 tokens of options
51
  - **Training:** Laya's RLCD objective (proper-scoring-rule rewards plus soft cross-entropy), with a temperature for each