| # Lex architecture | |
| Lex fine-tunes Kai's 571,909,635-parameter encoder architecture. Three independently parameterized 22-layer paths serve Choice, Noul and Score. Each path has its own final normalization, two interaction layers and a candidate scorer; token embeddings and embedding normalization are shared. | |
|  | |
| A question and all of its candidate descriptions are encoded jointly with the context. Marker states supply scalar candidate scores, normalized over the valid candidates. A shared type embedding is added after each path's final normalization. Each needed encoder path processes the full padded batch; interaction heads gather rows of that type. | |
|  | |
| Choice returns a candidate and probabilities; Noul returns the probability of yes; Score returns the ordered distribution and its expected value. Candidate IDs preserve associations outside the token stream. Multiple questions use batches, with repeated context encoding. | |
| The public API accepts complete packed inputs of at most 1,024 tokens per question. No field is silently truncated. | |
| [Main SVG](assets/architecture.svg) · [Residual layers](assets/residual-layers.svg) · [Attention and GEGLU](assets/attention-geglu.svg) · [PDF atlas](assets/architecture-atlas.pdf) | |