| --- |
| language: |
| - en |
| license: apache-2.0 |
| library_name: transformers |
| tags: |
| - toke |
| - code-generation |
| - programming-language |
| - qwen2 |
| - qlora |
| - fine-tuned |
| - awq |
| - 4bit |
| base_model: Qwen/Qwen2.5-Coder-7B-Instruct |
| pipeline_tag: text-generation |
| model-index: |
| - name: toke-7b-gate2 |
| results: |
| - task: |
| type: text-generation |
| name: Code Generation |
| metrics: |
| - name: Compilation Pass@1 |
| type: pass@1 |
| value: 100 |
| verified: true |
| - name: Functional Pass@1 |
| type: pass@1 |
| value: 8 |
| verified: true |
| --- |
| |
| # toke-7b-gate2 |
|
|
| A 7B parameter language model fine-tuned to generate code in **toke**, a programming language designed to reduce token cost of AI-generated code. This model writes syntactically valid toke **100% of the time**. |
|
|
| ## Model Details |
|
|
| | Property | Value | |
| |---|---| |
| | **Base model** | [Qwen 2.5 Coder 7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) | |
| | **Method** | QLoRA (rank 64, alpha 128, 3 epochs) | |
| | **Training data** | 25,953 records — 18,890 synthetic + 6,069 from [loke](https://github.com/karwalski/loke) production (87K lines) | |
| | **Training time** | 37 hours on NVIDIA A10G (24 GB) | |
| | **Weights** | AWQ 4-bit quantized (this repo) | |
| | **Context length** | 32,768 tokens | |
| | **License** | Apache 2.0 | |
|
|
|
|
| ## What is toke? |
|
|
| toke is a statically typed, compiled language with a **55-character alphabet** (lowercase a-z, digits 0-9, and 19 symbols). It compiles to native binaries via LLVM. A purpose-built BPE tokenizer achieves **52% fewer tokens** on average vs cl100k_base. |
| |
| - **13 keywords:** `m` `f` `t` `i` `if` `el` `lp` `br` `let` `mut` `as` `rt` `mt` |
| - **No comments** in source — documentation lives in companion files (.tkc.md) |
| - **Errors as values** — no exceptions, result types with `mt` (match) |
| - **Website:** [tokelang.dev](https://tokelang.dev) | **Console:** [console.tokelang.dev](https://console.tokelang.dev) |
| |
| ## Gate 2 Results (May 2026) |
| |
| | Metric | Gate 1 | Gate 2 | |
| |---|---|---| |
| | Compilation Pass@1 | 63.7% | **100%** | |
| | Tasks evaluated | 1,000 | 700 | |
| | Functional Pass@1 | — | ~8% | |
| | Training records | 73,000 | 25,953 | |
| |
| ## Usage |
| |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model = AutoModelForCausalLM.from_pretrained("karwalski/toke", device_map="auto") |
| tokenizer = AutoTokenizer.from_pretrained("karwalski/toke") |
|
|
| prompt = """<|im_start|>system |
| Write toke programs. m=mod; f=name(p:type):ret{body}; let x=42; <expr return. Semicolons everywhere. Start with m=. |
| <|im_end|> |
| <|im_start|>user |
| Write a hello world program |
| <|im_end|> |
| <|im_start|>assistant |
| """ |
|
|
| inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
| outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.2, do_sample=True) |
| print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)) |
| # m=hello;i=io:std.io;f=main():i64{io.println("hello world");<0}; |
| ``` |
| |
| ## API Access |
| |
| Free API access via [console.tokelang.dev](https://console.tokelang.dev) — no credit card required. |
| |
| ```bash |
| curl -X POST https://api.tokelang.dev/v1/generate \ |
| -H "X-Api-Key: YOUR_KEY" \ |
| -H "Content-Type: application/json" \ |
| -d '{"description": "Return the absolute value of an integer"}' |
| ``` |
| |
| ## Links |
| |
| - [tokelang.dev](https://tokelang.dev) — Project website |
| - [console.tokelang.dev](https://console.tokelang.dev) — Free API access |
| - [GitHub: toke](https://github.com/karwalski/toke) — Compiler, spec, stdlib |
| - [GitHub: loke](https://github.com/karwalski/loke) — 87K lines of production toke |
| - [Live tokenizer](https://tokelang.dev/tokenizer) — Compare token counts in-browser |
| |
