---
language:
  - en
license: apache-2.0
library_name: transformers
tags:
  - toke
  - code-generation
  - programming-language
  - qwen2
  - qlora
  - fine-tuned
  - awq
  - 4bit
base_model: Qwen/Qwen2.5-Coder-7B-Instruct
pipeline_tag: text-generation
model-index:
  - name: toke-7b-gate2
    results:
      - task:
          type: text-generation
          name: Code Generation
        metrics:
          - name: Compilation Pass@1
            type: pass@1
            value: 100
            verified: true
          - name: Functional Pass@1
            type: pass@1
            value: 8
            verified: true
---

# toke-7b-gate2

A 7B parameter language model fine-tuned to generate code in **toke**, a programming language designed to reduce token cost of AI-generated code. This model writes syntactically valid toke **100% of the time**.

## Model Details

| Property | Value |
|---|---|
| **Base model** | [Qwen 2.5 Coder 7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) |
| **Method** | QLoRA (rank 64, alpha 128, 3 epochs) |
| **Training data** | 25,953 records — 18,890 synthetic + 6,069 from [loke](https://github.com/karwalski/loke) production (87K lines) |
| **Training time** | 37 hours on NVIDIA A10G (24 GB) |
| **Weights** | AWQ 4-bit quantized (this repo) |
| **Context length** | 32,768 tokens |
| **License** | Apache 2.0 |


## What is toke?

toke is a statically typed, compiled language with a **55-character alphabet** (lowercase a-z, digits 0-9, and 19 symbols). It compiles to native binaries via LLVM. A purpose-built BPE tokenizer achieves **52% fewer tokens** on average vs cl100k_base.

- **13 keywords:** `m` `f` `t` `i` `if` `el` `lp` `br` `let` `mut` `as` `rt` `mt`
- **No comments** in source — documentation lives in companion files (.tkc.md)
- **Errors as values** — no exceptions, result types with `mt` (match)
- **Website:** [tokelang.dev](https://tokelang.dev) | **Console:** [console.tokelang.dev](https://console.tokelang.dev)

## Gate 2 Results (May 2026)

| Metric | Gate 1 | Gate 2 |
|---|---|---|
| Compilation Pass@1 | 63.7% | **100%** |
| Tasks evaluated | 1,000 | 700 |
| Functional Pass@1 | — | ~8% |
| Training records | 73,000 | 25,953 |

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("karwalski/toke", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("karwalski/toke")

prompt = """<|im_start|>system
Write toke programs. m=mod; f=name(p:type):ret{body}; let x=42; <expr return. Semicolons everywhere. Start with m=.
<|im_end|>
<|im_start|>user
Write a hello world program
<|im_end|>
<|im_start|>assistant
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.2, do_sample=True)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# m=hello;i=io:std.io;f=main():i64{io.println("hello world");<0};
```

## API Access

Free API access via [console.tokelang.dev](https://console.tokelang.dev) — no credit card required.

```bash
curl -X POST https://api.tokelang.dev/v1/generate \
  -H "X-Api-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"description": "Return the absolute value of an integer"}'
```

## Links

- [tokelang.dev](https://tokelang.dev) — Project website
- [console.tokelang.dev](https://console.tokelang.dev) — Free API access
- [GitHub: toke](https://github.com/karwalski/toke) — Compiler, spec, stdlib
- [GitHub: loke](https://github.com/karwalski/loke) — 87K lines of production toke
- [Live tokenizer](https://tokelang.dev/tokenizer) — Compare token counts in-browser