README.md

---
license: apache-2.0
language:
  - en
tags:
  - text-to-legend
  - pure
  - legend
  - tokenformer
  - text-generation
base_model: Qwen/Qwen3-14B
library_name: transformers
pipeline_tag: text-generation
---

# Legend Query Generation

Fine-tuned from [Qwen/Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B) using
[ScalarLM](https://github.com/supermassive-intelligence/scalarlm) with
tokenformer adapters.

## Model Details

- **Base model**: Qwen/Qwen3-14B
- **Fine-tuning method**: Tokenformer adapters (additive MLP adapters with learned attention)
- **Checkpoint step**: 6656
- **Trained parameters**: 442 (attention projections, layernorms, embeddings, tokenformer adapters)
- **Tokenformer config**: num_heads=4, r=32

## How to Use

This model uses custom tokenformer adapter layers on top of the base model.
It requires the `TokenformerAdapter` module for inference. Standard
`AutoModelForCausalLM.from_pretrained` will not load the adapter weights
correctly.

See the [ScalarLM repository](https://github.com/supermassive-intelligence/scalarlm)
for inference instructions.

## Training

Trained on Legend query generation data using the Qwen chat template format.

**Prompt format:**
```
<|im_start|>user
{question}

Here is some schema information:
{schema_text}

First, plan how to write the query, then write it in a ```pure SQL``` code block.
<|im_end|>
<|im_start|>assistant
```

## License

Apache 2.0