---
license: mit
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
tags:
  - iol-ai-2026
  - reasoning
  - awq
  - rag
---

# DeepSeek-R1-Distill-Qwen-7B-AWQ book-RAG submission

This is a training-free IOL-AI submission using a 4-bit AWQ conversion of
`deepseek-ai/DeepSeek-R1-Distill-Qwen-7B`. It retrieves only from the public,
string-only extraction of Vlad A. Neacșu's *Linguistics Olympiad: Training
guide*. Private curated training data are not present or required at runtime.

The checkpoint uses `Qwen2ForCausalLM`, so it is compatible with the challenge's
Transformers 4.44.1 and AutoAWQ 0.2.7 runtime. The included weights are from
`casperhansen/deepseek-r1-distill-qwen-7b-awq`, revision
`305e6f12907dc78ae61a1f0bb7a19faa2b25e8a3`, which is an AWQ conversion of the
official DeepSeek model.

## Challenge execution

```bash
python script.py
```

The script reads `/tmp/data/test.csv` and writes `submission.csv`. Its columns
are `id,pred,explanation`; `pred` is a JSON-encoded list of answer strings.
Explanation generation is disabled by default, leaving that optional column
blank. It can be restored with:

```bash
python script.py --explanations on
```

For local data:

```bash
python script.py --input path/to/test.csv --output submission.csv
python script.py --self-test
```

After the model loads successfully, the output file is initialized and then
atomically rewritten after every completed row. If evaluation reaches its time
limit, predictions already completed remain in a valid submission file.

## DeepSeek reasoning and generation

DeepSeek's solver instructions, retrieved context, and current problem are
placed in one user message rather than a separate system message. The assistant
prompt is prefixed with `<think>` to engage the distilled reasoning behavior.
Reasoning stops when `</think>` appears or at its configured cap. The runtime
then starts a separate `FINAL ANSWERS:` stage, guaranteeing that reasoning
cannot consume the answer budget. Only that answer block is serialized into
`pred`.

Defaults follow DeepSeek's recommended sampling values while retaining a bounded
challenge-time output:

- `temperature=0.6`
- `top_p=0.95`
- sampling enabled
- `max_reasoning_tokens=4096`
- `max_answer_tokens=1024`
- one answer-only retry with `answer_retry_tokens=512`

The model context is capped at 32,768 tokens. The script loads AWQ directly in
FP16, enables the KV cache, and applies an inference-only last-token `lm_head`
hook to avoid materializing full-prompt FP32 vocabulary logits on the T4.

## Book-only retrieval

The default prompt includes one general book method and two worked book
examples. Retrieval combines dependency-free BM25, character 3–5-gram TF-IDF,
task-family inference, and book metadata boosts. It uses only the current
problem's `context + query`; answers and private curated datasets are never
indexed.

Settings are in `rag_resources/config.json`. Environment overrides are available
for `IOL_TOP_METHODS`, `IOL_TOP_EXAMPLES`, `IOL_CHAR_TFIDF_WEIGHT`,
`IOL_RAG_MAX_CHARS`, `IOL_MAX_REASONING_TOKENS`,
`IOL_MAX_ANSWER_TOKENS`, `IOL_ANSWER_RETRY_TOKENS`,
`IOL_EXPLANATION_MAX_NEW_TOKENS`,
`IOL_ENABLE_EXPLANATIONS`, and `IOL_SEED`.

## Sources and licenses

- [DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B)
- [AWQ conversion](https://huggingface.co/casperhansen/deepseek-r1-distill-qwen-7b-awq)
- [IOL-AI 2026 challenge](https://iolai.org/)
- [Language Science Press book source](https://github.com/langsci/420)

The model and conversion are marked MIT. The book-derived resources retain
their own license and attribution in `rag_resources/ATTRIBUTION.md`.