| --- |
| license: mit |
| base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B |
| tags: |
| - iol-ai-2026 |
| - reasoning |
| - awq |
| - rag |
| --- |
| |
| # DeepSeek-R1-Distill-Qwen-7B-AWQ book-RAG submission |
|
|
| This is a training-free IOL-AI submission using a 4-bit AWQ conversion of |
| `deepseek-ai/DeepSeek-R1-Distill-Qwen-7B`. It retrieves only from the public, |
| string-only extraction of Vlad A. Neacșu's *Linguistics Olympiad: Training |
| guide*. Private curated training data are not present or required at runtime. |
|
|
| The checkpoint uses `Qwen2ForCausalLM`, so it is compatible with the challenge's |
| Transformers 4.44.1 and AutoAWQ 0.2.7 runtime. The included weights are from |
| `casperhansen/deepseek-r1-distill-qwen-7b-awq`, revision |
| `305e6f12907dc78ae61a1f0bb7a19faa2b25e8a3`, which is an AWQ conversion of the |
| official DeepSeek model. |
|
|
| ## Challenge execution |
|
|
| ```bash |
| python script.py |
| ``` |
|
|
| The script reads `/tmp/data/test.csv` and writes `submission.csv`. Its columns |
| are `id,pred,explanation`; `pred` is a JSON-encoded list of answer strings. |
| Explanation generation is disabled by default, leaving that optional column |
| blank. It can be restored with: |
|
|
| ```bash |
| python script.py --explanations on |
| ``` |
|
|
| For local data: |
|
|
| ```bash |
| python script.py --input path/to/test.csv --output submission.csv |
| python script.py --self-test |
| ``` |
|
|
| After the model loads successfully, the output file is initialized and then |
| atomically rewritten after every completed row. If evaluation reaches its time |
| limit, predictions already completed remain in a valid submission file. |
|
|
| ## DeepSeek reasoning and generation |
|
|
| DeepSeek's solver instructions, retrieved context, and current problem are |
| placed in one user message rather than a separate system message. The assistant |
| prompt is prefixed with `<think>` to engage the distilled reasoning behavior. |
| Reasoning stops when `</think>` appears or at its configured cap. The runtime |
| then starts a separate `FINAL ANSWERS:` stage, guaranteeing that reasoning |
| cannot consume the answer budget. Only that answer block is serialized into |
| `pred`. |
|
|
| Defaults follow DeepSeek's recommended sampling values while retaining a bounded |
| challenge-time output: |
|
|
| - `temperature=0.6` |
| - `top_p=0.95` |
| - sampling enabled |
| - `max_reasoning_tokens=4096` |
| - `max_answer_tokens=1024` |
| - one answer-only retry with `answer_retry_tokens=512` |
|
|
| The model context is capped at 32,768 tokens. The script loads AWQ directly in |
| FP16, enables the KV cache, and applies an inference-only last-token `lm_head` |
| hook to avoid materializing full-prompt FP32 vocabulary logits on the T4. |
|
|
| ## Book-only retrieval |
|
|
| The default prompt includes one general book method and two worked book |
| examples. Retrieval combines dependency-free BM25, character 3–5-gram TF-IDF, |
| task-family inference, and book metadata boosts. It uses only the current |
| problem's `context + query`; answers and private curated datasets are never |
| indexed. |
|
|
| Settings are in `rag_resources/config.json`. Environment overrides are available |
| for `IOL_TOP_METHODS`, `IOL_TOP_EXAMPLES`, `IOL_CHAR_TFIDF_WEIGHT`, |
| `IOL_RAG_MAX_CHARS`, `IOL_MAX_REASONING_TOKENS`, |
| `IOL_MAX_ANSWER_TOKENS`, `IOL_ANSWER_RETRY_TOKENS`, |
| `IOL_EXPLANATION_MAX_NEW_TOKENS`, |
| `IOL_ENABLE_EXPLANATIONS`, and `IOL_SEED`. |
|
|
| ## Sources and licenses |
|
|
| - [DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) |
| - [AWQ conversion](https://huggingface.co/casperhansen/deepseek-r1-distill-qwen-7b-awq) |
| - [IOL-AI 2026 challenge](https://iolai.org/) |
| - [Language Science Press book source](https://github.com/langsci/420) |
|
|
| The model and conversion are marked MIT. The book-derived resources retain |
| their own license and attribution in `rag_resources/ATTRIBUTION.md`. |
|
|
