--- base_model: unsloth/Qwen2.5-3B-Instruct-bnb-4bit library_name: peft pipeline_tag: text-generation language: - en license: other license_name: qwen-research license_link: https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE tags: - qlora - lora - peft - trl - sft - unsloth - theory-of-mind - tombench - social-reasoning - small-language-model widget: - text: >- Story: Anne puts the ball in the basket and leaves the room. While she is away, Sally moves the ball to the box. Question: When Anne returns, where will she look for the ball? (A) the basket (B) the box. Answer with the letter only. example_title: False belief (first-order, ToMi-style) - text: >- Story: Tom takes a cookie from the jar and hides it in his pocket without Jerry noticing, because Jerry was looking out the window at that moment. Question: Does Jerry know that Tom has a cookie in his pocket? (A) Yes (B) No. Answer with the letter only. example_title: Knowledge / information access - text: >- Situation: Jan arrived at the party and found out it was a surprise party for her. Question: How would Jan likely feel? (A) surprised and happy (B) angry (C) indifferent. Answer with the letter only. example_title: Social commonsense (SocialIQa-style) - text: >- Story: Alice hides a key in a drawer while Bob watches from another room. Bob does not know that Carol, who was also watching, saw Alice hide the key. Question: What does Bob think Carol believes about the key's location? (A) Bob thinks Carol knows the key is in the drawer (B) Bob thinks Carol does not know where the key is. Answer with the letter only. example_title: Higher-order belief (Hi-ToM-style) model-index: - name: qwen2.5-3b-tombench-qlora results: - task: type: text-generation name: Theory of Mind Reasoning dataset: type: tombench name: ToMBench (held-out test split, seed 42) metrics: - type: accuracy value: 75.18 name: Accuracy - task: type: text-generation name: Theory of Mind Reasoning dataset: type: tomi name: ToMi metrics: - type: accuracy value: 83.00 name: Accuracy - task: type: text-generation name: Theory of Mind Reasoning dataset: type: opentom name: OpenToM metrics: - type: accuracy value: 59.50 name: Accuracy - type: f1 value: 47.91 name: Macro F1 - task: type: text-generation name: Theory of Mind Reasoning dataset: type: social_i_qa name: SocialIQa metrics: - type: accuracy value: 61.70 name: Accuracy - task: type: text-generation name: Theory of Mind Reasoning dataset: type: hi_tom name: Hi-ToM metrics: - type: accuracy value: 62.30 name: Accuracy --- # Qwen2.5-3B-Instruct + QLoRA — Theory-of-Mind (ToMBench) A QLoRA adapter for `unsloth/Qwen2.5-3B-Instruct-bnb-4bit`, fine-tuned on a stratified split of **ToMBench** to study whether Theory-of-Mind (ToM) reasoning gains in a 3B-parameter model generalize across structurally different social-reasoning tasks. The short answer from our evaluation: **they do not generalize uniformly, and the robustness of that transfer itself differs by task.** Fine-tuning produces a large in-domain gain (replicated across five training seeds) and positive transfer to a structurally similar task (ToMi, also robust across five seeds). It produces a robust *negative* transfer to social-commonsense reasoning (SocialIQa, the most seed-stable effect in the study). Narrative-based belief tracking (OpenToM) shows a significant decline in the primary run but is seed-sensitive, not confirmed as robust, and substantially attributable to a response-position bias interacting with this benchmark's severe class imbalance rather than genuine negative transfer (see below). Higher-order recursive belief reasoning (Hi-ToM) shows **no reliable net effect** across five seeds at the aggregate level; a follow-up check found a significant residual gain once the most extreme gold-answer-position stratum is excluded, but that residual disappears under a complementary position-balanced accuracy check, so we treat it as an open hypothesis rather than an established finding. We call this **selective transfer**, and it is the central finding of the accompanying manuscript. - **Code, data-split indices, per-item predictions, and statistics:** https://github.com/NextMina1131/tom-slm-qlora (commit `e3f155c`, 2026-09-10) --- ## Model details | | | |---|---| | **Developed by** | Ji-Hyeong Hong (Hoseo Technical College) | | **Advised by** | Prof. Sang-Hyun Yoo (corresponding author), School of Computer Science and Engineering, Soongsil University | | **Model type** | Causal decoder-only LM adapter (QLoRA / LoRA) | | **Language** | English | | **Base model** | [`unsloth/Qwen2.5-3B-Instruct-bnb-4bit`](https://huggingface.co/unsloth/Qwen2.5-3B-Instruct-bnb-4bit) (4-bit pre-quantized) | | **Adapter config** | LoRA, r = 16, α = 16 | | **Training framework** | Unsloth + TRL `SFTTrainer` + PEFT + bitsandbytes | | **License** | Inherits the base model's license (see YAML header) | This repository contains **adapter weights only** (trained with training seed 42; four additional replication seeds 43–46 were evaluated for the robustness checks below but their adapters are not separately hosted). The base model must be loaded separately. --- ## Intended use ### Direct use Research on Theory-of-Mind and social reasoning in small language models. Specifically: - Reproducing the cross-benchmark generalization results reported below. - Studying how supervised fine-tuning on one slice of ToM reasoning affects performance on structurally different social-reasoning tasks, and how robust that effect is across training seeds. - As a baseline for work on explicit belief-state representation, higher-order belief tracking, or parameter-efficient adaptation of small models to social reasoning. ### Downstream use The adapter can serve as a starting point for further parameter-efficient fine-tuning on ToM-adjacent tasks. Given the robust negative transfer we observed on SocialIQa, we recommend evaluating on your target distribution before assuming any gain carries over — and, per the findings below, evaluating across multiple training seeds rather than a single run. ### Out-of-scope use This model is **not** suitable for: - Any application that infers the mental states, beliefs, or intentions of real people — including psychological assessment, clinical screening, hiring, education, or user profiling. The model was trained and evaluated on synthetic and crowd-authored benchmark narratives, not on real human interaction, and our own results show its ToM performance does not transfer reliably even across benchmarks. - Deployment in high-stakes or safety-critical settings. - Non-English input. All training and evaluation data were English. --- ## Bias, risks, and limitations > **⚠️ Deployment warning — SocialIQa decline:** Fine-tuning on ToMBench produces a **robust > negative transfer to broad social-commonsense reasoning.** Across all five training seeds, > accuracy on SocialIQa consistently declined (mean −5.28 ± 0.55 pp; negative under every seed, > McNemar p < 0.001 in all five seeds). This is the single most seed-stable finding in the study. > **Do not use this adapter in any application where social-commonsense reasoning matters.** > See Section 5.5 of the accompanying manuscript for full details. - **Selective transfer, with task-dependent robustness, is a limitation of the model, not just a finding.** Across five training seeds, accuracy dropped robustly on SocialIQa (−6.00pp in the primary run; mean −5.28 ± 0.55pp, negative under every seed). The decline on OpenToM (−3.90pp in the primary run) was significant but **not robust**: its five-seed mean crossed zero (−0.56 ± 1.96pp), and a position-balanced accuracy check shows the decline shrinks substantially (−2.42pp vs. −3.90pp full-sample), consistent with this benchmark's severe class imbalance (59.7% gold-B) contributing substantially to the full-sample figure. Hi-ToM showed **no reliable net effect** in either direction (mean +0.94 ± 0.84pp across five seeds) — a follow-up item-level analysis found a large internal redistribution tied to the *position* of the correct answer (gold-answer-E items dropped 16.67pp; permutation test p = 0.0002), and excluding that stratum reveals a significant +3.84pp residual gain; however, a complementary position-balanced accuracy check across all letters is essentially null (+0.05pp), so whether this residual reflects a genuine effect remains an open question rather than an established result. - **In-domain gain is partly inflated by story overlap.** 55.1% of ToMBench's underlying stories are reused across multiple items; a story-disjoint group split reduced the in-domain gain from +13.66pp to +10.39pp (still significant, McNemar exact p = 7.2×10⁻¹⁰), suggesting part of the original item-level gain reflects recognizing a previously seen story rather than novel generalization. - **Benchmark-format sensitivity.** We measured strict output-format compliance directly: it rose from 96.56% to 100.00% on ToMBench after fine-tuning, but this can account for at most a small fraction of the 13.66pp gain, and it moved in the *opposite* direction from the SocialIQa and OpenToM declines — so those declines are not artifacts of parsing failure. - **Multi-seed replication was performed, but adapter diversity was not.** Five training seeds (42–46) were used to check the robustness of the *evaluation* results (see below); only the seed-42 adapter is hosted in this repository. - **Inherited base-model biases.** The adapter does not mitigate, and may amplify, social and cultural biases present in Qwen2.5-3B-Instruct and in the benchmark data. - **Narrative-only evaluation.** All benchmarks consist of short written scenarios. Nothing here speaks to mental-state inference in dialogue, multimodal, or real-time settings. - **Pretraining contamination cannot be ruled out.** Qwen2.5's pretraining corpus is not fully disclosed. ### Recommendations Treat reported gains as in-distribution only, and treat any single-seed cross-benchmark result with caution — this study found that some apparent effects (OpenToM) do not survive multi-seed replication, and others (Hi-ToM) show a null aggregate that may or may not conceal a genuine residual effect depending on which position-bias check is used. If you use this adapter, evaluate on your own task before assuming improvement, and prefer the base model when broad social-commonsense reasoning matters, since that is this study's most robust negative finding. --- ## How to get started ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel BASE = "unsloth/Qwen2.5-3B-Instruct-bnb-4bit" ADAPTER = "nextmina/qwen2.5-3b-tombench-qlora" tokenizer = AutoTokenizer.from_pretrained(BASE) model = AutoModelForCausalLM.from_pretrained(BASE, device_map="auto") model = PeftModel.from_pretrained(model, ADAPTER) model.eval() messages = [ { "role": "user", "content": ( "Story: Anne puts the ball in the basket and leaves the room. " "While she is away, Sally moves the ball to the box.\n" "Question: When Anne returns, where will she look for the ball?\n" "(A) the basket (B) the box\n" "Answer with the letter only." ), } ] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) inputs = tokenizer(prompt, return_tensors="pt").to(model.device) out = model.generate(**inputs, max_new_tokens=8, do_sample=False) print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)) ``` With Unsloth: ```python from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="nextmina/qwen2.5-3b-tombench-qlora", max_seq_length=2048, load_in_4bit=True, ) FastLanguageModel.for_inference(model) ``` --- ## Training details ### Training data **ToMBench**, partitioned into train / validation / test splits **stratified by ATOMS ability category** with `seed = 42`, so that every ToM ability category is proportionally represented in each split (1,607 train / 411 validation / 842 test items, out of 2,860 total). Raw benchmark data are **not redistributed** here. The exact item indices used for each split are published in the `splits/` directory of the [GitHub repository](https://github.com/NextMina1131/tom-slm-qlora), so the partition can be reconstructed once ToMBench is obtained from its original source. ### Training procedure Supervised fine-tuning (SFT) with TRL's `SFTTrainer` on a 4-bit pre-quantized base model, using a QLoRA adapter. Only adapter parameters were updated; base weights remained frozen and quantized. #### Training hyperparameters | Parameter | Value | |---|---| | Training regime | 4-bit base (NF4) + LoRA adapter (fp16; the T4 GPU used does not support bf16 tensor cores) | | LoRA rank (r) | 16 | | LoRA alpha (α) | 16 | | LoRA dropout | 0.0 | | Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | | Epochs | 4 max (epoch-3 checkpoint selected, lowest validation loss) | | Learning rate | 2e-4 | | LR scheduler | Cosine (warmup ratio 0.03) | | Optimizer | AdamW 8-bit (`adamw_8bit`) | | Batch size (per device × grad. accum.) | 2 × 4 (effective batch size 8) | | Max sequence length | 2048 | | Training seed (this adapter) | 42 (four additional seeds 43–46 used for replication only; see below) | #### Compute | | | |---|---| | Hardware (training) | NVIDIA T4 (16GB) on Google Colab | | Training time | ~902.3 seconds (~15.0 minutes) | | Inference latency (measured, NVIDIA L4) | Base: 350.8ms/item avg; fine-tuned: 416.2ms/item avg (+16–19% overhead), +0.18GB peak VRAM | | Software | Unsloth, TRL, PEFT 0.19.1, transformers, bitsandbytes, accelerate | --- ## Evaluation ### Testing data, factors, and metrics The fine-tuned model was evaluated **without any further training** on the held-out ToMBench test split (in-domain) and on four external benchmarks spanning structurally different kinds of social reasoning: | Benchmark | What it tests | |---|---| | **ToMBench** (held-out) | In-domain: false-belief and knowledge-state items across ATOMS abilities | | **ToMi** | Classic Sally–Anne style false-belief tasks | | **OpenToM** | Narrative-based ToM with longer, character-driven stories | | **SocialIQa** | Broad social-commonsense reasoning | | **Hi-ToM** | Higher-order recursive belief reasoning (up to fourth order) | **Metrics.** Accuracy for all benchmarks; macro-F1 additionally (descriptive) for OpenToM due to potential class imbalance. **Statistical validation.** Every base-vs-fine-tuned comparison was tested with an exact **paired McNemar test**, with **Holm–Bonferroni correction** across the five benchmarks and **story-cluster bootstrap 95% confidence intervals** on the accuracy difference. All results below are from a content-hash-based, item-identity-verified pipeline (zero item-identifier or prompt-hash mismatches across every result file). Per-item predictions and the full statistics tables are published in `results/` and `results/stats/` in the GitHub repository. ### Results (primary run, seed 42) | Benchmark | Metric | Base | Fine-tuned | Δ | Holm-corrected p | Robust across 5 seeds? | |---|---|---|---|---|---|---| | ToMBench (in-domain, held-out) | Accuracy | 61.52 | 75.18 | **+13.66 pp** | < 0.0001 | ✅ yes (+14.32 ± 0.75pp) | | ToMi | Accuracy | 76.30 | 83.00 | **+6.70 pp** | < 0.001 | ✅ yes (+9.06 ± 1.94pp) | | OpenToM | Accuracy (primary) | 63.40 | 59.50 | −3.90 pp | 0.0033 | ⚠️ no — mean crosses zero (−0.56 ± 1.96pp) | | OpenToM | macro-F1 (descriptive) | 49.06 | 47.91 | −1.15 pp | — | — | | SocialIQa | Accuracy | 67.70 | 61.70 | **−6.00 pp** | < 0.001 | ✅ yes, most robust effect in study (−5.28 ± 0.55pp) | | Hi-ToM | Accuracy | 62.40 | 62.30 | −0.10 pp | 1.0000 (n.s.) | No net effect (mean +0.94 ± 0.84pp); but see position-bias note below | **Additional robustness checks:** - **Story-disjoint split:** re-splitting ToMBench so no test story appears in training reduces the in-domain gain to **+10.39pp** (base 60.57% → fine-tuned 70.97%), still significant (McNemar exact p = 7.2×10⁻¹⁰). - **Hi-ToM gold-answer-position analysis:** grouping the 1,000 Hi-ToM items by the position (A–E) of the correct answer reveals a large internal redistribution invisible in the flat aggregate above — items with gold answer **E** dropped 16.67pp, while B and D *improved*. A 5,000-permutation label-shuffling test confirms this gap is not chance (p = 0.0002). Excluding the gold-E stratum reveals a significant +3.84pp residual gain, but a complementary position-balanced accuracy check across all letters is essentially null (+0.05pp); because the two checks disagree, whether this reflects a genuine change in higher-order belief-tracking ability remains an open question rather than an established finding. - **Training-free few-shot baselines:** a corrected rationale-style 3-shot prompt (no weight updates) matched or exceeded the fine-tuned model's ToMi transfer (+11.87pp vs. +9.06pp multi-seed mean), showing that some of the fine-tuning gain on structurally similar tasks is also reachable without training. #### Summary Fine-tuning on ToMBench produced a large in-domain improvement, part of which (≈24%) reflects train/test story overlap rather than pure novel-story generalization. The effect transferred **robustly positively** to a structurally similar false-belief benchmark (ToMi), and **robustly negatively** to broad social-commonsense reasoning (SocialIQa) — the single most seed-stable finding in this study. Its effect on narrative ToM (OpenToM) was significant in the primary run but is seed-sensitive, should not be treated as confirmed, and is substantially, though not completely, attributable to a response-position bias interacting with this benchmark's severe class imbalance. Its aggregate effect on higher-order recursive belief reasoning (Hi-ToM) was null once evaluated under a verified-stable pipeline and replicated across five seeds, though a position-bias follow-up check found a significant residual gain under one method that did not replicate under a complementary method, leaving this an open question for future confirmatory work rather than an established result. We interpret this as evidence that Theory of Mind is not a single learnable capability in small language models, but a set of structurally distinct sub-skills that **transfer selectively — and with differing robustness** — training on one can leave, robustly degrade, or only apparently affect the others depending on task structure and measurement artifacts. --- ## Citation This manuscript is in preparation for submission to *Applied Sciences* (MDPI), September 2026. **BibTeX** ```bibtex @article{hong2026tomqlora, title = {QLoRA Fine-Tuning for Theory-of-Mind Reasoning in Small Language Models: A ToMBench-Based Cross-Benchmark Generalization Study}, author = {Hong, Ji-Hyeong and Yoo, Sang-Hyun}, year = {2026}, note = {In preparation for submission to Applied Sciences (MDPI), September 2026} } ``` **APA** Hong, J.-H., & Yoo, S.-H. (2026). *QLoRA fine-tuning for Theory-of-Mind reasoning in small language models: A ToMBench-based cross-benchmark generalization study.* Manuscript in preparation for submission to *Applied Sciences* (MDPI). --- ## Model card authors Ji-Hyeong Hong ## Model card contact saccai1131@gmail.com · https://github.com/NextMina1131 ### Framework versions - PEFT 0.19.1