Upload folder using huggingface_hub

README.md

@@ -0,0 +1,128 @@
+---
+license: apache-2.0
+base_model: HuggingFaceTB/SmolLM2-135M
+tags:
+- peft
+- lora
+- gsm8k
+- math
+- reasoning
+- curriculum-learning
+datasets:
+- gsm8k
+metrics:
+- accuracy
+library_name: peft
+---
+
+# SmolLM2-135M LoRA Adapter for GSM8K
+
+This is a LoRA adapter for [HuggingFaceTB/SmolLM2-135M](https://huggingface.co/HuggingFaceTB/SmolLM2-135M) fine-tuned on the [GSM8K dataset](https://huggingface.co/datasets/gsm8k) for mathematical reasoning.
+
+## Model Description
+
+- **Base Model:** HuggingFaceTB/SmolLM2-135M
+- **Training Method:** Curriculum learning - Complexity Score
+- **Dataset:** GSM8K (Grade School Math 8K)
+- **Task:** Mathematical word problem solving
+- **Exact Match Accuracy:** 2.93%
+
+## Training Details
+
+LoRA adapter for SmolLM2 trained with curriculum learning (complexity score method)
+
+### Training Configuration
+- **Method:** LoRA (Low-Rank Adaptation)
+- **Rank:** 16
+- **Alpha:** 32
+- **Target Modules:** q_proj, k_proj, v_proj, o_proj
+- **Dropout:** 0.1
+- **Epochs:** 3
+- **Batch Size:** 4 (with gradient accumulation of 4)
+- **Learning Rate:** 3e-4
+
+### Curriculum Learning
+
+This model was trained using curriculum learning, where the model is exposed to progressively harder problems:
+1. **Easy Stage:** Simple problems with fewer steps
+2. **Normal Stage:** Moderate complexity problems
+3. **Difficult Stage:** Complex multi-step problems
+
+The curriculum was determined based on problem complexity (number of solution steps × operation complexity).
+
+
+## Usage
+
+### Loading the Adapter
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+# Load base model
+base_model = AutoModelForCausalLM.from_pretrained(
+    "HuggingFaceTB/SmolLM2-135M",
+    device_map="auto",
+    torch_dtype="auto"
+)
+
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-135M")
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(base_model, "CrystalRaindropsFall/smollm2-gsm8k-curriculum-complexity")
+
+# Inference
+prompt = "Question: Janet's ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?\nAnswer:"
+
+inputs = tokenizer(prompt, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=256)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+### Using with Pipeline
+
+```python
+from transformers import pipeline
+from peft import PeftModel, AutoPeftModelForCausalLM
+
+# Load model with adapter
+model = AutoPeftModelForCausalLM.from_pretrained(
+    "YOUR_USERNAME/REPO_NAME",
+    device_map="auto"
+)
+
+# Create pipeline
+pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
+
+# Generate
+result = pipe("Question: A robe takes 2 bolts of blue fiber and half that much white fiber. How many bolts in total does it take?\nAnswer:")
+print(result[0]['generated_text'])
+```
+
+## Performance
+
+Evaluated on GSM8K test set (512 samples):
+
+| Metric | Score |
+|--------|-------|
+| Exact Match | 2.93% |
+| Format Correct | 100% |
+
+## Limitations
+
+- Trained on grade school level math problems
+- May struggle with problems requiring external knowledge
+- Performance depends on problem complexity and wording
+- Best used with base model's standard generation settings
+
+
+## Acknowledgments
+
+- Base model: HuggingFaceTB/SmolLM2-135M
+- Dataset: GSM8K by Cobbe et al.
+- Training framework: HuggingFace PEFT
+
+## License
+
+Apache 2.0 (following base model license)

adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "experiment_results/curriculum_complexity_score_smolLM2_gsm8k_20251112_222922/stage_2_normal/merged_model",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "k_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdc1fabc1021f6abd0e9577d83db1f8d72b8a226e052e8adda868eb9f8c9982e
+size 7404368

special_tokens_map.json

@@ -0,0 +1,49 @@
+{
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "<|im_start|>",
+    "<|im_end|>",
+    "<repo_name>",
+    "<reponame>",
+    "<file_sep>",
+    "<filename>",
+    "<gh_stars>",
+    "<issue_start>",
+    "<issue_comment>",
+    "<issue_closed>",
+    "<jupyter_start>",
+    "<jupyter_text>",
+    "<jupyter_code>",
+    "<jupyter_output>",
+    "<jupyter_script>",
+    "<empty_output>"
+  ],
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,169 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<repo_name>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<reponame>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<file_sep>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<filename>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<gh_stars>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<issue_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<issue_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<issue_closed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<jupyter_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<jupyter_text>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<jupyter_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<jupyter_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<jupyter_script>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<empty_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "<|im_start|>",
+    "<|im_end|>",
+    "<repo_name>",
+    "<reponame>",
+    "<file_sep>",
+    "<filename>",
+    "<gh_stars>",
+    "<issue_start>",
+    "<issue_comment>",
+    "<issue_closed>",
+    "<jupyter_start>",
+    "<jupyter_text>",
+    "<jupyter_code>",
+    "<jupyter_output>",
+    "<jupyter_script>",
+    "<empty_output>"
+  ],
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "extra_special_tokens": {},
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>",
+  "vocab_size": 49152
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebe69044be4b205a2fb5fbf1482e23a773dc62ff6a19e21c93d1424c639e56ce
+size 5944