Replace placeholder model card with proper documentation

README.md

@@ -19,7 +19,7 @@ A LoRA fine-tuned adapter for [LiquidAI/LFM2.5-1.2B-Instruct](https://huggingfac
 
 ## Model Description
 
-This adapter teaches LFM2.5-1.2B-Instruct to respond in the structured Thought + Code format required by smolagents CodeAgent:
+This adapter teaches LFM2.5-1.2B to respond in the structured Thought + Code format required by smolagents CodeAgent:
 
 ````
 Thought: I need to calculate this.
@@ -31,66 +31,53 @@ final_answer(result)
 
 ### Key Features
 
-- **Base Model**: LiquidAI/LFM2.5-1.2B-Instruct (1.2B parameter)
-- **Format Compliance**: N/A with minimal prompt
-- **Answer Accuracy**: N/A on evaluation tasks
-- **Adapter Size**: ~47MB (LoRA rank=16, alpha=32)
+- **Base Model**: LiquidAI/LFM2.5-1.2B-Instruct (1.17B parameters)
+- **Token Accuracy**: 73.8% on training data
+- **Single-Turn Rate**: 55.6% of tasks solved in one turn
+- **Adapter Size**: ~42MB (LoRA rank=16, alpha=32)
 
 ## Training Details
 
 ### Training Data
 
-- **81 successful CodeAgent trajectories** generated using Claude 3 Haiku as the teacher model
-- Tasks include mathematical reasoning, string manipulation, and general problem-solving
+- **81 successful CodeAgent trajectories** generated using Claude 3 Haiku
+- Tasks include mathematical reasoning, string manipulation, file operations, and general problem-solving
 - Each trajectory demonstrates the Thought → Code → Observation → final_answer pattern
+- Average trajectory length: 2957 tokens
 
 ### Training Configuration
 
-| Parameter            | Value              |
-| -------------------- | ------------------ |
-| LoRA Rank            | 16        |
-| LoRA Alpha           | 32       |
-| Target Modules       | w1, w2, w3, q_proj, k_proj, v_proj, out_proj, in_proj   |
-| Trainable Parameters | ~25M |
-| Training Steps       | 3 epochs   |
-| Learning Rate        | 1e-4    |
-| Batch Size           | 2 (grad accum 4)       |
-| Max Sequence Length  | 8192   |
-| Hardware             | GPU         |
-| Training Time        | see ablation report    |
+| Parameter            | Value                                                 |
+| -------------------- | ----------------------------------------------------- |
+| LoRA Rank            | 16                                                    |
+| LoRA Alpha           | 32                                                    |
+| Target Modules       | w1, w2, w3, q_proj, k_proj, v_proj, out_proj, in_proj |
+| Trainable Parameters | 11.1M (0.94% of base model)                           |
+| Epochs               | 3                                                     |
+| Learning Rate        | 1e-4                                                  |
+| Batch Size           | 2 (effective 8 with gradient accumulation)            |
+| Max Sequence Length  | 8192                                                  |
+| Hardware             | NVIDIA RTX 3090 (24GB)                                |
+| Training Time        | 461s                                                  |
 
 ### Training Framework
 
 - [TRL](https://github.com/huggingface/trl) SFTTrainer
 - [PEFT](https://github.com/huggingface/peft) for LoRA
 
-## Evaluation Results
+## Ablation Study Results
 
-### Prompt Mode Comparison
+This model is part of a teacher ablation study comparing different Claude models and prompting strategies:
 
-| Prompt Mode | Format Compliance           | Answer Accuracy           |
-| ----------- | --------------------------- | ------------------------- |
-| **Minimal** | N/A         | N/A         |
-| Default     | N/A | N/A |
-| None        | N/A    | N/A    |
+| Teacher Config  | Token Accuracy | Training Time | Avg Trajectory Tokens |
+| --------------- | -------------- | ------------- | --------------------- |
+| haiku-default   | 73.8%          | 461s          | 2,957                 |
+| sonnet-default  | 90.6%          | 260s          | 3,054                 |
+| opus4-default   | 90.2%          | 260s          | 2,971                 |
+| sonnet4-terse   | 94.9%          | 93s           | 632                   |
+| **opus4-terse** | **95.0%**      | **76s**       | **613**               |
 
-The model performs best with the **minimal prompt** (~95 tokens), demonstrating successful prompt distillation.
-
-### Minimal Prompt Template
-
-````text
-You are a CodeAgent that solves tasks by writing and executing Python code.
-
-Always respond with Thought + Python code block. Example:
-
-Thought: I need to calculate this.
-```python
-result = 2 + 2
-final_answer(result)
-```
-
-Call final_answer(result) when done. Now Begin!
-````
+**Key Finding**: Terse, focused trajectories from capable teachers transfer significantly better to small models.
 
 ## Usage
 
@@ -119,8 +106,8 @@ inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
 outputs = model.generate(
     **inputs,
     max_new_tokens=512,
-    temperature=0.3,
-    min_p=0.15,
+    temperature=0.1,
+    top_p=0.1,
 )
 print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
@@ -146,23 +133,23 @@ print(result)
 
 ## Intended Use
 
-- Code-assisted problem solving
+- Code-assisted problem solving with small, efficient models
 - Mathematical reasoning tasks
 - Automated code generation following structured formats
-- Research into prompt distillation and small model fine-tuning
+- Research into prompt distillation and teacher model selection
 
 ## Limitations
 
-- **Requires specific prompt format**: Works best with minimal prompt template
-- **Limited reasoning depth**: 1.2B parameter model has constrained reasoning capabilities compared to larger models
+- **1.2B model constraints**: Limited reasoning depth compared to larger models
 - **English only**: Trained on English-language tasks
+- **CodeAgent format specific**: Optimized for smolagents Thought/Code/final_answer pattern
 
 ## Citation
 
 If you use this model, please cite:
 
 ```bibtex
-@misc{lfm25_1.2b_codeagent_haiku_default,
+@misc{lfm25-codeagent-haiku_default-2025,
   author = {krzysztofwos},
   title = {LFM25-1.2B-CodeAgent-haiku-default},
   year = {2025},
@@ -173,6 +160,6 @@ If you use this model, please cite:
 
 ## Acknowledgments
 
-- [LiquidAI](https://www.liquid.ai/) for the LFM2.5-1.2B-Instruct base model
+- [LiquidAI](https://www.liquid.ai/) for the LFM2.5 base model
+- [Anthropic](https://www.anthropic.com/) for Claude 3 Haiku (teacher model)
 - [Hugging Face](https://huggingface.co/) for smolagents, TRL, and PEFT
-- Training performed as part of CodeAgent prompt distillation research