Add README

README.md

@@ -0,0 +1,38 @@
+---
+base_model: Qwen/Qwen2.5-Coder-7B-Instruct
+tags:
+  - reinforcement-learning
+  - ppo
+  - lora
+  - code-generation
+  - clarification
+license: apache-2.0
+---
+
+# rl-clarify-orig-prompt-d1-1
+
+LoRA fine-tune of [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)
+trained with PPO-Lagrangian constrained RL on [HumanEvalComm](https://huggingface.co/datasets/jie-jw-wu/HumanEvalComm).
+
+PPO-Lagrangian RL fine-tune of Qwen2.5-Coder-7B-Instruct (LoRA rank 16) on HumanEvalComm with question budget d1=1. All iter_* checkpoints included. Eval checkpoint: iter_0029.
+
+## Training setup
+- **Algorithm:** PPO with Lagrangian constraint on avg questions per episode
+- **LoRA rank:** 16, alpha 32
+- **Question budget (d1):** 1.0
+- **Eval checkpoint:** `iter_0029`
+- **Eval set:** 469 problems (full HumanEvalComm eval split, greedy decoding)
+
+## Checkpoints
+Each `iter_XXXX/` folder contains LoRA adapter weights and a `log.json`
+with per-iteration training metrics (avg_reward, avg_questions, lambda1).
+
+## Usage
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")
+model = PeftModel.from_pretrained(base, "acv1229/rl-clarify-orig-prompt-d1-1", subfolder="iter_0029")
+tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")
+```