---
license: apache-2.0
base_model: Qwen/Qwen3-8B
tags: [gin-rummy, grpo, reinforcement-learning, lora, game-playing, reward-shaping]
datasets: [self-play]
library_name: peft
---

# Gin Rummy GRPO LoRA v2 — Run 9 (Qwen3-8B)

LoRA trained via GRPO on Gin Rummy with shaped rewards and WeakKnocker opponent. Final run of a 9-run iteration.

| Field | Value |
|-------|-------|
| **Base model** | [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) |
| **Method** | GRPO (TRL GRPOTrainer) |
| **LoRA** | r=16, alpha=32, q/k/v/o_proj |
| **Steps** | 500 (~2000 games) |
| **Opponent** | WeakKnocker (random draws, smart discards, always knocks) |
| **Win rate** | 0.5% |
| **Infra** | Together AI 8x H100 80GB, ~3.5 hours |
| **Git commit** | `2a987af docs: add Phase 3 (V2 migration) to experiment log` |
| **W&B** | [good-start-labs/ginrummy-grpo](https://wandb.ai/good-start-labs/ginrummy-grpo) |
| **Dataset** | Self-play rollouts vs WeakKnocker bot |

## Config
- `enable_thinking=True`, simplified rules-only prompt, single vLLM call/turn
- `num_generations=8`, `lr=1e-5`, `beta=0.1`, `max_completion_length=2048`
- Shaped reward: 40% terminal + 40% deadwood progress + 20% game length

## Key Finding
GRPO alone cannot teach gin rummy from scratch. The game is too complex for random exploration. SFT on expert data should precede RL. See [full experiment log](https://github.com/GoodStartLabs/GinRummy_RLenv/blob/main/grpo_training/experiment_log.md).

## Usage
```python
from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "GoodStartLabs/ginrummy-qwen3-8b-grpo-lora-v2")
```