Reinforcement Learning
stable-baselines3
PyTorch
English
Korean
deep-reinforcement-learning
ppo
continuous-control
mujoco
pusher
pusher-v5
robotics
robot
robot-arm
robotic-manipulation
7-dof
gymnasium
Eval Results (legacy)
Instructions to use hwihwalab/pusher-v5-ppo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- stable-baselines3
How to use hwihwalab/pusher-v5-ppo with stable-baselines3:
from huggingface_sb3 import load_from_hub checkpoint = load_from_hub( repo_id="hwihwalab/pusher-v5-ppo", filename="{MODEL FILENAME}.zip", ) - Notebooks
- Google Colab
- Kaggle
feat: deploy Pusher-v5 PPO model, telemetry cockpit, and video gallery
Browse files- README.md +58 -2
- README_KR.md +58 -3
README.md
CHANGED
|
@@ -121,13 +121,69 @@ python deploy_to_hf.py
|
|
| 121 |
### 4. Standalone CLI Training & Evaluation
|
| 122 |
```bash
|
| 123 |
# Train PPO agent
|
| 124 |
-
python train.py --timesteps
|
| 125 |
|
| 126 |
# Evaluate trained model
|
| 127 |
-
python evaluate.py --model_path ./results/ppo_pusher.zip --episodes
|
| 128 |
```
|
| 129 |
|
| 130 |
---
|
| 131 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 132 |
## ๐ License
|
| 133 |
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
|
|
|
|
| 121 |
### 4. Standalone CLI Training & Evaluation
|
| 122 |
```bash
|
| 123 |
# Train PPO agent
|
| 124 |
+
python train.py --timesteps 300000 --eval_freq 30000
|
| 125 |
|
| 126 |
# Evaluate trained model
|
| 127 |
+
python evaluate.py --model_path ./results/ppo_pusher.zip --episodes 5
|
| 128 |
```
|
| 129 |
|
| 130 |
---
|
| 131 |
|
| 132 |
+
## ๐ Quick Python Evaluation Snippet
|
| 133 |
+
|
| 134 |
+
You can load and evaluate this pre-trained agent in 5 lines of Python using Stable-Baselines3:
|
| 135 |
+
|
| 136 |
+
```python
|
| 137 |
+
import gymnasium as gym
|
| 138 |
+
from stable_baselines3 import PPO
|
| 139 |
+
|
| 140 |
+
# 1. Initialize Pusher-v5 environment & load model
|
| 141 |
+
env = gym.make("Pusher-v5", render_mode="human")
|
| 142 |
+
model = PPO.load("results/ppo_pusher.zip")
|
| 143 |
+
|
| 144 |
+
# 2. Run deterministic pushing evaluation
|
| 145 |
+
obs, _ = env.reset()
|
| 146 |
+
done = False
|
| 147 |
+
while not done:
|
| 148 |
+
action, _ = model.predict(obs, deterministic=True)
|
| 149 |
+
obs, reward, terminated, truncated, _ = env.step(action)
|
| 150 |
+
done = terminated or truncated
|
| 151 |
+
|
| 152 |
+
env.close()
|
| 153 |
+
```
|
| 154 |
+
|
| 155 |
+
---
|
| 156 |
+
|
| 157 |
+
## โจ๏ธ Keyboard Shortcuts Reference
|
| 158 |
+
|
| 159 |
+
| Key | Action | Description |
|
| 160 |
+
| :---: | :--- | :--- |
|
| 161 |
+
| **`Space`** | **Start / Pause** | Toggle 30 FPS MuJoCo physical simulation stream |
|
| 162 |
+
| **`R`** | **Reset Environment** | Reset robotic arm, cylinder object, and target goal to new random positions |
|
| 163 |
+
| **`S`** | **Step Once** | Advance physics engine forward by 1 discrete timestep (0.05s) |
|
| 164 |
+
| **`H`** | **Toggle HUD** | Show or hide on-canvas telemetry data overlay |
|
| 165 |
+
|
| 166 |
+
---
|
| 167 |
+
|
| 168 |
+
## ๐ก๏ธ AI Governance & Documentation Architecture
|
| 169 |
+
|
| 170 |
+
This repository is governed by rigorous engineering protocols to ensure simulation fidelity and prevent vibe-coding drift:
|
| 171 |
+
|
| 172 |
+
- **[`.cursorrules`](.cursorrules)**: AI Vibe-Coding Defense Master Protocol
|
| 173 |
+
- **[`DOCS_AI_CODING_PROTOCOL.md`](DOCS_AI_CODING_PROTOCOL.md)**: Coding Standards & Master Documentation Map
|
| 174 |
+
- **[`DOCS_SYSTEM_ARCHITECTURE.md`](DOCS_SYSTEM_ARCHITECTURE.md)**: Full-Stack System & WebSocket Architecture Spec
|
| 175 |
+
- **[`DOCS_DATA_SCHEMA.md`](DOCS_DATA_SCHEMA.md)**: WebSocket Telemetry Protocol & REST Data Schema
|
| 176 |
+
- **[`DOCS_MODEL_EVALUATION_AND_HF_DEPLOY.md`](DOCS_MODEL_EVALUATION_AND_HF_DEPLOY.md)**: Benchmark Evaluation & Hugging Face Hub Pipeline
|
| 177 |
+
|
| 178 |
+
---
|
| 179 |
+
|
| 180 |
+
## ๐ Open Source Hubs & Project Links
|
| 181 |
+
|
| 182 |
+
- ๐ **GitHub Repository**: [https://github.com/Hwihwa-Lab/pusher-v5-ppo](https://github.com/Hwihwa-Lab/pusher-v5-ppo)
|
| 183 |
+
- ๐ค **Hugging Face Model Hub**: [https://huggingface.co/hwihwalab/pusher-v5-ppo](https://huggingface.co/hwihwalab/pusher-v5-ppo)
|
| 184 |
+
|
| 185 |
+
---
|
| 186 |
+
|
| 187 |
## ๐ License
|
| 188 |
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
| 189 |
+
|
README_KR.md
CHANGED
|
@@ -100,7 +100,7 @@ python app.py
|
|
| 100 |
```
|
| 101 |
๋ธ๋ผ์ฐ์ ์์ **`http://localhost:8000`** ์ ์.
|
| 102 |
|
| 103 |
-
### 3. ํ๊น
ํ์ด์ค
|
| 104 |
```bash
|
| 105 |
python deploy_to_hf.py
|
| 106 |
```
|
|
@@ -108,13 +108,68 @@ python deploy_to_hf.py
|
|
| 108 |
### 4. CLI ๊ธฐ๋ฐ ๋
๋ฆฝ ํ์ต ๋ฐ ๋ชจ๋ธ ํ๊ฐ
|
| 109 |
```bash
|
| 110 |
# PPO ์์ด์ ํธ ํ์ต ์คํ
|
| 111 |
-
python train.py --timesteps
|
| 112 |
|
| 113 |
# ํ์ต ์๋ฃ๋ ๋ชจ๋ธ ๋
๋ฆฝ ํ๊ฐ ๋ฐ ๋น๋์ค ์ถ์ถ
|
| 114 |
-
python evaluate.py --model_path ./results/ppo_pusher.zip --episodes
|
| 115 |
```
|
| 116 |
|
| 117 |
---
|
| 118 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 119 |
## ๐ ๋ผ์ด์ ์ค
|
| 120 |
๋ณธ ํ๋ก์ ํธ๋ MIT License๋ฅผ ๋ฐ๋ฆ
๋๋ค.
|
|
|
|
| 100 |
```
|
| 101 |
๋ธ๋ผ์ฐ์ ์์ **`http://localhost:8000`** ์ ์.
|
| 102 |
|
| 103 |
+
### 3. ํ๊น
ํ์ด์ค ์ํด๋ฆญ ์๋ ๋ฐฐํฌ
|
| 104 |
```bash
|
| 105 |
python deploy_to_hf.py
|
| 106 |
```
|
|
|
|
| 108 |
### 4. CLI ๊ธฐ๋ฐ ๋
๋ฆฝ ํ์ต ๋ฐ ๋ชจ๋ธ ํ๊ฐ
|
| 109 |
```bash
|
| 110 |
# PPO ์์ด์ ํธ ํ์ต ์คํ
|
| 111 |
+
python train.py --timesteps 300000 --eval_freq 30000
|
| 112 |
|
| 113 |
# ํ์ต ์๋ฃ๋ ๋ชจ๋ธ ๋
๋ฆฝ ํ๊ฐ ๋ฐ ๋น๋์ค ์ถ์ถ
|
| 114 |
+
python evaluate.py --model_path ./results/ppo_pusher.zip --episodes 5
|
| 115 |
```
|
| 116 |
|
| 117 |
---
|
| 118 |
|
| 119 |
+
## ๐ 5์ค ํ์ด์ฌ ๋น ๋ฅธ ํ๊ฐ ์ค๋ํซ (Quick Evaluation)
|
| 120 |
+
|
| 121 |
+
๋ณธ ๋ฆฌํฌ์งํ ๋ฆฌ์ ํ์ต ์๋ฃ ๊ฐ์ค์น๋ฅผ ๋ถ๋ฌ์ 5์ค์ ํ์ด์ฌ ์ฝ๋๋ก ์ฆ์ ์๋ฎฌ๋ ์ด์
์ ์คํํ ์ ์์ต๋๋ค:
|
| 122 |
+
|
| 123 |
+
```python
|
| 124 |
+
import gymnasium as gym
|
| 125 |
+
from stable_baselines3 import PPO
|
| 126 |
+
|
| 127 |
+
# 1. Pusher-v5 ํ๊ฒฝ ์ด๊ธฐํ ๋ฐ ์์ฑ ๊ฐ์ค์น ๋ก๋
|
| 128 |
+
env = gym.make("Pusher-v5", render_mode="human")
|
| 129 |
+
model = PPO.load("results/ppo_pusher.zip")
|
| 130 |
+
|
| 131 |
+
# 2. ๊ฒฐ์ ๋ก ์ ํธ์ฑ ์ ์ด ๋กค์์ ์คํ
|
| 132 |
+
obs, _ = env.reset()
|
| 133 |
+
done = False
|
| 134 |
+
while not done:
|
| 135 |
+
action, _ = model.predict(obs, deterministic=True)
|
| 136 |
+
obs, reward, terminated, truncated, _ = env.step(action)
|
| 137 |
+
done = terminated or truncated
|
| 138 |
+
|
| 139 |
+
env.close()
|
| 140 |
+
```
|
| 141 |
+
|
| 142 |
+
---
|
| 143 |
+
|
| 144 |
+
## โจ๏ธ ํค๋ณด๋ ๋จ์ถํค ์๋ด (Keyboard Shortcuts)
|
| 145 |
+
|
| 146 |
+
| ๋จ์ถํค | ์กฐ์ ๊ธฐ๋ฅ | ์ค๋ช
|
|
| 147 |
+
| :---: | :--- | :--- |
|
| 148 |
+
| **`Space`** | **์์ / ์ผ์์ ์ง** | ์ค์๊ฐ 30 FPS MuJoCo ๋ฌผ๋ฆฌ ์๋ฎฌ๋ ์ด์
ํ ๊ธ |
|
| 149 |
+
| **`R`** | **ํ๊ฒฝ ์ด๊ธฐํ (Reset)** | ๋ก๋ด ํ, ์ํต ๋ฌผ์ฒด, ๋ชฉํ ๊ณจ๋๋ฅผ ์๋ก์ด ๋๋ค ์์น๋ก ์ฌ๋ฐฐ์น |
|
| 150 |
+
| **`S`** | **1์คํ
์ ์ง (Step Once)** | ๋ฌผ๋ฆฌ ์์ง์ 1๋จ์ ํ์์คํ
(0.05์ด) ์ ์ง |
|
| 151 |
+
| **`H`** | **HUD ์จ์คํ ํ ๊ธ** | ์บ๋ฒ์ค ํ๋ฉด ์ ํ
๋ ๋ฉํธ๋ฆฌ ์ค๋ฒ๋ ์ด ํ์/์จ๊น |
|
| 152 |
+
|
| 153 |
+
---
|
| 154 |
+
|
| 155 |
+
## ๐ก๏ธ AI ์์ง๋์ด๋ง ๊ฑฐ๋ฒ๋์ค ๋ฐ ๋ฌธ์ ์ฒด๊ณ
|
| 156 |
+
|
| 157 |
+
๋ณธ ์์คํ
์ ๊ฐํํ์ต ์๋ฎฌ๋ ์ด์
์ ๋ฌผ๋ฆฌ์ ๋ฌด๊ฒฐ์ฑ์ ๋ณด์กดํ๊ณ ๋ฐ์ด๋ธ-์ฝ๋ฉ ๋๋ฆฌํํธ๋ฅผ ๋ฐฉ์งํ๊ธฐ ์ํด ์ ๋ฐํ ์์ง๋์ด๋ง ๋ฌธ์ ํ๋กํ ์ฝ์ ์ค์ํฉ๋๋ค:
|
| 158 |
+
|
| 159 |
+
- **[`.cursorrules`](.cursorrules)**: AI ์ฝ๋ฉ ๋ฐฉ์ด ๋ฐ ๊ท์น ๋ง์คํฐ ํ๋ฒ
|
| 160 |
+
- **[`DOCS_AI_CODING_PROTOCOL.md`](DOCS_AI_CODING_PROTOCOL.md)**: ์ฝ๋ฉ ํ์ค ๋ฐ ์ ์ฒด ๋ฌธ์ ๋งต
|
| 161 |
+
- **[`DOCS_SYSTEM_ARCHITECTURE.md`](DOCS_SYSTEM_ARCHITECTURE.md)**: ํ์คํ ์์คํ
๋ฐ WebSocket ์ํคํ
์ฒ ๋ช
์ธ์
|
| 162 |
+
- **[`DOCS_DATA_SCHEMA.md`](DOCS_DATA_SCHEMA.md)**: ํ
๋ ๋ฉํธ๋ฆฌ ํจํท ํ๋กํ ์ฝ ๋ฐ REST ๋ฐ์ดํฐ ์คํค๋ง
|
| 163 |
+
- **[`DOCS_MODEL_EVALUATION_AND_HF_DEPLOY.md`](DOCS_MODEL_EVALUATION_AND_HF_DEPLOY.md)**: ๋ฒค์น๋งํฌ ํ๊ฐ ๋ฐ ํ๊น
ํ์ด์ค ๋ฐฐํฌ ๊ท๊ฒฉ์
|
| 164 |
+
|
| 165 |
+
---
|
| 166 |
+
|
| 167 |
+
## ๐ ์คํ์์ค ๊ณต์ ๋งํฌ (Open Source Hubs)
|
| 168 |
+
|
| 169 |
+
- ๐ **GitHub ์ ์ฅ์**: [https://github.com/Hwihwa-Lab/pusher-v5-ppo](https://github.com/Hwihwa-Lab/pusher-v5-ppo)
|
| 170 |
+
- ๐ค **Hugging Face ๋ชจ๋ธ ํ๋ธ**: [https://huggingface.co/hwihwalab/pusher-v5-ppo](https://huggingface.co/hwihwalab/pusher-v5-ppo)
|
| 171 |
+
|
| 172 |
+
---
|
| 173 |
+
|
| 174 |
## ๐ ๋ผ์ด์ ์ค
|
| 175 |
๋ณธ ํ๋ก์ ํธ๋ MIT License๋ฅผ ๋ฐ๋ฆ
๋๋ค.
|