hwihwalab commited on
Commit
d8929ca
ยท
verified ยท
1 Parent(s): df0c994

feat: deploy Pusher-v5 PPO model, telemetry cockpit, and video gallery

Browse files
Files changed (2) hide show
  1. README.md +58 -2
  2. README_KR.md +58 -3
README.md CHANGED
@@ -121,13 +121,69 @@ python deploy_to_hf.py
121
  ### 4. Standalone CLI Training & Evaluation
122
  ```bash
123
  # Train PPO agent
124
- python train.py --timesteps 50000 --eval_freq 10000
125
 
126
  # Evaluate trained model
127
- python evaluate.py --model_path ./results/ppo_pusher.zip --episodes 3
128
  ```
129
 
130
  ---
131
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
132
  ## ๐Ÿ“„ License
133
  This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
 
 
121
  ### 4. Standalone CLI Training & Evaluation
122
  ```bash
123
  # Train PPO agent
124
+ python train.py --timesteps 300000 --eval_freq 30000
125
 
126
  # Evaluate trained model
127
+ python evaluate.py --model_path ./results/ppo_pusher.zip --episodes 5
128
  ```
129
 
130
  ---
131
 
132
+ ## ๐Ÿ Quick Python Evaluation Snippet
133
+
134
+ You can load and evaluate this pre-trained agent in 5 lines of Python using Stable-Baselines3:
135
+
136
+ ```python
137
+ import gymnasium as gym
138
+ from stable_baselines3 import PPO
139
+
140
+ # 1. Initialize Pusher-v5 environment & load model
141
+ env = gym.make("Pusher-v5", render_mode="human")
142
+ model = PPO.load("results/ppo_pusher.zip")
143
+
144
+ # 2. Run deterministic pushing evaluation
145
+ obs, _ = env.reset()
146
+ done = False
147
+ while not done:
148
+ action, _ = model.predict(obs, deterministic=True)
149
+ obs, reward, terminated, truncated, _ = env.step(action)
150
+ done = terminated or truncated
151
+
152
+ env.close()
153
+ ```
154
+
155
+ ---
156
+
157
+ ## โŒจ๏ธ Keyboard Shortcuts Reference
158
+
159
+ | Key | Action | Description |
160
+ | :---: | :--- | :--- |
161
+ | **`Space`** | **Start / Pause** | Toggle 30 FPS MuJoCo physical simulation stream |
162
+ | **`R`** | **Reset Environment** | Reset robotic arm, cylinder object, and target goal to new random positions |
163
+ | **`S`** | **Step Once** | Advance physics engine forward by 1 discrete timestep (0.05s) |
164
+ | **`H`** | **Toggle HUD** | Show or hide on-canvas telemetry data overlay |
165
+
166
+ ---
167
+
168
+ ## ๐Ÿ›ก๏ธ AI Governance & Documentation Architecture
169
+
170
+ This repository is governed by rigorous engineering protocols to ensure simulation fidelity and prevent vibe-coding drift:
171
+
172
+ - **[`.cursorrules`](.cursorrules)**: AI Vibe-Coding Defense Master Protocol
173
+ - **[`DOCS_AI_CODING_PROTOCOL.md`](DOCS_AI_CODING_PROTOCOL.md)**: Coding Standards & Master Documentation Map
174
+ - **[`DOCS_SYSTEM_ARCHITECTURE.md`](DOCS_SYSTEM_ARCHITECTURE.md)**: Full-Stack System & WebSocket Architecture Spec
175
+ - **[`DOCS_DATA_SCHEMA.md`](DOCS_DATA_SCHEMA.md)**: WebSocket Telemetry Protocol & REST Data Schema
176
+ - **[`DOCS_MODEL_EVALUATION_AND_HF_DEPLOY.md`](DOCS_MODEL_EVALUATION_AND_HF_DEPLOY.md)**: Benchmark Evaluation & Hugging Face Hub Pipeline
177
+
178
+ ---
179
+
180
+ ## ๐Ÿ”— Open Source Hubs & Project Links
181
+
182
+ - ๐Ÿ™ **GitHub Repository**: [https://github.com/Hwihwa-Lab/pusher-v5-ppo](https://github.com/Hwihwa-Lab/pusher-v5-ppo)
183
+ - ๐Ÿค— **Hugging Face Model Hub**: [https://huggingface.co/hwihwalab/pusher-v5-ppo](https://huggingface.co/hwihwalab/pusher-v5-ppo)
184
+
185
+ ---
186
+
187
  ## ๐Ÿ“„ License
188
  This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
189
+
README_KR.md CHANGED
@@ -100,7 +100,7 @@ python app.py
100
  ```
101
  ๋ธŒ๋ผ์šฐ์ €์—์„œ **`http://localhost:8000`** ์ ‘์†.
102
 
103
- ### 3. ํ—ˆ๊น…ํŽ˜์ด์Šค ์ŠคํŽ˜์ด์Šค ์›ํด๋ฆญ ์ž๋™ ๋ฐฐํฌ
104
  ```bash
105
  python deploy_to_hf.py
106
  ```
@@ -108,13 +108,68 @@ python deploy_to_hf.py
108
  ### 4. CLI ๊ธฐ๋ฐ˜ ๋…๋ฆฝ ํ•™์Šต ๋ฐ ๋ชจ๋ธ ํ‰๊ฐ€
109
  ```bash
110
  # PPO ์—์ด์ „ํŠธ ํ•™์Šต ์‹คํ–‰
111
- python train.py --timesteps 50000 --eval_freq 10000
112
 
113
  # ํ•™์Šต ์™„๋ฃŒ๋œ ๋ชจ๋ธ ๋…๋ฆฝ ํ‰๊ฐ€ ๋ฐ ๋น„๋””์˜ค ์ถ”์ถœ
114
- python evaluate.py --model_path ./results/ppo_pusher.zip --episodes 3
115
  ```
116
 
117
  ---
118
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119
  ## ๐Ÿ“„ ๋ผ์ด์„ ์Šค
120
  ๋ณธ ํ”„๋กœ์ ํŠธ๋Š” MIT License๋ฅผ ๋”ฐ๋ฆ…๋‹ˆ๋‹ค.
 
100
  ```
101
  ๋ธŒ๋ผ์šฐ์ €์—์„œ **`http://localhost:8000`** ์ ‘์†.
102
 
103
+ ### 3. ํ—ˆ๊น…ํŽ˜์ด์Šค ์›ํด๋ฆญ ์ž๋™ ๋ฐฐํฌ
104
  ```bash
105
  python deploy_to_hf.py
106
  ```
 
108
  ### 4. CLI ๊ธฐ๋ฐ˜ ๋…๋ฆฝ ํ•™์Šต ๋ฐ ๋ชจ๋ธ ํ‰๊ฐ€
109
  ```bash
110
  # PPO ์—์ด์ „ํŠธ ํ•™์Šต ์‹คํ–‰
111
+ python train.py --timesteps 300000 --eval_freq 30000
112
 
113
  # ํ•™์Šต ์™„๋ฃŒ๋œ ๋ชจ๋ธ ๋…๋ฆฝ ํ‰๊ฐ€ ๋ฐ ๋น„๋””์˜ค ์ถ”์ถœ
114
+ python evaluate.py --model_path ./results/ppo_pusher.zip --episodes 5
115
  ```
116
 
117
  ---
118
 
119
+ ## ๐Ÿ 5์ค„ ํŒŒ์ด์ฌ ๋น ๋ฅธ ํ‰๊ฐ€ ์Šค๋‹ˆํŽซ (Quick Evaluation)
120
+
121
+ ๋ณธ ๋ฆฌํฌ์ง€ํ† ๋ฆฌ์˜ ํ•™์Šต ์™„๋ฃŒ ๊ฐ€์ค‘์น˜๋ฅผ ๋ถˆ๋Ÿฌ์™€ 5์ค„์˜ ํŒŒ์ด์ฌ ์ฝ”๋“œ๋กœ ์ฆ‰์‹œ ์‹œ๋ฎฌ๋ ˆ์ด์…˜์„ ์‹คํ–‰ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค:
122
+
123
+ ```python
124
+ import gymnasium as gym
125
+ from stable_baselines3 import PPO
126
+
127
+ # 1. Pusher-v5 ํ™˜๊ฒฝ ์ดˆ๊ธฐํ™” ๋ฐ ์™„์„ฑ ๊ฐ€์ค‘์น˜ ๋กœ๋“œ
128
+ env = gym.make("Pusher-v5", render_mode="human")
129
+ model = PPO.load("results/ppo_pusher.zip")
130
+
131
+ # 2. ๊ฒฐ์ •๋ก ์  ํ‘ธ์‹ฑ ์ œ์–ด ๋กค์•„์›ƒ ์‹คํ–‰
132
+ obs, _ = env.reset()
133
+ done = False
134
+ while not done:
135
+ action, _ = model.predict(obs, deterministic=True)
136
+ obs, reward, terminated, truncated, _ = env.step(action)
137
+ done = terminated or truncated
138
+
139
+ env.close()
140
+ ```
141
+
142
+ ---
143
+
144
+ ## โŒจ๏ธ ํ‚ค๋ณด๋“œ ๋‹จ์ถ•ํ‚ค ์•ˆ๋‚ด (Keyboard Shortcuts)
145
+
146
+ | ๋‹จ์ถ•ํ‚ค | ์กฐ์ž‘ ๊ธฐ๋Šฅ | ์„ค๋ช… |
147
+ | :---: | :--- | :--- |
148
+ | **`Space`** | **์‹œ์ž‘ / ์ผ์‹œ์ •์ง€** | ์‹ค์‹œ๊ฐ„ 30 FPS MuJoCo ๋ฌผ๋ฆฌ ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ํ† ๊ธ€ |
149
+ | **`R`** | **ํ™˜๊ฒฝ ์ดˆ๊ธฐํ™” (Reset)** | ๋กœ๋ด‡ ํŒ”, ์›ํ†ต ๋ฌผ์ฒด, ๋ชฉํ‘œ ๊ณจ๋Œ€๋ฅผ ์ƒˆ๋กœ์šด ๋žœ๋ค ์œ„์น˜๋กœ ์žฌ๋ฐฐ์น˜ |
150
+ | **`S`** | **1์Šคํ… ์ „์ง„ (Step Once)** | ๋ฌผ๋ฆฌ ์—”์ง„์„ 1๋‹จ์œ„ ํƒ€์ž„์Šคํ…(0.05์ดˆ) ์ „์ง„ |
151
+ | **`H`** | **HUD ์˜จ์˜คํ”„ ํ† ๊ธ€** | ์บ”๋ฒ„์Šค ํ™”๋ฉด ์œ„ ํ…”๋ ˆ๋ฉ”ํŠธ๋ฆฌ ์˜ค๋ฒ„๋ ˆ์ด ํ‘œ์‹œ/์ˆจ๊น€ |
152
+
153
+ ---
154
+
155
+ ## ๐Ÿ›ก๏ธ AI ์—”์ง€๋‹ˆ์–ด๋ง ๊ฑฐ๋ฒ„๋„Œ์Šค ๋ฐ ๋ฌธ์„œ ์ฒด๊ณ„
156
+
157
+ ๋ณธ ์‹œ์Šคํ…œ์€ ๊ฐ•ํ™”ํ•™์Šต ์‹œ๋ฎฌ๋ ˆ์ด์…˜์˜ ๋ฌผ๋ฆฌ์  ๋ฌด๊ฒฐ์„ฑ์„ ๋ณด์กดํ•˜๊ณ  ๋ฐ”์ด๋ธŒ-์ฝ”๋”ฉ ๋“œ๋ฆฌํ”„ํŠธ๋ฅผ ๋ฐฉ์ง€ํ•˜๊ธฐ ์œ„ํ•ด ์ •๋ฐ€ํ•œ ์—”์ง€๋‹ˆ์–ด๋ง ๋ฌธ์„œ ํ”„๋กœํ† ์ฝœ์„ ์ค€์ˆ˜ํ•ฉ๋‹ˆ๋‹ค:
158
+
159
+ - **[`.cursorrules`](.cursorrules)**: AI ์ฝ”๋”ฉ ๋ฐฉ์–ด ๋ฐ ๊ทœ์น™ ๋งˆ์Šคํ„ฐ ํ—Œ๋ฒ•
160
+ - **[`DOCS_AI_CODING_PROTOCOL.md`](DOCS_AI_CODING_PROTOCOL.md)**: ์ฝ”๋”ฉ ํ‘œ์ค€ ๋ฐ ์ „์ฒด ๋ฌธ์„œ ๋งต
161
+ - **[`DOCS_SYSTEM_ARCHITECTURE.md`](DOCS_SYSTEM_ARCHITECTURE.md)**: ํ’€์Šคํƒ ์‹œ์Šคํ…œ ๋ฐ WebSocket ์•„ํ‚คํ…์ฒ˜ ๋ช…์„ธ์„œ
162
+ - **[`DOCS_DATA_SCHEMA.md`](DOCS_DATA_SCHEMA.md)**: ํ…”๋ ˆ๋ฉ”ํŠธ๋ฆฌ ํŒจํ‚ท ํ”„๋กœํ† ์ฝœ ๋ฐ REST ๋ฐ์ดํ„ฐ ์Šคํ‚ค๋งˆ
163
+ - **[`DOCS_MODEL_EVALUATION_AND_HF_DEPLOY.md`](DOCS_MODEL_EVALUATION_AND_HF_DEPLOY.md)**: ๋ฒค์น˜๋งˆํฌ ํ‰๊ฐ€ ๋ฐ ํ—ˆ๊น…ํŽ˜์ด์Šค ๋ฐฐํฌ ๊ทœ๊ฒฉ์„œ
164
+
165
+ ---
166
+
167
+ ## ๐Ÿ”— ์˜คํ”ˆ์†Œ์Šค ๊ณต์‹ ๋งํฌ (Open Source Hubs)
168
+
169
+ - ๐Ÿ™ **GitHub ์ €์žฅ์†Œ**: [https://github.com/Hwihwa-Lab/pusher-v5-ppo](https://github.com/Hwihwa-Lab/pusher-v5-ppo)
170
+ - ๐Ÿค— **Hugging Face ๋ชจ๋ธ ํ—ˆ๋ธŒ**: [https://huggingface.co/hwihwalab/pusher-v5-ppo](https://huggingface.co/hwihwalab/pusher-v5-ppo)
171
+
172
+ ---
173
+
174
  ## ๐Ÿ“„ ๋ผ์ด์„ ์Šค
175
  ๋ณธ ํ”„๋กœ์ ํŠธ๋Š” MIT License๋ฅผ ๋”ฐ๋ฆ…๋‹ˆ๋‹ค.