hwihwalab/microduck-3d-bipedal-teleop

🦆 MicroDuck 14-DOF Bipedal // Live 3D Teleoperation Cockpit & Physical AI Benchmark

High-Precision 14-DOF Bipedal Duck Robot Reinforcement Learning Policies, Sim-to-Real BAM Actuator Dynamics, 50Hz Teleoperation Dataset, and Interactive Three.js WebGL Real-time 3D Teleoperation Cockpit.
🌐 English Documentation | 🇰🇷 한국어 매뉴얼

🎮 Real-time 3D Web Teleoperation Space: 🤗 hwihwalab/microduck-3d-bipedal-teleop
📦 Official Model Hub: 🤗 hwihwalab/microduck-3d-bipedal-teleop
💻 Local Execution: Single-click launch via .\run_web.ps1 (Standalone Web 3D App) or .\run_simulator.ps1 (Native MuJoCo C++ Viewer).

Figure 1-A: 14-DOF Bipedal Walking Locomotion (50Hz ONNX / BAM 7.4V)

Figure 1-B: High-Speed Roller-Skating & Gliding on Passive Wheels

---

⚡ Key Engineering Highlights & Novelties

• BAM 1.0.1 Sim-to-Real Dynamic Solvers: Recreates realistic voltage-drop resistance, back-EMF, and non-linear Coulomb friction loss of 14x Dynamixel XL330-M6 servos running on a 2S 7.4V LiPo battery (6.5V ~ 8.2V).
• Actuator Latency Buffering: Enforces randomized $15\text{ms} \sim 30\text{ms}$ (delay_lag = 3 ~ 6 steps) communications delay during training, preventing real-world servo chattering and high-frequency resonance.
• Massive 196.6M Experience Scale: $4,096\text{ envs} \times 24\text{ steps} \times 2,000\text{ iterations} = \mathbf{196,608,000\text{ steps}}$, equivalent to 1,092 hours (over 45 days) of bipedal walking compressed into ~35 minutes on massively parallel GPU cluster.
• Dual-Track Operational Architecture:
  • Track 1️⃣ (Web 3D Cockpit): FastAPI + WebSockets (40 Hz telemetry) + Three.js WebGL binary mesh streaming directly from compiled MuJoCo memory buffers.
  • Track 2️⃣ (C++ Native Simulator): MuJoCo 3.12 passive OpenGL viewer with a single-line live terminal HUD (msvcrt Windows polling).
• 3-Speed Gait Transmission: On-the-fly velocity limit switching (SLOW 0.10 / NORM 0.20 / TURBO 0.35 m/s).
• Real-Time Dual Ankle Ground Contact Sensing: Continuous 50 Hz Z-coordinate tracking of left & right foot contact states (● STANCE vs ○ SWING).

---

🏛️ System Architecture & Real-Time Dataflow

The system connects the 50Hz ONNX Reinforcement Learning Policy Loop, MuJoCo BAM Sim-to-Real Actuator Dynamics, and the Three.js WebGL 3D Digital Twin Cockpit into an ultra-low-latency real-time pipeline:

flowchart TD
    subgraph Client_Layer ["1️⃣ Live 3D Web Teleoperation Cockpit (Three.js WebGL)"]
        UI1["Three.js 3D Duck Mesh Renderer"]
        UI2["Real-Time HUD Telemetry (Velocities, Gear, Dual Foot Contact, Power)"]
        UI3["Interactive Teleoperation Controls (Keyboard Shortcuts & 3-Speed Gear)"]
    end

    subgraph Server_Layer ["2️⃣ High-Throughput Streaming Server (FastAPI)"]
        WS["WebSocket Bidirectional Channel (/ws/teleop @ 50Hz)"]
        SRV["Telemetry Serializer & Command Dispatcher"]
    end

    subgraph AI_Engine ["3️⃣ Physical AI Policy Inference Engine (ONNX Runtime)"]
        OBS["61D Unified Observation Vector Builder"]
        POL["9 Core ONNX Reinforcement Learning Policies (50Hz Loop)"]
        ACT["14D Target Servo Positions (Default Pose + Action Scaling)"]
    end

    subgraph Physics_Layer ["4️⃣ Sim-to-Real Physics Engine (MuJoCo 3.12 & BAM)"]
        MJ["MuJoCo 200Hz Multibody Dynamics Solver (dt=0.005s)"]
        BAM["BAM 1.0.1 Dynamixel XL330-M6 Voltage-Drop & Friction Model"]
        SEN["Left & Right Ankle Ground Contact & Battery Power Sensor"]
    end

    subgraph Dataset_Pipeline ["5️⃣ 50Hz Teleoperation Dataset Pipeline"]
        LOG["High-Precision 50Hz Telemetry Stream Logger"]
        PQ["train.parquet (50 Episodes / 12,500 Timesteps)"]
    end

    UI3 -->|"Control Commands (Twist, Posture Flags)"| WS
    WS --> SRV
    SRV --> OBS
    OBS -->|"61D State Vector"| POL
    POL -->|"14D Joint Action Offsets"| ACT
    ACT -->|"14 Servo Target Positions"| BAM
    BAM -->|"Simulated Motor Torques & Voltage Sag"| MJ
    MJ --> SEN
    SEN -->|"State Feedback (Gyro, Joint Pos/Vel, Foot Contact)"| OBS
    SEN -->|"50Hz Live Telemetry Data Packet"| SRV
    SRV -->|"JSON Broadcast Stream"| WS
    WS --> UI1 & UI2
    SEN & ACT --> LOG --> PQ

---

📁 Repository Contents

• microduck-3d-bipedal-teleop.zip: Standalone One-Click 3D Web Cockpit & Simulator Production Archive (28.2 MB ⬇️ Direct Download).
• README.md: Official Model Card & 14-DOF Bipedal Locomotion Research Specifications.
• README_KR.md: Comprehensive Korean Manual & Teleoperation Guide.
• experiment_results.json: 4,096 Parallel Environments Training Metrics & Benchmark Results.
• checkpoints/model_final.pt: 2,000 PPO Iterations (196.6M Timesteps) PyTorch Actor-Critic Weights.
• policies/: 9 Core 50Hz ONNX Policy Weights (alpha_walking, alpha_stand, alpha_sitstand, alpha_ground_pick, roulade, ball_kick_left/right, roller, roller_crouch).
• LICENSE: Apache-2.0 License.

---

🐍 Quickstart: Python ONNX Standalone Inference

You can run any of the 9 policy ONNX models directly with onnxruntime and numpy:

import onnxruntime as ort
import numpy as np

# 1. Load the 14-DOF Bipedal Walking ONNX Policy
session = ort.InferenceSession("policies/alpha_walking.onnx")
input_name = session.get_inputs()[0].name  # 'obs'

# 2. Construct the 61-dimensional Unified Observation Vector (Batch Size = 1)
obs = np.zeros((1, 61), dtype=np.float32)

# Example: Command forward walking at 0.20 m/s (NORM gear)
obs[0, 3:6] = [0.0, 0.0, -1.0]  # Projected gravity vector (pointing down)
obs[0, 48]  = 0.20             # cmd_vx = 0.20 m/s
obs[0, 49]  = 0.00             # cmd_vy = 0.00 m/s
obs[0, 50]  = 0.00             # cmd_wz = 0.00 rad/s

# 3. Compute 50 Hz Target Joint Action Offsets (14D)
action = session.run(None, {input_name: obs})[0]
print("Computed 14D Action Offsets (rad):", action.squeeze())

# 4. Decode to physical Dynamixel Motor Positions
DEFAULT_POSE = np.array([
    0.0, -0.0873, -0.4579, -0.0049,  0.4530,  # Left Leg:  yaw, roll, pitch, knee, ankle
    0.3491,  0.3491,  0.0,     0.0,            # Neck/Head: neck_pitch, head_pitch, head_yaw, head_roll
    0.0,  0.0873,  0.4579,  0.0049, -0.4530   # Right Leg: yaw, roll, pitch, knee, ankle
], dtype=np.float32)

ACTION_SCALE = 0.25
target_motor_positions = DEFAULT_POSE + (action.squeeze() * ACTION_SCALE)
print("Target Dynamixel Angles (rad):", target_motor_positions)

---

🦾 14-DOF Robot Kinematics & Motor ID Mapping

The MicroDuck robot features 14 high-precision Robotis Dynamixel XL330-M6 smart servos. Target actions are indexed in the following strict order:

Index	Joint Name	Anatomical Group	Default (rad)	Default (deg)	Physical Motor Role
0	left_hip_yaw	Left Leg	0.0000	0.00°	Left leg steering / yaw rotation
1	left_hip_roll	Left Leg	-0.0873	-5.00°	Left lateral abduction / adduction
2	left_hip_pitch	Left Leg	-0.4579	-26.24°	Left sagittal thigh flexion / extension
3	left_knee	Left Leg	-0.0049	-0.28°	Left knee articulation
4	left_ankle	Left Leg	+0.4530	+25.95°	Left ankle pitch / foot sole ground contact
5	neck_pitch	Head & Neck	+0.3491	+20.00°	Neck base tilt / pitch
6	head_pitch	Head & Neck	+0.3491	+20.00°	Duck beak elevation / depression
7	head_yaw	Head & Neck	0.0000	0.00°	Duck head horizontal gaze scan
8	head_roll	Head & Neck	0.0000	0.00°	Duck head tilt / expression
9	right_hip_yaw	Right Leg	0.0000	0.00°	Right leg steering / yaw rotation
10	right_hip_roll	Right Leg	+0.0873	+5.00°	Right lateral abduction / adduction
11	right_hip_pitch	Right Leg	+0.4579	+26.24°	Right sagittal thigh flexion / extension
12	right_knee	Right Leg	+0.0049	+0.28°	Right knee articulation
13	right_ankle	Right Leg	-0.4530	-25.95°	Right ankle pitch / foot sole ground contact

Center of Mass (CoM) Forward Shift (STAND2 Keyframe): The trunk is intentionally pitched and shifted $\sim 5\text{mm}$ forward over the ankle rotation axis. This ensures the robot's center of mass sits securely inside the foot support polygon, preventing backward toppling without needing counterweight head drooping.

---

🔬 MicroDuck Sim-to-Real Research Specifications

1. 61D Unified Observation & 13D Command Vector Breakdown

All policies operate on a unified 61-dimensional state representation:

[obs_00 : obs_03] ( 3) -> Base Angular Velocity (Roll, Pitch, Yaw gyro [rad/s])
[obs_03 : obs_06] ( 3) -> Projected Gravity Vector (gx, gy, gz)
[obs_06 : obs_20] (14) -> Joint Positions Error (current_pos - DEFAULT_POSE [rad])
[obs_20 : obs_34] (14) -> Joint Velocities (14 joint angular rates [rad/s])
[obs_34 : obs_48] (14) -> Last Applied Action (t-1 action offsets)
[obs_48 : obs_61] (13) -> Unified Command Vector (Twist [3], Head [4], Body [4], Flags [2])
--------------------------------------------------------------------------------
Total Dimension = 61 float32 values

Slot Range	Field Name	Type	Units / Range	Description
00 : 03	base_ang_vel	float32[3]	rad/s	Base angular velocity in robot frame
03 : 06	projected_gravity	float32[3]	normalized	Gravity vector projected to base frame
06 : 20	joint_pos_error	float32[14]	rad	Angle error relative to reference DEFAULT_POSE
20 : 34	joint_vel	float32[14]	rad/s	Current angular velocity of 14 servo joints
34 : 48	last_action	float32[14]	offset	Action executed at control step $t-1$
48 : 51	cmd_velocity	float32[3]	m/s, rad/s	Commanded velocities [vx, vy, wz]
51 : 55	cmd_head_pose	float32[4]	rad	Head orientation [neck_pitch, head_pitch, head_yaw, head_roll]
55 : 59	cmd_body_pose	float32[4]	m, rad	Body offset [body_z, body_pitch, body_roll, body_yaw]
59 : 60	cmd_sitstand	float32[1]	0.0 or 1.0	Posture mode (0.0 = Stand, 1.0 = Sit)
60 : 61	cmd_aux	float32[1]	0.0	Auxiliary reserved slot

---

2. BAM (Better Actuator Models) Dynamic Parameters

• Actuator Hardware: 14x Robotis Dynamixel XL330-M6
• Nominal Battery Voltage: 7.4V (2S LiPo, operating range 6.5V ~ 8.2V)
• Peak Torque Limit: $0.963\text{ N}\cdot\text{m}$ (stall current $\approx 1.85\text{A}$)
• Firmware Stiffness: $K_{p,\text{fw}} = 200.0$ (internal Dynamixel PID loop modeling)
• Actuator Communication Latency: Random delay $\tau_{\text{lag}} \in [3, 6]\text{ steps}$ ($15\text{ms} \sim 30\text{ms}$)
• Coulomb & Viscous Friction Model:
  • stiff_solref_friction = [-50000.0, -200.0]
  • stiff_solimp_friction = [0.99, 0.9999, 0.001, 0.5, 2.0]

---

3. Massively Parallel RL & Domain Randomization

• Parallel Environments: 4,096 concurrent MuJoCo instances (--env.scene.num-envs 4096)
• Ground Friction Randomization: Uniform sampling $\mu \in [0.4, 1.25]$
• Base Mass & Inertia Jitter: Added payload $\Delta m \in [-0.05, +0.08]\text{ kg}$
• External Force Disturbance (Push): Impulse vector $F_{\text{push}} \in [5.0, 10.0]\text{ N}$ applied every $3 \sim 5\text{ s}$
• Joint Backlash Simulation: Mechanical play $\delta_{\text{backlash}} \in [\pm 0.015\text{ rad}]$

---

4. RSL-RL PPO Training Hyperparameters

• RL Algorithm: Proximal Policy Optimization (PPO) with Actor-Critic MLP
• Control Frequency: 50 Hz ($\Delta t_{\text{ctrl}} = 0.02\text{s}$)
• Physics Solver: MuJoCo 3.12 @ 200 Hz ($dt = 0.005\text{s}$, decimation = 4)
• Actor / Critic Architecture: [512, 256, 128] MLP with ELU activations
• Learning Rate: $1.0 \times 10^{-3}$ (adaptive decay schedule)
• Discount Factor ($\gamma$) / GAE ($\lambda$): $\gamma = 0.99$, $\lambda = 0.95$
• PPO Clip Ratio ($\epsilon$): $\epsilon = 0.2$
• Entropy Regularization: $c_{\text{entropy}} = 0.01$

---

5. Reward Function Design & Weight Matrix

$${\mathcal{R}}_{\text{total}}=\sum _i{w}_i\cdot r_i\backslash mathcal\{R\}\_\{\backslash text\{total\}\}\; =\; \backslash sum\_\{i\}\; w\_i\; \backslash cdot\; r\_i$$

Reward Component	Mathematical Formulation	Weight ($w_i$)	Objective
Linear Velocity ($v_x, v_y$)	$\exp(-|v_{xy} - v_{xy}^{\text{cmd}}|^2 / \sigma_v^2)$	$+1.50$	Accurate forward/lateral velocity tracking
Angular Velocity ($\omega_z$)	$\exp(-(\omega_z - \omega_z^{\text{cmd}})^2 / \sigma_\omega^2)$	$+1.00$	Precise heading and yaw rate tracking
Base Height Stability	$-|z_{\text{base}} - z_{\text{target}}|^2$	$-2.00$	Prevent body sagging or over-extension
Base Orientation Tilt	$-|g_{\text{proj}, xy}|^2$	$-1.20$	Maintain upright posture (minimize Roll/Pitch)
Torque & Jerk Minimization	$-\sum |\tau|^2 - \sum |\Delta \tau / \Delta t|^2$	$-0.002$	Energy efficiency & smooth servo motions
Foot Clearance & Airtime	$(t_{\text{air}} - 0.12)\cdot \mathbf{1}_{\text{contact}}$	$+0.40$	Dynamic foot lifting and natural bipedal stepping
Joint Limit Penalty	$-\sum \max(0,	q_j	- q_j^{\text{limit}})^2$

---

6. 3-Speed Locomotion Benchmarks & Evaluation

Figure 2: Massively Parallel 4,096 Envs PPO Convergence (196.6M Transitions), 3-Speed Velocity RMSE, Stance Duty Factor, and BAM 7.4V Power Distribution 🔍 Click to view high-res

Metric \ Speed Gear	🟢 SLOW Gear (1)	🟡 NORM Gear (2)	🔴 TURBO Gear (3)
Target Velocity ($v_x^{\text{cmd}}$)	0.10 m/s	0.20 m/s	0.35 m/s
Velocity Tracking RMSE	$\mathbf{0.009\text{ m/s}}$	$\mathbf{0.016\text{ m/s}}$	$\mathbf{0.031\text{ m/s}}$
Bipedal Balance Success Rate	100.0%	100.0%	100.0%
Average Power Consumption	4.2 W	6.4 W	11.4 W
Ground Contact Ratio (Duty Factor)	64.2%	56.8%	48.5%

---

🧠 Included 9 Core ONNX Reinforcement Learning Policies

All policies operate on the unified 61D observation vector and output 14D action offsets:

Policy Filename	Task Description	Input Shape	Output Shape	Recovery Mechanism
alpha_walking.onnx	Omnidirectional velocity-tracking walking (0.05 ~ 0.35 m/s)	[1, 61]	[1, 14]	Continuous closed-loop
alpha_stand.onnx	In-place zero-velocity standing balance & perturbation rejection	[1, 61]	[1, 14]	Continuous closed-loop
alpha_sitstand.onnx	Unified posture transition (Sit ↔ Stand on flag flip)	[1, 61]	[1, 14]	Smooth 2.0s posture glide
alpha_ground_pick.onnx	Dynamic forward bending to peck/pick ground with beak and stand up	[1, 61]	[1, 14]	4.0s phase-encoded cycle
roulade.onnx	Forward 360° tumbling roll acrobatics & self-righting standup	[1, 61]	[1, 14]	2.0s automatic return
ball_kick_left.onnx	Dynamic single-leg balance and left-foot soccer shoot	[1, 61]	[1, 14]	3.0s automatic return
ball_kick_right.onnx	Dynamic single-leg balance and right-foot soccer shoot	[1, 61]	[1, 14]	3.0s automatic return
roller.onnx	Roller-skating locomotion and steering	[1, 61]	[1, 14]	Continuous closed-loop
roller_crouch.onnx	Low-profile crouching glide on roller skates	[1, 61]	[1, 14]	Continuous closed-loop

---

📊 Teleoperation Dataset & Imitation Learning Architecture

High-frequency 50Hz teleoperation telemetry generated in the MicroDuck simulation environment follows standard Parquet time-series schemas optimized for Imitation Learning and offline RL:

Telemetry Dataset Features:

• 50Hz Synchronized Time-Series: Full 61D observation vectors, 14D target motor actions, linear/angular velocities, dual-foot contact status, and BAM battery power metrics.
• Physical AI Benchmark Compatibility: Behavioral Cloning (BC), Action Chunking with Transformers (ACT), and Diffusion Policy for Robotics.
• Refer to the Sim-to-Real Research Specifications section above for full tensor and entity breakdown.

---

🕹️ Teleoperation & Unified Keyboard Shortcuts

Key	Action	Description
↑ / ↓	Forward / Backward	Walk forward / backward acceleration
← / →	Strafe Left / Right	Lateral side-step gliding
A / D	Turn Left / Right	Heading yaw rotational velocity
1 / 2 / 3	3-Speed Gait Gear	1: SLOW (0.10m/s) | 2: NORM (0.20m/s) | 3: TURBO (0.35m/s)
SPACE	Stop / Coast	Reset all velocity commands to zero for immediate balance
S	🪑 Sit ↔ Stand	Toggle sitting down and standing back up smoothly
R	🤸 Forward Roll (Roulade)	360° forward tumbling acrobatics with self-recovery
P / G	🌾 Ground Pick	Duck beak pecking motion to touch ground and return
Q / E	⚽ Kick Left / Right	Left / Right single-leg kick motion (or K / L)
B	⚽ Reset Ball	Teleport soccer ball 22cm in front of robot
F	⚡ Force Push	Inject random physical perturbation impulse for stability tests
T	⏸️ Pause / Resume	Pause / resume policy inference loop
ESC / X	❌ Quit	Clean shutdown of simulation runtime

---

🚀 Quickstart & One-Click Launchers

# 1. Launch Standalone Web 3D Teleoperation Cockpit (Recommended)
.\run_web.ps1
# (or double-click run_web.bat)

# 2. Launch Native C++ MuJoCo Physics Viewer
.\run_simulator.ps1
# (or double-click run_simulator.bat)

# 3. Launch Roller-Skating Simulator
.\run_rollers.ps1
# (or double-click run_rollers.bat)

🌐 Official Project Links

• 🎮 Hugging Face Live Space: https://huggingface.co/spaces/hwihwalab/microduck-3d-bipedal-teleop
• 🤗 Hugging Face Model Hub: https://huggingface.co/hwihwalab/microduck-3d-bipedal-teleop

---

📄 License

This project is distributed under the Apache-2.0 License. See the LICENSE file for complete details.

---

Developed and deployed with MicroDuck & MuJoCo 3D Teleoperation by Hwihwa Lab.