NiyonshutiDavid/Uruti-pitch_coach

Model Card: Uruti Pitch Coach

Model Details

• Model Name: Uruti Pitch Coach (Multimodal RL Agent)
• Model Type: Reinforcement Learning Policy (PPO/DQN) paired with a PyTorch Multi-Layer Perceptron (MLP) Reward Model.
• Architecture: * Reward Model: A 3-layer MLP (128 -> 64 -> 1) mapping multimodal features to a 0-100 sentiment-based quality score.
  • RL Agent: Deep neural network policy trained over 300,000+ timesteps in a custom Gymnasium environment (PitchEnv-v0).
• Input Format: A 9-dimensional continuous feature vector: [slide_progress, time_progress, eye_contact, smile, stability, pitch, energy, rate, readability].
• Output Format: * Discrete Action Space (0-5): 0: Next Slide, 1: Increase Energy, 2: Decrease Energy, 3: Improve Eye Contact, 4: Speak Faster, 5: Speak Slower.
  • Continuous Score: $0.0$ to $100.0$ predicted pitch quality.

Intended Use

• Primary Use Case: Real-time presentation coaching for tech founders and startup teams. The model analyzes live multimodal streams (visual, acoustic, contextual) to provide immediate, actionable feedback during presentation practice.
• Out-of-Scope Uses: The model is not designed for psychiatric evaluation, absolute sentiment classification outside the context of a business pitch, or long-form conversational analysis.

Training Data

• Source Dataset: CMU-MOSEI (Multimodal Opinion Sentiment and Emotion Intensity) dataset.
• Data Processing: Features were extracted using OpenFace visual markers (gaze, Action Unit 12 for smiles, head pose variance) and COVAREP acoustic markers (pitch, RMS energy). Data was synthesized and mathematically correlated to simulate ideal presentation pacing, and then standardized to a normalized feature space.

Evaluation & Metrics

• Reward Model Evaluation: Evaluated using Mean Squared Error (MSE) and $R^2$ variance scoring to ensure accurate simulation of "investor sentiment."
• Policy Agent Evaluation: Evaluated over 500-episode rollouts. The final agent achieved a target mean episode reward of $\ge 80$ out of $100$, demonstrating stable convergence in navigating the simulated pitch environment.

Ethical Considerations & Limitations

• Algorithmic Bias: Facial landmark extraction and acoustic pitch tracking models (like MediaPipe and Librosa) may exhibit varying accuracy across different skin tones, facial structures, or non-native English accents.
• Latency Constraints: Real-time performance relies heavily on client-side feature extraction efficiency (e.g., in-browser WebRTC or on-device ML Kit) rather than the model inference itself. The model assumes features are pre-extracted and passed as a lightweight numerical array.

How to Get Started with the Model

You can use this model in an inference pipeline using stable-baselines3 and torch:

from stable_baselines3 import DQN
import torch

# Load the RL Agent
agent = DQN.load("path/to/DQN_pitch_coach.zip")

# Load the Reward Model
reward_model = RewardNet(input_dim=7)
reward_model.load_state_dict(torch.load("path/to/reward_model.pt"))
reward_model.eval()

# Run Inference (Example 9-dimensional feature vector)
features = [0.1, 0.05, 0.8, 0.6, 0.9, 0.5, 0.7, 0.6, 0.8]
action, _ = agent.predict(features, deterministic=True)
print(f"Recommended Coaching Action Code: {action}")