Upload folder using huggingface_hub

.gitattributes

@@ -1,35 +1,4 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+benchmark/charts/intent_accuracy_delta.png filter=lfs diff=lfs merge=lfs -text
+benchmark/charts/per_intent_accuracy.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,57 @@
+---
+license: mit
+tags:
+  - onnx
+  - smartwatch
+  - intent-classification
+  - text-generation
+  - wearable
+library_name: onnxruntime
+pipeline_tag: text-generation
+---
+
+# Smartwatch LM v0.2
+
+Exported from [collab-run-2](../collab-run-2) training. Small GPT for wrist-wearable chat with intent tags like `<INTENT:GET_STEPS>`.
+
+## Model details
+
+| Property | Value |
+|----------|-------|
+| Architecture | 6-layer causal GPT (**~15.4M params**) |
+| Context length | 256 tokens |
+| Vocab size | 5533 (BPE) |
+| Best val loss | 0.3243 |
+| Training data | tinydata.txt, deepdata.txt, tinydata1.txt, data1.txt, data3.txt |
+| Export version | 0.2 |
+
+## Files
+
+| File | Purpose |
+|------|---------|
+| `smartwatch_lm_merged.onnx` | On-device inference (ONNX Runtime, opset 17, ~0 MB) |
+| `checkpoint.pt` | PyTorch weights |
+| `tokenizer.json` | BPE tokenizer |
+| `config.json` | Architecture + ONNX I/O |
+| `model.py` / `chat.py` | PyTorch load + REPL |
+| `reply_utils.py` | BPE cleanup, intent parse, slot fill |
+| `onnx_sample.py` | ONNX generate sample |
+
+## Quick start
+
+```bash
+pip install numpy onnxruntime tokenizers
+python onnx_sample.py "How many steps today?"
+```
+
+```bash
+pip install torch tokenizers
+python chat.py
+```
+
+## ONNX I/O
+
+- **Input:** `input_ids` int64 `[batch, seq]` (max seq = 256)
+- **Output:** `logits` float `[batch, seq, vocab_size]`
+
+See [Export-0.1 docs](docs/) for gibberish cleanup, intents, and device integration.

benchmark/README.md

@@ -0,0 +1,100 @@
+# Smartwatch LM Quality Benchmark
+
+Side-by-side quality comparison for exported models (default: **Export-0.1** vs **export-0.2**).
+
+## Setup
+
+```bash
+pip install -r benchmark/requirements.txt
+```
+
+Both export folders must contain `checkpoint.pt` and `tokenizer.json`.
+
+## Run
+
+```bash
+python benchmark/benchmark_quality.py
+```
+
+Common options:
+
+```bash
+python benchmark/benchmark_quality.py --verbose
+python benchmark/benchmark_quality.py --output benchmark/report.json
+python benchmark/benchmark_quality.py --model-a Export-0.1 --model-b export-0.2 --seed 42
+```
+
+Generation defaults match production guidance: `max_new_tokens=40`, `temperature=0.5`, `top_k=40`, fresh history per prompt, fixed seed for reproducibility.
+
+## Metrics
+
+| Metric | Meaning |
+|--------|---------|
+| **Intent accuracy** | Predicted intent matches expected (or is in `expected_intents` for combo cases) |
+| **Intent parse rate** | Reply contains a valid `<INTENT:...>` tag |
+| **Clean output rate** | Raw decode has no BPE junk (`Ġ`, `Ċ`) before cleanup |
+| **Slot presence** | Fraction of expected slot placeholders found in the cleaned template |
+
+Checkpoint `best_val_loss` is printed for reference only. v0.1 and v0.2 used different tokenizers and training corpora, so val loss is **not** a fair head-to-head score.
+
+## Golden prompts
+
+[`benchmark_prompts.json`](benchmark_prompts.json) holds ~39 prompts (one per intent found in `tinydata.txt`, plus Colab demo prompts).
+
+Regenerate after training data changes:
+
+```bash
+python benchmark/extract_prompts.py
+python benchmark/extract_prompts.py --data tinydata/tinydata.txt --output benchmark/benchmark_prompts.json
+```
+
+Each entry:
+
+```json
+{
+  "id": "get_steps_...",
+  "prompt": "How many steps today?",
+  "expected_intent": "GET_STEPS",
+  "expected_intents": ["GET_STEPS"],
+  "expected_slots": ["STEPS_TODAY", "STEP_GOAL"],
+  "source": "tinydata.txt"
+}
+```
+
+## Output
+
+The runner prints:
+
+- Overall metric table for both models
+- Per-intent accuracy breakdown
+- Counts of prompts only v0.1 got right, only v0.2 got right, and both missed
+
+With `--verbose`, it also prints per-prompt predicted intents and replies.
+
+With `--output`, it writes a JSON report suitable for CI or regression tracking.
+
+## Charts
+
+By default the runner saves four PNG charts to `benchmark/charts/`:
+
+| Chart | Description |
+|-------|-------------|
+| `overall_metrics.png` | Grouped bar chart of all four quality metrics |
+| `per_intent_accuracy.png` | Side-by-side accuracy for each intent |
+| `head_to_head_wins.png` | Prompts both got right, only v0.1, only v0.2, both wrong |
+| `intent_accuracy_delta.png` | Per-intent delta (export-0.2 minus Export-0.1) |
+
+```bash
+python benchmark/benchmark_quality.py --output benchmark/report.json
+python benchmark/benchmark_quality.py --no-charts
+python benchmark/benchmark_quality.py --charts-dir path/to/charts
+```
+
+Regenerate charts from an existing JSON report without re-running models:
+
+```bash
+python benchmark/benchmark_charts.py --report benchmark/report.json
+python benchmark/benchmark_charts.py --report benchmark/report.json --output-dir benchmark/charts
+```
+
+Requires `matplotlib` (included in `requirements.txt`).

benchmark/benchmark_prompts.json

@@ -0,0 +1,487 @@
+[
+  {
+    "id": "none_hi_00",
+    "prompt": "hi",
+    "expected_intent": "NONE",
+    "expected_intents": [
+      "NONE"
+    ],
+    "expected_slots": [],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_steps_hey_am_i_doing_okay_with_steps_today_or_01",
+    "prompt": "Hey — am I doing okay with steps today or should I try to move more?",
+    "expected_intent": "GET_STEPS",
+    "expected_intents": [
+      "GET_STEPS"
+    ],
+    "expected_slots": [
+      "STEPS_TODAY",
+      "STEP_GOAL"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "set_step_goal_i_want_to_change_my_default_daily_step_t_02",
+    "prompt": "I want to change my default daily step target to 10,000 steps — can you update it?",
+    "expected_intent": "SET_STEP_GOAL",
+    "expected_intents": [
+      "SET_STEP_GOAL"
+    ],
+    "expected_slots": [
+      "STEP_GOAL"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_workout_status_how_long_have_i_been_on_my_run_and_what_03",
+    "prompt": "How long have I been on my run, and what's my heart rate right now?",
+    "expected_intent": "GET_WORKOUT_STATUS",
+    "expected_intents": [
+      "GET_WORKOUT_STATUS"
+    ],
+    "expected_slots": [
+      "WORKOUT_ELAPSED",
+      "WORKOUT_TYPE"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_distance_quick_check_what_s_my_total_distance_tod_04",
+    "prompt": "Quick check — what's my total distance today?",
+    "expected_intent": "GET_DISTANCE",
+    "expected_intents": [
+      "GET_DISTANCE"
+    ],
+    "expected_slots": [
+      "DISTANCE_TODAY",
+      "DISTANCE_UNIT"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_battery_quick_question_is_my_battery_high_enough_05",
+    "prompt": "Quick question — is my battery high enough to track a walk? Also, how many steps have I done today?",
+    "expected_intent": "GET_BATTERY",
+    "expected_intents": [
+      "GET_BATTERY"
+    ],
+    "expected_slots": [
+      "BATTERY_PCT"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_heart_rate_can_you_check_my_heart_rate_now_i_m_a_bi_06",
+    "prompt": "Can you check my heart rate now? I'm a bit worried.",
+    "expected_intent": "GET_HEART_RATE",
+    "expected_intents": [
+      "GET_HEART_RATE"
+    ],
+    "expected_slots": [
+      "HR_CURRENT_BPM"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "measure_heart_rate_please_take_my_pulse_now_i_feel_a_little_07",
+    "prompt": "Please take my pulse now — I feel a little fluttery.",
+    "expected_intent": "MEASURE_HEART_RATE",
+    "expected_intents": [
+      "MEASURE_HEART_RATE"
+    ],
+    "expected_slots": [
+      "HR_CURRENT_BPM"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_sleep_hey_how_long_did_i_sleep_last_night_08",
+    "prompt": "Hey, how long did I sleep last night?",
+    "expected_intent": "GET_SLEEP",
+    "expected_intents": [
+      "GET_SLEEP"
+    ],
+    "expected_slots": [
+      "SLEEP_HOURS_LAST_NIGHT"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "log_nap_just_woke_up_from_a_quick_power_nap_can_09",
+    "prompt": "Just woke up from a quick power nap — can you log it?",
+    "expected_intent": "LOG_NAP",
+    "expected_intents": [
+      "LOG_NAP"
+    ],
+    "expected_slots": [
+      "DURATION",
+      "SLEEP_HOURS_LAST_NIGHT"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_calories_hey_how_many_calories_have_i_burned_sinc_10",
+    "prompt": "Hey, how many calories have I burned since midnight?",
+    "expected_intent": "GET_CALORIES",
+    "expected_intents": [
+      "GET_CALORIES"
+    ],
+    "expected_slots": [
+      "CALORIES_TODAY",
+      "CALORIE_GOAL"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "set_calorie_goal_lower_my_daily_active_calorie_burn_targe_11",
+    "prompt": "Lower my daily active calorie burn target to 400, please.",
+    "expected_intent": "SET_CALORIE_GOAL",
+    "expected_intents": [
+      "SET_CALORIE_GOAL"
+    ],
+    "expected_slots": [
+      "CALORIE_GOAL"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_active_minutes_how_many_active_minutes_have_i_done_toda_12",
+    "prompt": "How many active minutes have I done today?",
+    "expected_intent": "GET_ACTIVE_MINUTES",
+    "expected_intents": [
+      "GET_ACTIVE_MINUTES"
+    ],
+    "expected_slots": [
+      "ACTIVE_MINUTES_TODAY",
+      "ACTIVE_MINUTES_GOAL"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "set_active_goal_can_you_raise_my_daily_active_minutes_go_13",
+    "prompt": "Can you raise my daily active minutes goal to 45 minutes?",
+    "expected_intent": "SET_ACTIVE_GOAL",
+    "expected_intents": [
+      "SET_ACTIVE_GOAL"
+    ],
+    "expected_slots": [
+      "ACTIVE_MINUTES_GOAL"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "enable_power_save_battery_s_low_turn_on_power_saving_now_14",
+    "prompt": "Battery's low — turn on power saving now.",
+    "expected_intent": "ENABLE_POWER_SAVE",
+    "expected_intents": [
+      "ENABLE_POWER_SAVE"
+    ],
+    "expected_slots": [
+      "BATTERY_PCT"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "disable_aod_please_turn_off_the_always_on_display_to_15",
+    "prompt": "Please turn off the always-on display to save battery.",
+    "expected_intent": "DISABLE_AOD",
+    "expected_intents": [
+      "DISABLE_AOD"
+    ],
+    "expected_slots": [
+      "BATTERY_PCT"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "set_alarm_i_need_an_alarm_for_tomorrow_morning_to_16",
+    "prompt": "I need an alarm for tomorrow morning to make sure I wake up for my run.",
+    "expected_intent": "SET_ALARM",
+    "expected_intents": [
+      "SET_ALARM"
+    ],
+    "expected_slots": [
+      "ALARM_TIME",
+      "DATE",
+      "ALARM_LABEL"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "list_alarms_quick_check_what_alarms_are_set_on_my_wa_17",
+    "prompt": "Quick check — what alarms are set on my watch?",
+    "expected_intent": "LIST_ALARMS",
+    "expected_intents": [
+      "LIST_ALARMS"
+    ],
+    "expected_slots": [
+      "ALARM_TIME",
+      "ALARM_LABEL",
+      "ALARM_TIME",
+      "ALARM_LABEL",
+      "ALARM_TIME",
+      "ALARM_LABEL"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "delete_alarm_please_delete_my_standard_morning_wake_u_18",
+    "prompt": "Please delete my standard morning wake-up alarm.",
+    "expected_intent": "DELETE_ALARM",
+    "expected_intents": [
+      "DELETE_ALARM"
+    ],
+    "expected_slots": [
+      "ALARM_LABEL"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "explain_nudge_hey_why_did_my_watch_just_vibrate_19",
+    "prompt": "hey, why did my watch just vibrate?",
+    "expected_intent": "EXPLAIN_NUDGE",
+    "expected_intents": [
+      "EXPLAIN_NUDGE"
+    ],
+    "expected_slots": [
+      "NUDGE_REASON",
+      "DURATION"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "mute_reminders_i_m_getting_hourly_move_reminders_mute_t_20",
+    "prompt": "I'm getting hourly move reminders — mute them entirely, please.",
+    "expected_intent": "MUTE_REMINDERS",
+    "expected_intents": [
+      "MUTE_REMINDERS"
+    ],
+    "expected_slots": [],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "snooze_alarm_ugh_it_s_buzzing_snooze_this_alarm_for_a_21",
+    "prompt": "Ugh, it's buzzing — snooze this alarm for a few minutes.",
+    "expected_intent": "SNOOZE_ALARM",
+    "expected_intents": [
+      "SNOOZE_ALARM"
+    ],
+    "expected_slots": [
+      "ALARM_LABEL",
+      "ALARM_TIME",
+      "DURATION"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "set_reminder_i_keep_forgetting_to_drink_water_can_you_22",
+    "prompt": "I keep forgetting to drink water — can you set a repeating reminder every two hours to drink water, please?",
+    "expected_intent": "SET_REMINDER",
+    "expected_intents": [
+      "SET_REMINDER"
+    ],
+    "expected_slots": [
+      "ALARM_LABEL",
+      "REMINDER_INTERVAL"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "start_timer_start_a_twenty_minute_timer_23",
+    "prompt": "Start a twenty-minute timer.",
+    "expected_intent": "START_TIMER",
+    "expected_intents": [
+      "START_TIMER"
+    ],
+    "expected_slots": [
+      "DURATION"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_timer_remaining_how_much_time_is_left_on_the_cooking_tim_24",
+    "prompt": "How much time is left on the cooking timer? I'm juggling a few things here.",
+    "expected_intent": "GET_TIMER_REMAINING",
+    "expected_intents": [
+      "GET_TIMER_REMAINING"
+    ],
+    "expected_slots": [
+      "TIMER_REMAINING",
+      "DURATION"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "pause_timer_pause_the_countdown_now_freeze_it_25",
+    "prompt": "Pause the countdown now — freeze it.",
+    "expected_intent": "PAUSE_TIMER",
+    "expected_intents": [
+      "PAUSE_TIMER"
+    ],
+    "expected_slots": [
+      "DURATION",
+      "TIMER_REMAINING"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "start_stopwatch_start_the_stopwatch_app_and_begin_timing_26",
+    "prompt": "Start the stopwatch app and begin timing.",
+    "expected_intent": "START_STOPWATCH",
+    "expected_intents": [
+      "START_STOPWATCH"
+    ],
+    "expected_slots": [],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "lap_stopwatch_mark_a_lap_split_27",
+    "prompt": "Mark a lap split.",
+    "expected_intent": "LAP_STOPWATCH",
+    "expected_intents": [
+      "LAP_STOPWATCH"
+    ],
+    "expected_slots": [
+      "LAP_TIME",
+      "DURATION"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "reset_stopwatch_hey_reset_my_stopwatch_and_clear_its_lap_28",
+    "prompt": "Hey, reset my stopwatch and clear its laps please.",
+    "expected_intent": "RESET_STOPWATCH",
+    "expected_intents": [
+      "RESET_STOPWATCH"
+    ],
+    "expected_slots": [
+      "DURATION",
+      "LAP_TIME"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "cancel_timer_stop_and_clear_my_running_timer_please_29",
+    "prompt": "Stop and clear my running timer, please.",
+    "expected_intent": "CANCEL_TIMER",
+    "expected_intents": [
+      "CANCEL_TIMER"
+    ],
+    "expected_slots": [
+      "TIMER_REMAINING",
+      "DURATION"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "start_workout_i_m_heading_out_for_an_outdoor_walk_star_30",
+    "prompt": "I'm heading out for an outdoor walk — start tracking, please.",
+    "expected_intent": "START_WORKOUT",
+    "expected_intents": [
+      "START_WORKOUT"
+    ],
+    "expected_slots": [
+      "WORKOUT_TYPE"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "stop_workout_i_m_done_with_my_run_stop_tracking_and_s_31",
+    "prompt": "I'm done with my run — stop tracking and save the session.",
+    "expected_intent": "STOP_WORKOUT",
+    "expected_intents": [
+      "STOP_WORKOUT"
+    ],
+    "expected_slots": [
+      "WORKOUT_TYPE",
+      "WORKOUT_ELAPSED"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "pause_workout_traffic_light_pause_my_workout_while_i_w_32",
+    "prompt": "Traffic light—pause my workout while I wait?",
+    "expected_intent": "PAUSE_WORKOUT",
+    "expected_intents": [
+      "PAUSE_WORKOUT"
+    ],
+    "expected_slots": [
+      "WORKOUT_ELAPSED",
+      "WORKOUT_STATE"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "resume_workout_hey_resume_my_cycling_workout_from_earli_33",
+    "prompt": "hey, resume my cycling workout from earlier",
+    "expected_intent": "RESUME_WORKOUT",
+    "expected_intents": [
+      "RESUME_WORKOUT"
+    ],
+    "expected_slots": [
+      "WORKOUT_TYPE",
+      "WORKOUT_ELAPSED",
+      "WORKOUT_STATE"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "discard_workout_delete_the_current_run_and_erase_its_dat_34",
+    "prompt": "Delete the current run and erase its data.",
+    "expected_intent": "DISCARD_WORKOUT",
+    "expected_intents": [
+      "DISCARD_WORKOUT"
+    ],
+    "expected_slots": [
+      "WORKOUT_TYPE",
+      "WORKOUT_STATE"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_workout_summary_just_finished_a_walk_show_the_final_summ_35",
+    "prompt": "Just finished a walk — show the final summary screen.",
+    "expected_intent": "GET_WORKOUT_SUMMARY",
+    "expected_intents": [
+      "GET_WORKOUT_SUMMARY"
+    ],
+    "expected_slots": [
+      "WORKOUT_TYPE",
+      "WORKOUT_ELAPSED",
+      "DISTANCE_TODAY"
+    ],
+    "source": "tinydata.txt"
+  },
+  {
+    "id": "get_steps_how_many_steps_have_i_taken_today_36",
+    "prompt": "How many steps have I taken today?",
+    "expected_intent": "GET_STEPS",
+    "expected_intents": [
+      "GET_STEPS"
+    ],
+    "expected_slots": [],
+    "source": "DEMO_PROMPTS"
+  },
+  {
+    "id": "start_timer_start_a_10_minute_timer_37",
+    "prompt": "Start a 10 minute timer",
+    "expected_intent": "START_TIMER",
+    "expected_intents": [
+      "START_TIMER"
+    ],
+    "expected_slots": [],
+    "source": "DEMO_PROMPTS"
+  },
+  {
+    "id": "none_hey_38",
+    "prompt": "hey",
+    "expected_intent": "NONE",
+    "expected_intents": [
+      "NONE"
+    ],
+    "expected_slots": [],
+    "source": "DEMO_PROMPTS"
+  }
+]

benchmark/report.json

@@ -0,0 +1,1347 @@
+{
+  "models": [
+    {
+      "name": "Export-0.1",
+      "export_dir": "D:\\Projects\\electron-v1\\Export-0.1",
+      "device": "cpu",
+      "best_val_loss": 2.0472678637504576,
+      "intent_accuracy": 0.717948717948718,
+      "intent_parse_rate": 1.0,
+      "clean_output_rate": 1.0,
+      "slot_presence": 0.717948717948718,
+      "results": [
+        {
+          "case_id": "none_hi_00",
+          "prompt": "hi",
+          "expected_intent": "NONE",
+          "predicted_intent": "NONE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:NONE> That makes sense — if you want , I can help you keep an eye on it . \n \n user : hey , do I have enough charge to make it through overnight",
+          "display_reply": "That makes sense — if you want , I can help you keep an eye on it ."
+        },
+        {
+          "case_id": "get_steps_hey_am_i_doing_okay_with_steps_today_or_01",
+          "prompt": "Hey — am I doing okay with steps today or should I try to move more?",
+          "expected_intent": "GET_STEPS",
+          "predicted_intent": "GET_ACTIVE_MINUTES",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:GET_ACTIVE_MINUTES> You're at <ACTIVE_MINUTES_TODAY> active minutes today , with <ACTIVE_MINUTES_REMAINING> left to reach your < ACTIVE _ MINUTES _",
+          "display_reply": "You're at <ACTIVE_MINUTES_TODAY> active minutes today , with <ACTIVE_MINUTES_REMAINING> left to reach your < ACTIVE _ MINUTES _"
+        },
+        {
+          "case_id": "set_step_goal_i_want_to_change_my_default_daily_step_t_02",
+          "prompt": "I want to change my default daily step target to 10,000 steps — can you update it?",
+          "expected_intent": "SET_STEP_GOAL",
+          "predicted_intent": "SET_STEP_GOAL",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:SET_STEP_GOAL> Sure — I can help compare it with your <ALARM_GOAL>. Nice job keeping the rest of your wrist . \n user : Thanks , that helps .",
+          "display_reply": "Sure — I can help compare it with your <ALARM_GOAL>. Nice job keeping the rest of your wrist ."
+        },
+        {
+          "case_id": "get_workout_status_how_long_have_i_been_on_my_run_and_what_03",
+          "prompt": "How long have I been on my run, and what's my heart rate right now?",
+          "expected_intent": "GET_WORKOUT_STATUS",
+          "predicted_intent": "GET_HEART_RATE",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:GET_HEART_RATE> Your heart rate stayed elevated for <HR_ELEVATED_MINUTES> over the last <DURATION> — nice work . \n user : Thanks , that helps .",
+          "display_reply": "Your heart rate stayed elevated for <HR_ELEVATED_MINUTES> over the last <DURATION> — nice work ."
+        },
+        {
+          "case_id": "get_distance_quick_check_what_s_my_total_distance_tod_04",
+          "prompt": "Quick check — what's my total distance today?",
+          "expected_intent": "GET_DISTANCE",
+          "predicted_intent": "GET_DISTANCE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_DISTANCE> You're at <DISTANCE_TODAY> <DISTANCE_UNIT> today . \n user : nice , thanks \n bot : <INTENT:NONE> Anytime !",
+          "display_reply": "You're at <DISTANCE_TODAY> <DISTANCE_UNIT> today ."
+        },
+        {
+          "case_id": "get_battery_quick_question_is_my_battery_high_enough_05",
+          "prompt": "Quick question — is my battery high enough to track a walk? Also, how many steps have I done today?",
+          "expected_intent": "GET_BATTERY",
+          "predicted_intent": "GET_STEPS",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:GET_STEPS> You' re at <STEPS_TODAY> right now , so you should be set for <DURATION> — nice pace . \n user : Great , thanks .",
+          "display_reply": "You' re at <STEPS_TODAY> right now , so you should be set for <DURATION> — nice pace ."
+        },
+        {
+          "case_id": "get_heart_rate_can_you_check_my_heart_rate_now_i_m_a_bi_06",
+          "prompt": "Can you check my heart rate now? I'm a bit worried.",
+          "expected_intent": "GET_HEART_RATE",
+          "predicted_intent": "GET_HEART_RATE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_HEART_RATE> I' m checking your heart rate now . You' re at <HR_CURRENT_BPM>, and your resting rate is < HR _ RESTING _ BPM",
+          "display_reply": "I' m checking your heart rate now . You' re at <HR_CURRENT_BPM>, and your resting rate is < HR _ RESTING _ BPM"
+        },
+        {
+          "case_id": "measure_heart_rate_please_take_my_pulse_now_i_feel_a_little_07",
+          "prompt": "Please take my pulse now — I feel a little fluttery.",
+          "expected_intent": "MEASURE_HEART_RATE",
+          "predicted_intent": "NONE",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:NONE> Nice , you' re in the day . \n \n user : hey , what's my usual resting heart rate ? \n bot : < INTENT : GET _ HEART _ RATE",
+          "display_reply": "Nice , you' re in the day ."
+        },
+        {
+          "case_id": "get_sleep_hey_how_long_did_i_sleep_last_night_08",
+          "prompt": "Hey, how long did I sleep last night?",
+          "expected_intent": "GET_SLEEP",
+          "predicted_intent": "GET_SLEEP",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_SLEEP> You logged <SLEEP_HOURS_LAST_NIGHT> last night — nice work getting some rest . \n user : Thanks , I just wanted to check . \n bot",
+          "display_reply": "You logged <SLEEP_HOURS_LAST_NIGHT> last night — nice work getting some rest ."
+        },
+        {
+          "case_id": "log_nap_just_woke_up_from_a_quick_power_nap_can_09",
+          "prompt": "Just woke up from a quick power nap — can you log it?",
+          "expected_intent": "LOG_NAP",
+          "predicted_intent": "LOG_NAP",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:LOG_NAP> Got it — I logged a nap of <DURATION>. You slept <SLEEP_HOURS_LAST_NIGHT> last night , so this little reset makes sense .",
+          "display_reply": "Got it — I logged a nap of <DURATION>. You slept <SLEEP_HOURS_LAST_NIGHT> last night , so this little reset makes sense ."
+        },
+        {
+          "case_id": "get_calories_hey_how_many_calories_have_i_burned_sinc_10",
+          "prompt": "Hey, how many calories have I burned since midnight?",
+          "expected_intent": "GET_CALORIES",
+          "predicted_intent": "GET_CALORIES",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_CALORIES> You' ve burned <CALORIES_TODAY> so far today , and your goal is <CALORIE_GOAL> — nice progress . \n user : Thanks , that helps",
+          "display_reply": "You' ve burned <CALORIES_TODAY> so far today , and your goal is <CALORIE_GOAL> — nice progress ."
+        },
+        {
+          "case_id": "set_calorie_goal_lower_my_daily_active_calorie_burn_targe_11",
+          "prompt": "Lower my daily active calorie burn target to 400, please.",
+          "expected_intent": "SET_CALORIE_GOAL",
+          "predicted_intent": "NONE",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:NONE> Anytime ! \n \n user : hey , how much of my eight - hour sleep goal have i finished today ? \n bot : <INTENT:GET_SLEEP> You'",
+          "display_reply": "Anytime !"
+        },
+        {
+          "case_id": "get_active_minutes_how_many_active_minutes_have_i_done_toda_12",
+          "prompt": "How many active minutes have I done today?",
+          "expected_intent": "GET_ACTIVE_MINUTES",
+          "predicted_intent": "GET_ACTIVE_MINUTES",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_ACTIVE_MINUTES> You' ve got <ACTIVE_MINUTES_TODAY> active minutes today out of <ACTIVE_MINUTES_GOAL> — nice progress . \n user : Thanks",
+          "display_reply": "You' ve got <ACTIVE_MINUTES_TODAY> active minutes today out of <ACTIVE_MINUTES_GOAL> — nice progress ."
+        },
+        {
+          "case_id": "set_active_goal_can_you_raise_my_daily_active_minutes_go_13",
+          "prompt": "Can you raise my daily active minutes goal to 45 minutes?",
+          "expected_intent": "SET_ACTIVE_GOAL",
+          "predicted_intent": "SET_ACTIVE_GOAL",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:SET_ACTIVE_GOAL> Sure — I' ve set your daily active minutes goal to <ACTIVE_MINUTES_GOAL>. \n user : Thanks , that should help . \n bot :",
+          "display_reply": "Sure — I' ve set your daily active minutes goal to <ACTIVE_MINUTES_GOAL>."
+        },
+        {
+          "case_id": "enable_power_save_battery_s_low_turn_on_power_saving_now_14",
+          "prompt": "Battery's low — turn on power saving now.",
+          "expected_intent": "ENABLE_POWER_SAVE",
+          "predicted_intent": "ENABLE_POWER_SAVE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:ENABLE_POWER_SAVE> Power saving is on now . Battery is at <BATTERY_PCT>, so this should help it last longer . \n user : thanks , that should help .",
+          "display_reply": "Power saving is on now . Battery is at <BATTERY_PCT>, so this should help it last longer ."
+        },
+        {
+          "case_id": "disable_aod_please_turn_off_the_always_on_display_to_15",
+          "prompt": "Please turn off the always-on display to save battery.",
+          "expected_intent": "DISABLE_AOD",
+          "predicted_intent": "DISABLE_AOD",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:DISABLE_AOD> Done — always - on display is off now , and your battery is at <BATTERY_PCT>. \n user : Thanks , that should help . \n bot : <",
+          "display_reply": "Done — always - on display is off now , and your battery is at <BATTERY_PCT>."
+        },
+        {
+          "case_id": "set_alarm_i_need_an_alarm_for_tomorrow_morning_to_16",
+          "prompt": "I need an alarm for tomorrow morning to make sure I wake up for my run.",
+          "expected_intent": "SET_ALARM",
+          "predicted_intent": "SET_ALARM",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:SET_ALARM> Done — I set your alarm for <ALARM_TIME> on <DATE> with the label <ALARM_LABEL>. \n user : Thanks , that should help .",
+          "display_reply": "Done — I set your alarm for <ALARM_TIME> on <DATE> with the label <ALARM_LABEL>."
+        },
+        {
+          "case_id": "list_alarms_quick_check_what_alarms_are_set_on_my_wa_17",
+          "prompt": "Quick check — what alarms are set on my watch?",
+          "expected_intent": "LIST_ALARMS",
+          "predicted_intent": "GET_HEART_RATE",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:GET_HEART_RATE> You're at <HR_CURRENT_BPM> bpm , which puts you in <HR_ZONE>. Nice work staying aware of your effort . \n user",
+          "display_reply": "You're at <HR_CURRENT_BPM> bpm , which puts you in <HR_ZONE>. Nice work staying aware of your effort ."
+        },
+        {
+          "case_id": "delete_alarm_please_delete_my_standard_morning_wake_u_18",
+          "prompt": "Please delete my standard morning wake-up alarm.",
+          "expected_intent": "DELETE_ALARM",
+          "predicted_intent": "DELETE_ALARM",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:DELETE_ALARM> Deleted <ALARM_LABEL>. Thanks . \n user : Thanks , that' s perfect . \n bot : <INTENT:NONE> Anytime ! \n \n user :",
+          "display_reply": "Deleted <ALARM_LABEL>. Thanks ."
+        },
+        {
+          "case_id": "explain_nudge_hey_why_did_my_watch_just_vibrate_19",
+          "prompt": "hey, why did my watch just vibrate?",
+          "expected_intent": "EXPLAIN_NUDGE",
+          "predicted_intent": "EXPLAIN_NUDGE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:EXPLAIN_NUDGE> It looks like a sedentary move nudge for <NUDGE_REASON> — you' ve been still for <DURATION>, so it' s reminding you to get up and",
+          "display_reply": "It looks like a sedentary move nudge for <NUDGE_REASON> — you' ve been still for <DURATION>, so it' s reminding you to get up and"
+        },
+        {
+          "case_id": "mute_reminders_i_m_getting_hourly_move_reminders_mute_t_20",
+          "prompt": "I'm getting hourly move reminders — mute them entirely, please.",
+          "expected_intent": "MUTE_REMINDERS",
+          "predicted_intent": "NONE",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:NONE> Anytime ! \n \n user : my alarm is buzzing , snooze it for 5 minutes \n bot : <INTENT:SNOOZE_ALARM> Snoozed <ALARM_LABEL> set for",
+          "display_reply": "Anytime !"
+        },
+        {
+          "case_id": "snooze_alarm_ugh_it_s_buzzing_snooze_this_alarm_for_a_21",
+          "prompt": "Ugh, it's buzzing �� snooze this alarm for a few minutes.",
+          "expected_intent": "SNOOZE_ALARM",
+          "predicted_intent": "SNOOZE_ALARM",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:SNOOZE_ALARM> Snoozed <ALARM_LABEL> set for <ALARM_TIME> for <DURATION>. \n user : Thanks , that helps . \n bot : <INTENT:NONE>",
+          "display_reply": "Snoozed <ALARM_LABEL> set for <ALARM_TIME> for <DURATION>."
+        },
+        {
+          "case_id": "set_reminder_i_keep_forgetting_to_drink_water_can_you_22",
+          "prompt": "I keep forgetting to drink water — can you set a repeating reminder every two hours to drink water, please?",
+          "expected_intent": "SET_REMINDER",
+          "predicted_intent": "SET_REMINDER",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:SET_REMINDER> Sure — I' ll set a repeating reminder for <ALARM_LABEL> every <REMINDER_INTERVAL>. \n user : Thanks , that helps . \n bot : <",
+          "display_reply": "Sure — I' ll set a repeating reminder for <ALARM_LABEL> every <REMINDER_INTERVAL>."
+        },
+        {
+          "case_id": "start_timer_start_a_twenty_minute_timer_23",
+          "prompt": "Start a twenty-minute timer.",
+          "expected_intent": "START_TIMER",
+          "predicted_intent": "START_TIMER",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:START_TIMER> Sure — starting a timer for <DURATION>. \n user : Thanks , that' s perfect . \n bot : <INTENT:NONE> Anytime ! \n \n user",
+          "display_reply": "Sure — starting a timer for <DURATION>."
+        },
+        {
+          "case_id": "get_timer_remaining_how_much_time_is_left_on_the_cooking_tim_24",
+          "prompt": "How much time is left on the cooking timer? I'm juggling a few things here.",
+          "expected_intent": "GET_TIMER_REMAINING",
+          "predicted_intent": "GET_HEART_RATE",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_HEART_RATE> You' ve got <TIMER_REMAINING> left on your <DURATION> timer . \n user : Nice , thanks . \n bot : < INTENT : NONE",
+          "display_reply": "You' ve got <TIMER_REMAINING> left on your <DURATION> timer ."
+        },
+        {
+          "case_id": "pause_timer_pause_the_countdown_now_freeze_it_25",
+          "prompt": "Pause the countdown now — freeze it.",
+          "expected_intent": "PAUSE_TIMER",
+          "predicted_intent": "PAUSE_TIMER",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.5,
+          "raw_reply": "<INTENT:PAUSE_TIMER> Paused with <TIMER_REMAINING> left on the clock . \n user : Thanks , keep it there for now . \n bot : <INTENT:NONE> Anytime !",
+          "display_reply": "Paused with <TIMER_REMAINING> left on the clock ."
+        },
+        {
+          "case_id": "start_stopwatch_start_the_stopwatch_app_and_begin_timing_26",
+          "prompt": "Start the stopwatch app and begin timing.",
+          "expected_intent": "START_STOPWATCH",
+          "predicted_intent": "START_STOPWATCH",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:START_STOPWATCH> Stopwatch is open and ready . \n user : Thanks . \n bot : <INTENT:NONE> Anytime ! \n \n user : I'm on my run , can",
+          "display_reply": "Stopwatch is open and ready ."
+        },
+        {
+          "case_id": "lap_stopwatch_mark_a_lap_split_27",
+          "prompt": "Mark a lap split.",
+          "expected_intent": "LAP_STOPWATCH",
+          "predicted_intent": "LAP_STOPWATCH",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:LAP_STOPWATCH> Lap recorded : <LAP_TIME>. Total time is <DURATION>. \n user : Nice , thanks . \n bot : <INTENT:NONE> Anytime !",
+          "display_reply": "Lap recorded : <LAP_TIME>. Total time is <DURATION>."
+        },
+        {
+          "case_id": "reset_stopwatch_hey_reset_my_stopwatch_and_clear_its_lap_28",
+          "prompt": "Hey, reset my stopwatch and clear its laps please.",
+          "expected_intent": "RESET_STOPWATCH",
+          "predicted_intent": "RESET_STOPWATCH",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:RESET_STOPWATCH> Done — the stopwatch is back to zero , and any lap time is cleared too . \n user : Nice , thanks . \n bot : <INTENT:NONE> Anytime",
+          "display_reply": "Done — the stopwatch is back to zero , and any lap time is cleared too ."
+        },
+        {
+          "case_id": "cancel_timer_stop_and_clear_my_running_timer_please_29",
+          "prompt": "Stop and clear my running timer, please.",
+          "expected_intent": "CANCEL_TIMER",
+          "predicted_intent": "PAUSE_TIMER",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.5,
+          "raw_reply": "<INTENT:PAUSE_TIMER> Paused with <TIMER_REMAINING> left on the clock . \n user : Thanks , keep it there for now . \n bot : <INTENT:NONE> Anytime !",
+          "display_reply": "Paused with <TIMER_REMAINING> left on the clock ."
+        },
+        {
+          "case_id": "start_workout_i_m_heading_out_for_an_outdoor_walk_star_30",
+          "prompt": "I'm heading out for an outdoor walk — start tracking, please.",
+          "expected_intent": "START_WORKOUT",
+          "predicted_intent": "START_WORKOUT",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:START_WORKOUT> Got it — starting <WORKOUT_TYPE> for your ride . \n user : Nice , thanks . \n bot : <INTENT:NONE> Anytime !",
+          "display_reply": "Got it — starting <WORKOUT_TYPE> for your ride ."
+        },
+        {
+          "case_id": "stop_workout_i_m_done_with_my_run_stop_tracking_and_s_31",
+          "prompt": "I'm done with my run — stop tracking and save the session.",
+          "expected_intent": "STOP_WORKOUT",
+          "predicted_intent": "STOP_WORKOUT",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:STOP_WORKOUT> Got it — I' ve saved your <WORKOUT_TYPE> session at <WORKOUT_ELAPSED>. Nice work today . \n \n user : pause my workout while",
+          "display_reply": "Got it — I' ve saved your <WORKOUT_TYPE> session at <WORKOUT_ELAPSED>. Nice work today ."
+        },
+        {
+          "case_id": "pause_workout_traffic_light_pause_my_workout_while_i_w_32",
+          "prompt": "Traffic light—pause my workout while I wait?",
+          "expected_intent": "PAUSE_WORKOUT",
+          "predicted_intent": "GET_HEART_RATE",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:GET_HEART_RATE> You' re at <BATTERY_TODAY> right now , with <DURATION> because of <NUDGE_YESTERDAY>. If today feels off , I can help",
+          "display_reply": "You' re at <BATTERY_TODAY> right now , with <DURATION> because of <NUDGE_YESTERDAY>. If today feels off , I can help"
+        },
+        {
+          "case_id": "resume_workout_hey_resume_my_cycling_workout_from_earli_33",
+          "prompt": "hey, resume my cycling workout from earlier",
+          "expected_intent": "RESUME_WORKOUT",
+          "predicted_intent": "RESUME_WORKOUT",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:RESUME_WORKOUT> Resuming your <WORKOUT_TYPE> workout — <WORKOUT_STATE> with <WORKOUT_ELAPSED> already logged . \n user : nice , thanks \n bot : <",
+          "display_reply": "Resuming your <WORKOUT_TYPE> workout — <WORKOUT_STATE> with <WORKOUT_ELAPSED> already logged ."
+        },
+        {
+          "case_id": "discard_workout_delete_the_current_run_and_erase_its_dat_34",
+          "prompt": "Delete the current run and erase its data.",
+          "expected_intent": "DISCARD_WORKOUT",
+          "predicted_intent": "NONE",
+          "intent_correct": false,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:NONE> Anytime ! \n \n user : hey , why did my steps today compared with yesterday ? \n bot : <INTENT:GET_STEPS> You're at < STEPS _ TODAY",
+          "display_reply": "Anytime !"
+        },
+        {
+          "case_id": "get_workout_summary_just_finished_a_walk_show_the_final_summ_35",
+          "prompt": "Just finished a walk — show the final summary screen.",
+          "expected_intent": "GET_WORKOUT_SUMMARY",
+          "predicted_intent": "GET_WORKOUT_SUMMARY",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_WORKOUT_SUMMARY> Here' s your <WORKOUT_TYPE> summary : <WORKOUT_ELAPSED> total , and you covered <DISTANCE_TODAY> today . \n user",
+          "display_reply": "Here' s your <WORKOUT_TYPE> summary : <WORKOUT_ELAPSED> total , and you covered <DISTANCE_TODAY> today ."
+        },
+        {
+          "case_id": "get_steps_how_many_steps_have_i_taken_today_36",
+          "prompt": "How many steps have I taken today?",
+          "expected_intent": "GET_STEPS",
+          "predicted_intent": "GET_STEPS",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_STEPS> You' re at <STEPS_TODAY> of <STEP_GOAL> today — nice work . \n user : Thanks , am I close to my goal ?",
+          "display_reply": "You' re at <STEPS_TODAY> of <STEP_GOAL> today — nice work ."
+        },
+        {
+          "case_id": "start_timer_start_a_10_minute_timer_37",
+          "prompt": "Start a 10 minute timer",
+          "expected_intent": "START_TIMER",
+          "predicted_intent": "START_TIMER",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:START_TIMER> Sure — starting a <DURATION> timer now . \n user : Thanks . \n bot : <INTENT:NONE> Anytime ! \n \n user : hey , how",
+          "display_reply": "Sure — starting a <DURATION> timer now ."
+        },
+        {
+          "case_id": "none_hey_38",
+          "prompt": "hey",
+          "expected_intent": "NONE",
+          "predicted_intent": "NONE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:hey,howmuchdistancedoistillneedthisweek?bot:<INTENT:GET_DISTANCE> You' ve covered <DISTANCE_TODAY> <DISTANCE_UNIT> today",
+          "display_reply": "<INTENT:hey,howmuchdistancedoistillneedthisweek?bot:<INTENT:GET_DISTANCE> You' ve covered <DISTANCE_TODAY> <DISTANCE_UNIT> today"
+        }
+      ],
+      "elapsed_sec": 37.20833020005375,
+      "per_intent": {
+        "CANCEL_TIMER": {
+          "correct": 0,
+          "total": 1,
+          "accuracy": 0.0
+        },
+        "DELETE_ALARM": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "DISABLE_AOD": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "DISCARD_WORKOUT": {
+          "correct": 0,
+          "total": 1,
+          "accuracy": 0.0
+        },
+        "ENABLE_POWER_SAVE": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "EXPLAIN_NUDGE": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_ACTIVE_MINUTES": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_BATTERY": {
+          "correct": 0,
+          "total": 1,
+          "accuracy": 0.0
+        },
+        "GET_CALORIES": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_DISTANCE": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_HEART_RATE": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_SLEEP": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_STEPS": {
+          "correct": 1,
+          "total": 2,
+          "accuracy": 0.5
+        },
+        "GET_TIMER_REMAINING": {
+          "correct": 0,
+          "total": 1,
+          "accuracy": 0.0
+        },
+        "GET_WORKOUT_STATUS": {
+          "correct": 0,
+          "total": 1,
+          "accuracy": 0.0
+        },
+        "GET_WORKOUT_SUMMARY": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "LAP_STOPWATCH": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "LIST_ALARMS": {
+          "correct": 0,
+          "total": 1,
+          "accuracy": 0.0
+        },
+        "LOG_NAP": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "MEASURE_HEART_RATE": {
+          "correct": 0,
+          "total": 1,
+          "accuracy": 0.0
+        },
+        "MUTE_REMINDERS": {
+          "correct": 0,
+          "total": 1,
+          "accuracy": 0.0
+        },
+        "NONE": {
+          "correct": 2,
+          "total": 2,
+          "accuracy": 1.0
+        },
+        "PAUSE_TIMER": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "PAUSE_WORKOUT": {
+          "correct": 0,
+          "total": 1,
+          "accuracy": 0.0
+        },
+        "RESET_STOPWATCH": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "RESUME_WORKOUT": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SET_ACTIVE_GOAL": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SET_ALARM": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SET_CALORIE_GOAL": {
+          "correct": 0,
+          "total": 1,
+          "accuracy": 0.0
+        },
+        "SET_REMINDER": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SET_STEP_GOAL": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SNOOZE_ALARM": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "START_STOPWATCH": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "START_TIMER": {
+          "correct": 2,
+          "total": 2,
+          "accuracy": 1.0
+        },
+        "START_WORKOUT": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "STOP_WORKOUT": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        }
+      }
+    },
+    {
+      "name": "export-0.2",
+      "export_dir": "D:\\Projects\\electron-v1\\export-0.2",
+      "device": "cpu",
+      "best_val_loss": 0.32432237863540647,
+      "intent_accuracy": 1.0,
+      "intent_parse_rate": 1.0,
+      "clean_output_rate": 1.0,
+      "slot_presence": 0.9615384615384616,
+      "results": [
+        {
+          "case_id": "none_hi_00",
+          "prompt": "hi",
+          "expected_intent": "NONE",
+          "predicted_intent": "NONE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:NONE> Hey there — good to see you .",
+          "display_reply": "Hey there — good to see you ."
+        },
+        {
+          "case_id": "get_steps_hey_am_i_doing_okay_with_steps_today_or_01",
+          "prompt": "Hey — am I doing okay with steps today or should I try to move more?",
+          "expected_intent": "GET_STEPS",
+          "predicted_intent": "GET_STEPS",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_STEPS> You' re at <STEPS_TODAY> of <STEP_GOAL>. That leaves <STEPS_REMAINING> to go .",
+          "display_reply": "You' re at <STEPS_TODAY> of <STEP_GOAL>. That leaves <STEPS_REMAINING> to go ."
+        },
+        {
+          "case_id": "set_step_goal_i_want_to_change_my_default_daily_step_t_02",
+          "prompt": "I want to change my default daily step target to 10,000 steps — can you update it?",
+          "expected_intent": "SET_STEP_GOAL",
+          "predicted_intent": "SET_STEP_GOAL",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:SET_STEP_GOAL> Done — your daily step goal is now <STEP_GOAL>. I'll track progress on your watch and nudge you if you need a mid - afternoon .",
+          "display_reply": "Done — your daily step goal is now <STEP_GOAL>. I'll track progress on your watch and nudge you if you need a mid - afternoon ."
+        },
+        {
+          "case_id": "get_workout_status_how_long_have_i_been_on_my_run_and_what_03",
+          "prompt": "How long have I been on my run, and what's my heart rate right now?",
+          "expected_intent": "GET_WORKOUT_STATUS",
+          "predicted_intent": "GET_WORKOUT_STATUS",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_WORKOUT_STATUS> You're <WORKOUT_ELAPSED> into your <WORKOUT_TYPE> session .",
+          "display_reply": "You're <WORKOUT_ELAPSED> into your <WORKOUT_TYPE> session ."
+        },
+        {
+          "case_id": "get_distance_quick_check_what_s_my_total_distance_tod_04",
+          "prompt": "Quick check — what's my total distance today?",
+          "expected_intent": "GET_DISTANCE",
+          "predicted_intent": "GET_DISTANCE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_DISTANCE> You've logged <DISTANCE_TODAY> <DISTANCE_UNIT> today .",
+          "display_reply": "You've logged <DISTANCE_TODAY> <DISTANCE_UNIT> today ."
+        },
+        {
+          "case_id": "get_battery_quick_question_is_my_battery_high_enough_05",
+          "prompt": "Quick question — is my battery high enough to track a walk? Also, how many steps have I done today?",
+          "expected_intent": "GET_BATTERY",
+          "predicted_intent": "GET_BATTERY",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_BATTERY> Battery is at <BATTERY_PCT> — plenty for a walk .",
+          "display_reply": "Battery is at <BATTERY_PCT> — plenty for a walk ."
+        },
+        {
+          "case_id": "get_heart_rate_can_you_check_my_heart_rate_now_i_m_a_bi_06",
+          "prompt": "Can you check my heart rate now? I'm a bit worried.",
+          "expected_intent": "GET_HEART_RATE",
+          "predicted_intent": "GET_HEART_RATE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_HEART_RATE> Your current heart rate is <HR_CURRENT_BPM>. That's a bit elevated , but if you feel dizzy , short of breath , or chest pain",
+          "display_reply": "Your current heart rate is <HR_CURRENT_BPM>. That's a bit elevated , but if you feel dizzy , short of breath , or chest pain"
+        },
+        {
+          "case_id": "measure_heart_rate_please_take_my_pulse_now_i_feel_a_little_07",
+          "prompt": "Please take my pulse now — I feel a little fluttery.",
+          "expected_intent": "MEASURE_HEART_RATE",
+          "predicted_intent": "MEASURE_HEART_RATE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.0,
+          "raw_reply": "<INTENT:MEASURE_HEART_RATE> I' m measuring your pulse now . Please hold still for a moment .",
+          "display_reply": "I' m measuring your pulse now . Please hold still for a moment ."
+        },
+        {
+          "case_id": "get_sleep_hey_how_long_did_i_sleep_last_night_08",
+          "prompt": "Hey, how long did I sleep last night?",
+          "expected_intent": "GET_SLEEP",
+          "predicted_intent": "GET_SLEEP",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_SLEEP> You slept <SLEEP_HOURS_LAST_NIGHT> last night — nice work .",
+          "display_reply": "You slept <SLEEP_HOURS_LAST_NIGHT> last night — nice work ."
+        },
+        {
+          "case_id": "log_nap_just_woke_up_from_a_quick_power_nap_can_09",
+          "prompt": "Just woke up from a quick power nap — can you log it?",
+          "expected_intent": "LOG_NAP",
+          "predicted_intent": "LOG_NAP",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:LOG_NAP> Got it , logging a nap of <DURATION>. You had <SLEEP_HOURS_LAST_NIGHT> last night , so this should help a bit .",
+          "display_reply": "Got it , logging a nap of <DURATION>. You had <SLEEP_HOURS_LAST_NIGHT> last night , so this should help a bit ."
+        },
+        {
+          "case_id": "get_calories_hey_how_many_calories_have_i_burned_sinc_10",
+          "prompt": "Hey, how many calories have I burned since midnight?",
+          "expected_intent": "GET_CALORIES",
+          "predicted_intent": "GET_CALORIES",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_CALORIES> You' ve burned <CALORIES_TODAY> so far today , and your goal is <CALORIE_GOAL> — nice work .",
+          "display_reply": "You' ve burned <CALORIES_TODAY> so far today , and your goal is <CALORIE_GOAL> — nice work ."
+        },
+        {
+          "case_id": "set_calorie_goal_lower_my_daily_active_calorie_burn_targe_11",
+          "prompt": "Lower my daily active calorie burn target to 400, please.",
+          "expected_intent": "SET_CALORIE_GOAL",
+          "predicted_intent": "SET_CALORIE_GOAL",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:SET_CALORIE_GOAL> Done — your daily active calorie goal is now <CALORIE_GOAL>. I' ll keep track and let you know how you' re doing .",
+          "display_reply": "Done — your daily active calorie goal is now <CALORIE_GOAL>. I' ll keep track and let you know how you' re doing ."
+        },
+        {
+          "case_id": "get_active_minutes_how_many_active_minutes_have_i_done_toda_12",
+          "prompt": "How many active minutes have I done today?",
+          "expected_intent": "GET_ACTIVE_MINUTES",
+          "predicted_intent": "GET_ACTIVE_MINUTES",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_ACTIVE_MINUTES> You' ve got <ACTIVE_MINUTES_TODAY> active minutes so far , with a goal of <ACTIVE_MINUTES_GOAL>.",
+          "display_reply": "You' ve got <ACTIVE_MINUTES_TODAY> active minutes so far , with a goal of <ACTIVE_MINUTES_GOAL>."
+        },
+        {
+          "case_id": "set_active_goal_can_you_raise_my_daily_active_minutes_go_13",
+          "prompt": "Can you raise my daily active minutes goal to 45 minutes?",
+          "expected_intent": "SET_ACTIVE_GOAL",
+          "predicted_intent": "SET_ACTIVE_GOAL",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:SET_ACTIVE_GOAL> Done — your daily active minutes goal is now <ACTIVE_MINUTES_GOAL>.",
+          "display_reply": "Done — your daily active minutes goal is now <ACTIVE_MINUTES_GOAL>."
+        },
+        {
+          "case_id": "enable_power_save_battery_s_low_turn_on_power_saving_now_14",
+          "prompt": "Battery's low — turn on power saving now.",
+          "expected_intent": "ENABLE_POWER_SAVE",
+          "predicted_intent": "ENABLE_POWER_SAVE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:ENABLE_POWER_SAVE> Power saving is on . Battery is at <BATTERY_PCT>, so I'll dim the screen , limit background sync , and reduce sensor use .",
+          "display_reply": "Power saving is on . Battery is at <BATTERY_PCT>, so I'll dim the screen , limit background sync , and reduce sensor use ."
+        },
+        {
+          "case_id": "disable_aod_please_turn_off_the_always_on_display_to_15",
+          "prompt": "Please turn off the always-on display to save battery.",
+          "expected_intent": "DISABLE_AOD",
+          "predicted_intent": "DISABLE_AOD",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:DISABLE_AOD> Done — always - on display is off , and your battery is at <BATTERY_PCT>.",
+          "display_reply": "Done — always - on display is off , and your battery is at <BATTERY_PCT>."
+        },
+        {
+          "case_id": "set_alarm_i_need_an_alarm_for_tomorrow_morning_to_16",
+          "prompt": "I need an alarm for tomorrow morning to make sure I wake up for my run.",
+          "expected_intent": "SET_ALARM",
+          "predicted_intent": "SET_ALARM",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:SET_ALARM> Got it — I set an alarm for <ALARM_TIME> on <DATE> with the label <ALARM_LABEL>.",
+          "display_reply": "Got it — I set an alarm for <ALARM_TIME> on <DATE> with the label <ALARM_LABEL>."
+        },
+        {
+          "case_id": "list_alarms_quick_check_what_alarms_are_set_on_my_wa_17",
+          "prompt": "Quick check — what alarms are set on my watch?",
+          "expected_intent": "LIST_ALARMS",
+          "predicted_intent": "LIST_ALARMS",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:LIST_ALARMS> Here are your active alarms : <ALARM_TIME> — <ALARM_LABEL>; <ALARM_TIME> — <ALARM_LABEL>; <ALARM_TIME>",
+          "display_reply": "Here are your active alarms : <ALARM_TIME> — <ALARM_LABEL>; <ALARM_TIME> — <ALARM_LABEL>; <ALARM_TIME>"
+        },
+        {
+          "case_id": "delete_alarm_please_delete_my_standard_morning_wake_u_18",
+          "prompt": "Please delete my standard morning wake-up alarm.",
+          "expected_intent": "DELETE_ALARM",
+          "predicted_intent": "DELETE_ALARM",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:DELETE_ALARM> Got it — I deleted <ALARM_LABEL>.",
+          "display_reply": "Got it — I deleted <ALARM_LABEL>."
+        },
+        {
+          "case_id": "explain_nudge_hey_why_did_my_watch_just_vibrate_19",
+          "prompt": "hey, why did my watch just vibrate?",
+          "expected_intent": "EXPLAIN_NUDGE",
+          "predicted_intent": "EXPLAIN_NUDGE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:EXPLAIN_NUDGE> It looks like a sedentary move nudge for <NUDGE_REASON> — you' ve been still for <DURATION>, so it' s reminding you to get up and",
+          "display_reply": "It looks like a sedentary move nudge for <NUDGE_REASON> — you' ve been still for <DURATION>, so it' s reminding you to get up and"
+        },
+        {
+          "case_id": "mute_reminders_i_m_getting_hourly_move_reminders_mute_t_20",
+          "prompt": "I'm getting hourly move reminders — mute them entirely, please.",
+          "expected_intent": "MUTE_REMINDERS",
+          "predicted_intent": "MUTE_REMINDERS",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:MUTE_REMINDERS> Got it — I' ve muted the hourly move reminders on your watch .",
+          "display_reply": "Got it — I' ve muted the hourly move reminders on your watch ."
+        },
+        {
+          "case_id": "snooze_alarm_ugh_it_s_buzzing_snooze_this_alarm_for_a_21",
+          "prompt": "Ugh, it's buzzing — snooze this alarm for a few minutes.",
+          "expected_intent": "SNOOZE_ALARM",
+          "predicted_intent": "SNOOZE_ALARM",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:SNOOZE_ALARM> Got it — snoozing <ALARM_LABEL> set for <ALARM_TIME> for <DURATION>. I'll buzz you again then .",
+          "display_reply": "Got it — snoozing <ALARM_LABEL> set for <ALARM_TIME> for <DURATION>. I'll buzz you again then ."
+        },
+        {
+          "case_id": "set_reminder_i_keep_forgetting_to_drink_water_can_you_22",
+          "prompt": "I keep forgetting to drink water — can you set a repeating reminder every two hours to drink water, please?",
+          "expected_intent": "SET_REMINDER",
+          "predicted_intent": "SET_REMINDER",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:SET_REMINDER> Got it — I' ll set a repeating reminder for <ALARM_LABEL> every <REMINDER_INTERVAL>.",
+          "display_reply": "Got it — I' ll set a repeating reminder for <ALARM_LABEL> every <REMINDER_INTERVAL>."
+        },
+        {
+          "case_id": "start_timer_start_a_twenty_minute_timer_23",
+          "prompt": "Start a twenty-minute timer.",
+          "expected_intent": "START_TIMER",
+          "predicted_intent": "START_TIMER",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:START_TIMER> Got it — starting a timer for <DURATION>.",
+          "display_reply": "Got it — starting a timer for <DURATION>."
+        },
+        {
+          "case_id": "get_timer_remaining_how_much_time_is_left_on_the_cooking_tim_24",
+          "prompt": "How much time is left on the cooking timer? I'm juggling a few things here.",
+          "expected_intent": "GET_TIMER_REMAINING",
+          "predicted_intent": "GET_TIMER_REMAINING",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_TIMER_REMAINING> The timer has <TIMER_REMAINING> remaining out of the <DURATION>.",
+          "display_reply": "The timer has <TIMER_REMAINING> remaining out of the <DURATION>."
+        },
+        {
+          "case_id": "pause_timer_pause_the_countdown_now_freeze_it_25",
+          "prompt": "Pause the countdown now — freeze it.",
+          "expected_intent": "PAUSE_TIMER",
+          "predicted_intent": "PAUSE_TIMER",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:PAUSE_TIMER> Paused your timer — <TIMER_REMAINING> remaining of the <DURATION>. I'll keep it frozen until you say resume .",
+          "display_reply": "Paused your timer — <TIMER_REMAINING> remaining of the <DURATION>. I'll keep it frozen until you say resume ."
+        },
+        {
+          "case_id": "start_stopwatch_start_the_stopwatch_app_and_begin_timing_26",
+          "prompt": "Start the stopwatch app and begin timing.",
+          "expected_intent": "START_STOPWATCH",
+          "predicted_intent": "START_STOPWATCH",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:START_STOPWATCH> Stopwatch started — I'll record time on your wrist . Tap to lap or say \" stop \" to stop .",
+          "display_reply": "Stopwatch started — I'll record time on your wrist . Tap to lap or say \" stop \" to stop ."
+        },
+        {
+          "case_id": "lap_stopwatch_mark_a_lap_split_27",
+          "prompt": "Mark a lap split.",
+          "expected_intent": "LAP_STOPWATCH",
+          "predicted_intent": "LAP_STOPWATCH",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:LAP_STOPWATCH> Got it — lap recorded at <LAP_TIME>, with the run at <DURATION>.",
+          "display_reply": "Got it — lap recorded at <LAP_TIME>, with the run at <DURATION>."
+        },
+        {
+          "case_id": "reset_stopwatch_hey_reset_my_stopwatch_and_clear_its_lap_28",
+          "prompt": "Hey, reset my stopwatch and clear its laps please.",
+          "expected_intent": "RESET_STOPWATCH",
+          "predicted_intent": "RESET_STOPWATCH",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:RESET_STOPWATCH> Stopwatch reset to <DURATION>, and the last lap is cleared from <LAP_TIME>.",
+          "display_reply": "Stopwatch reset to <DURATION>, and the last lap is cleared from <LAP_TIME>."
+        },
+        {
+          "case_id": "cancel_timer_stop_and_clear_my_running_timer_please_29",
+          "prompt": "Stop and clear my running timer, please.",
+          "expected_intent": "CANCEL_TIMER",
+          "predicted_intent": "CANCEL_TIMER",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 0.5,
+          "raw_reply": "<INTENT:CANCEL_TIMER> Got it — I canceled the timer with <TIMER_REMAINING> left and cleared it out .",
+          "display_reply": "Got it — I canceled the timer with <TIMER_REMAINING> left and cleared it out ."
+        },
+        {
+          "case_id": "start_workout_i_m_heading_out_for_an_outdoor_walk_star_30",
+          "prompt": "I'm heading out for an outdoor walk — start tracking, please.",
+          "expected_intent": "START_WORKOUT",
+          "predicted_intent": "START_WORKOUT",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:START_WORKOUT> Got it — starting <WORKOUT_TYPE> now . Enjoy the walk .",
+          "display_reply": "Got it — starting <WORKOUT_TYPE> now . Enjoy the walk ."
+        },
+        {
+          "case_id": "stop_workout_i_m_done_with_my_run_stop_tracking_and_s_31",
+          "prompt": "I'm done with my run — stop tracking and save the session.",
+          "expected_intent": "STOP_WORKOUT",
+          "predicted_intent": "STOP_WORKOUT",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:STOP_WORKOUT> Got it , I stopped your <WORKOUT_TYPE> and saved the session at <WORKOUT_ELAPSED>. Nice work today .",
+          "display_reply": "Got it , I stopped your <WORKOUT_TYPE> and saved the session at <WORKOUT_ELAPSED>. Nice work today ."
+        },
+        {
+          "case_id": "pause_workout_traffic_light_pause_my_workout_while_i_w_32",
+          "prompt": "Traffic light—pause my workout while I wait?",
+          "expected_intent": "PAUSE_WORKOUT",
+          "predicted_intent": "PAUSE_WORKOUT",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:PAUSE_WORKOUT> Pausing your workout now . You've been active for <WORKOUT_ELAPSED>. Current state : <WORKOUT_STATE>. I'll keep it paused until you resume .",
+          "display_reply": "Pausing your workout now . You've been active for <WORKOUT_ELAPSED>. Current state : <WORKOUT_STATE>. I'll keep it paused until you resume ."
+        },
+        {
+          "case_id": "resume_workout_hey_resume_my_cycling_workout_from_earli_33",
+          "prompt": "hey, resume my cycling workout from earlier",
+          "expected_intent": "RESUME_WORKOUT",
+          "predicted_intent": "RESUME_WORKOUT",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:RESUME_WORKOUT> Resuming your <WORKOUT_TYPE> workout . You' re back in <WORKOUT_STATE>, with <WORKOUT_ELAPSED> already logged .",
+          "display_reply": "Resuming your <WORKOUT_TYPE> workout . You' re back in <WORKOUT_STATE>, with <WORKOUT_ELAPSED> already logged ."
+        },
+        {
+          "case_id": "discard_workout_delete_the_current_run_and_erase_its_dat_34",
+          "prompt": "Delete the current run and erase its data.",
+          "expected_intent": "DISCARD_WORKOUT",
+          "predicted_intent": "DISCARD_WORKOUT",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:DISCARD_WORKOUT> Got it — I' ll discard the <WORKOUT_TYPE> while it' s <WORKOUT_STATE>.",
+          "display_reply": "Got it — I' ll discard the <WORKOUT_TYPE> while it' s <WORKOUT_STATE>."
+        },
+        {
+          "case_id": "get_workout_summary_just_finished_a_walk_show_the_final_summ_35",
+          "prompt": "Just finished a walk — show the final summary screen.",
+          "expected_intent": "GET_WORKOUT_SUMMARY",
+          "predicted_intent": "GET_WORKOUT_SUMMARY",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_WORKOUT_SUMMARY> Here' s your <WORKOUT_TYPE> summary : <WORKOUT_ELAPSED> elapsed and <DISTANCE_TODAY> covered today .",
+          "display_reply": "Here' s your <WORKOUT_TYPE> summary : <WORKOUT_ELAPSED> elapsed and <DISTANCE_TODAY> covered today ."
+        },
+        {
+          "case_id": "get_steps_how_many_steps_have_i_taken_today_36",
+          "prompt": "How many steps have I taken today?",
+          "expected_intent": "GET_STEPS",
+          "predicted_intent": "GET_STEPS",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:GET_STEPS> You' re at <STEPS_TODAY> of <STEP_GOAL> so far .",
+          "display_reply": "You' re at <STEPS_TODAY> of <STEP_GOAL> so far ."
+        },
+        {
+          "case_id": "start_timer_start_a_10_minute_timer_37",
+          "prompt": "Start a 10 minute timer",
+          "expected_intent": "START_TIMER",
+          "predicted_intent": "START_TIMER",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:START_TIMER> Got it — starting a timer for <DURATION>.",
+          "display_reply": "Got it — starting a timer for <DURATION>."
+        },
+        {
+          "case_id": "none_hey_38",
+          "prompt": "hey",
+          "expected_intent": "NONE",
+          "predicted_intent": "NONE",
+          "intent_correct": true,
+          "has_intent_tag": true,
+          "clean_output": true,
+          "slot_presence": 1.0,
+          "raw_reply": "<INTENT:NONE> hi there — I'm listening .",
+          "display_reply": "hi there — I'm listening ."
+        }
+      ],
+      "elapsed_sec": 31.05121770000551,
+      "per_intent": {
+        "CANCEL_TIMER": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "DELETE_ALARM": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "DISABLE_AOD": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "DISCARD_WORKOUT": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "ENABLE_POWER_SAVE": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "EXPLAIN_NUDGE": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_ACTIVE_MINUTES": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_BATTERY": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_CALORIES": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_DISTANCE": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_HEART_RATE": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_SLEEP": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_STEPS": {
+          "correct": 2,
+          "total": 2,
+          "accuracy": 1.0
+        },
+        "GET_TIMER_REMAINING": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_WORKOUT_STATUS": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "GET_WORKOUT_SUMMARY": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "LAP_STOPWATCH": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "LIST_ALARMS": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "LOG_NAP": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "MEASURE_HEART_RATE": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "MUTE_REMINDERS": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "NONE": {
+          "correct": 2,
+          "total": 2,
+          "accuracy": 1.0
+        },
+        "PAUSE_TIMER": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "PAUSE_WORKOUT": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "RESET_STOPWATCH": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "RESUME_WORKOUT": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SET_ACTIVE_GOAL": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SET_ALARM": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SET_CALORIE_GOAL": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SET_REMINDER": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SET_STEP_GOAL": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "SNOOZE_ALARM": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "START_STOPWATCH": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "START_TIMER": {
+          "correct": 2,
+          "total": 2,
+          "accuracy": 1.0
+        },
+        "START_WORKOUT": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        },
+        "STOP_WORKOUT": {
+          "correct": 1,
+          "total": 1,
+          "accuracy": 1.0
+        }
+      }
+    }
+  ],
+  "comparison": {
+    "only_a": [],
+    "only_b": [
+      "get_steps_hey_am_i_doing_okay_with_steps_today_or_01",
+      "get_workout_status_how_long_have_i_been_on_my_run_and_what_03",
+      "get_battery_quick_question_is_my_battery_high_enough_05",
+      "measure_heart_rate_please_take_my_pulse_now_i_feel_a_little_07",
+      "set_calorie_goal_lower_my_daily_active_calorie_burn_targe_11",
+      "list_alarms_quick_check_what_alarms_are_set_on_my_wa_17",
+      "mute_reminders_i_m_getting_hourly_move_reminders_mute_t_20",
+      "get_timer_remaining_how_much_time_is_left_on_the_cooking_tim_24",
+      "cancel_timer_stop_and_clear_my_running_timer_please_29",
+      "pause_workout_traffic_light_pause_my_workout_while_i_w_32",
+      "discard_workout_delete_the_current_run_and_erase_its_dat_34"
+    ],
+    "both_fail": []
+  }
+}

chat.py

@@ -0,0 +1,127 @@
+"""
+Interactive chat with the exported Smartwatch LM.
+
+Usage:
+    pip install torch tokenizers
+    python chat.py
+"""
+
+from __future__ import annotations
+
+import re
+import sys
+from pathlib import Path
+
+import torch
+
+import config as cfg
+from model import load_model
+from reply_utils import build_prompt, extract_bot_reply_from_continuation, extract_intent_reply
+
+
+class ChatSession:
+    def __init__(
+        self,
+        checkpoint_path: Path | None = None,
+        tokenizer_path: Path | None = None,
+        device: str | None = None,
+        max_new_tokens: int | None = None,
+        temperature: float | None = None,
+        top_k: int | None = None,
+    ):
+        self.model, self.tokenizer, self.device = load_model(
+            checkpoint_path, tokenizer_path, device
+        )
+        self.max_new_tokens = max_new_tokens or cfg.SAMPLE_MAX_NEW_TOKENS
+        self.temperature = temperature if temperature is not None else cfg.SAMPLE_TEMPERATURE
+        self.top_k = top_k if top_k is not None else cfg.SAMPLE_TOP_K
+        self.history: list[tuple[str, str]] = []
+
+    def reset(self) -> None:
+        self.history.clear()
+
+    @torch.no_grad()
+    def say(self, user_message: str) -> str:
+        user_message = user_message.strip()
+        if not user_message:
+            return ""
+
+        prompt = build_prompt(self.history, user_message)
+        start_ids = self.tokenizer.encode(prompt).ids
+        x = torch.tensor([start_ids], dtype=torch.long, device=self.device)
+        y = self.model.generate(
+            x,
+            max_new_tokens=self.max_new_tokens,
+            temperature=self.temperature,
+            top_k=self.top_k,
+        )
+        new_ids = y[0, len(start_ids) :].tolist()
+        continuation = self.tokenizer.decode(new_ids)
+        reply = extract_bot_reply_from_continuation(continuation)
+        self.history.append((user_message, reply))
+        return reply
+
+    def say_display(self, user_message: str) -> tuple[str, str, str]:
+        """Return (raw_reply, intent, display_text)."""
+        raw = self.say(user_message)
+        parsed = extract_intent_reply(raw)
+        return raw, parsed.intent, parsed.template
+
+
+def print_banner() -> None:
+    print("Smartwatch LM chat — type a message and press Enter.")
+    print("Commands: quit/exit  |  reset (clear history)  |  history")
+    print("-" * 60)
+
+
+def run_repl() -> None:
+    try:
+        session = ChatSession()
+    except FileNotFoundError as exc:
+        print(exc, file=sys.stderr)
+        sys.exit(1)
+
+    val_loss = None
+    ckpt_path = cfg.OUTPUT_DIR / "checkpoint.pt"
+    if ckpt_path.is_file():
+        checkpoint = torch.load(ckpt_path, map_location="cpu", weights_only=False)
+        val_loss = checkpoint.get("best_val_loss")
+
+    print_banner()
+    print(f"device: {session.device}")
+    if val_loss is not None:
+        print(f"checkpoint val loss: {val_loss:.4f}")
+    print()
+
+    while True:
+        try:
+            user_input = input("you> ").strip()
+        except (EOFError, KeyboardInterrupt):
+            print("\nbye")
+            break
+
+        if not user_input:
+            continue
+        lowered = user_input.lower()
+        if lowered in {"quit", "exit"}:
+            print("bye")
+            break
+        if lowered == "reset":
+            session.reset()
+            print("(history cleared)")
+            continue
+        if lowered == "history":
+            if not session.history:
+                print("(empty)")
+            for user_text, bot_text in session.history:
+                print(f"user: {user_text}\nbot: {bot_text}\n")
+            continue
+
+        _, intent, display = session.say_display(user_input)
+        print(f"bot> {display}")
+        if intent and intent != "NONE":
+            print(f"     intent: {intent}")
+
+
+if __name__ == "__main__":
+    run_repl()

checkpoint.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac7d9d1a4688dd528c436bdc4e26e7b7d1b3a666310a4ced972acfe7f29e2c38
+size 52996277

config.json

@@ -0,0 +1,26 @@
+{
+  "model_type": "smartwatch_lm",
+  "vocab_size": 5533,
+  "n_layer": 6,
+  "n_head": 6,
+  "n_embd": 384,
+  "block_size": 256,
+  "dropout": 0.15,
+  "bias": false,
+  "onnx_input_names": [
+    "input_ids"
+  ],
+  "onnx_output_names": [
+    "logits"
+  ],
+  "onnx_opset": 17,
+  "onnx_file": "smartwatch_lm_merged.onnx",
+  "best_val_loss": 0.32432237863540647,
+  "data_files": [
+    "/content/tinydata.txt",
+    "/content/deepdata.txt",
+    "/content/tinydata1.txt",
+    "/content/data1.txt",
+    "/content/data3.txt"
+  ]
+}

config.py

@@ -0,0 +1,9 @@
+"""Generation defaults for the exported smartwatch LM."""
+
+from pathlib import Path
+
+OUTPUT_DIR = Path(__file__).parent
+
+SAMPLE_MAX_NEW_TOKENS = 120
+SAMPLE_TEMPERATURE = 0.8
+SAMPLE_TOP_K = 40

docs/avoiding-gibberish.md

@@ -0,0 +1,190 @@
+# Avoiding Gibberish — Output Cleanup Guide
+
+Smartwatch LM v0.1 is a **12M-parameter** domain model. It outputs structured replies with intent tags and slot placeholders. Raw tokenizer decode can still contain **BPE artifacts**, **encoding glitches**, or **run-on text**. This guide lists what to strip, how to truncate, and includes copy-paste scripts in this repo.
+
+**Scripts in this repo:**
+
+| File | Purpose |
+|------|---------|
+| [`reply_utils.py`](../reply_utils.py) | `clean_reply`, `extract_bot_reply`, `extract_intent_reply`, `fill_slots` |
+| [`onnx_sample.py`](../onnx_sample.py) | Full ONNX generate + cleanup pipeline |
+| [`chat.py`](../chat.py) | PyTorch REPL using the same helpers |
+
+---
+
+## Why gibberish appears
+
+| Cause | Example | Fix |
+|-------|---------|-----|
+| BPE space marker left in decode | `ĠYou'reĠat` | Replace `Ġ` → space |
+| BPE newline marker | `line oneĊline two` | Replace `Ċ` → newline |
+| UTF-8 mojibake | `âĢĶ` instead of `—` | Replace known bad sequences |
+| Model keeps generating | Fake next turn `\nuser: …` | Truncate at `\nuser:` |
+| Model rambling | Multiple paragraphs | Take first line only |
+| Broken intent tags | `< INTENT : GET_STEPS >` | Collapse spaces inside `<…>` |
+| Hallucinated metrics | `8432 steps` | Use `<STEPS_TODAY>` slots instead (training constraint) |
+
+---
+
+## Special characters to remove or replace
+
+Apply these **in order** after every `tokenizer.decode()`:
+
+### 1. BPE artifacts (always)
+
+| Character | Unicode | Replace with |
+|-----------|---------|--------------|
+| `Ġ` | U+0120 | space (` `) |
+| `Ċ` | U+010A | newline (`\n`) |
+
+### 2. Mojibake sequences (when present)
+
+| Bad sequence | Replace with |
+|--------------|--------------|
+| `âĢĶ` | `—` (em dash) |
+| `âĢĻ` | `'` |
+| `âĢĺ` | `'` |
+| `’` | `'` |
+| `–` | `—` |
+
+### 3. Whitespace normalization
+
+| Pattern | Action |
+|---------|--------|
+| Two or more spaces | Collapse to one space |
+| Space before apostrophe (` '`) | Remove the space → `'` |
+| Spaces inside angle brackets | Remove: `< STEP_GOAL >` → `<STEP_GOAL>` |
+
+### 4. Do **not** remove
+
+Keep these — they are part of the protocol:
+
+- `<INTENT:NAME>` tags
+- `<SLOT_NAME>` placeholders (filled by your app before display)
+- Normal punctuation: `. , ! ? ' —`
+
+---
+
+## Truncation rules (stop run-on output)
+
+After cleaning characters, cut the reply to **one bot utterance**:
+
+1. If `\nuser:` appears → discard everything from that point onward (model started a fake user turn).
+2. If `\n\n` appears → keep only the text before the blank line.
+3. Keep only the **first line** (everything before the first `\n`).
+
+These rules are implemented in `extract_bot_reply()` in [`reply_utils.py`](../reply_utils.py).
+
+---
+
+## Generation settings that reduce gibberish
+
+Use conservative sampling — this model is small (12M params):
+
+| Parameter | Recommended | Effect |
+|-----------|-------------|--------|
+| `temperature` | **0.5** (max 0.8) | Less random word choice |
+| `top_k` | **40** | Ignore unlikely tail tokens |
+| `max_new_tokens` | **40** | Stop before rambling |
+| `block_size` | **256** | Match model context; trim older history |
+| EOS token id | **0** | Stop generating after EOS (skip first 2 steps) |
+
+Store **unfilled** bot lines in history (`<STEPS_TODAY>` not `4,231`). Filled numbers in history confuse the model.
+
+---
+
+## Prompt format
+
+```
+user: How many steps today?
+bot: <INTENT:GET_STEPS> You're at <STEPS_TODAY> of <STEP_GOAL> — keep going!
+user: thanks
+bot:
+```
+
+Build with `build_prompt()` from [`reply_utils.py`](../reply_utils.py). The model continues after the final `bot:`.
+
+---
+
+## Sample: clean text only (no model)
+
+```bash
+python reply_utils.py
+```
+
+```python
+from reply_utils import clean_reply, extract_intent_reply, fill_slots
+
+messy = "Ġ<INTENT:GET_STEPS>ĠYou'reĠatĠ<STEPS_TODAY>ĠâĢĶĠnice!\nuser: more"
+
+parsed = extract_intent_reply(messy)
+display = fill_slots(parsed.template, {"STEPS_TODAY": "4,231"})
+
+print(parsed.intent)   # GET_STEPS
+print(display)         # You're at 4,231 — nice!
+```
+
+---
+
+## Sample: PyTorch chat with cleanup
+
+```bash
+pip install torch tokenizers
+python chat.py
+```
+
+```python
+from chat import ChatSession
+
+bot = ChatSession(temperature=0.5, max_new_tokens=40, top_k=40)
+print(bot.say("How many steps today?"))
+```
+
+---
+
+## Sample: ONNX with cleanup
+
+```bash
+pip install numpy onnxruntime tokenizers
+python onnx_sample.py "How many steps today?"
+```
+
+Pipeline inside `onnx_sample.py`:
+
+```
+build_prompt → encode → ONNX loop (top-k, temp 0.5) → decode new tokens only
+→ extract_bot_reply → extract_intent_reply → fill_slots → print
+```
+
+---
+
+## Full runtime checklist
+
+```
+1. build_prompt(history with raw unfilled bot lines, user_message)
+2. encode → generate (temp 0.5, top_k 40, max 40, EOS stop)
+3. decode ONLY new token ids (not the full prompt)
+4. clean_reply → extract_bot_reply → extract_intent_reply
+5. fill_slots(template, sensor_map) → show / speak
+6. append (user_message, raw_bot_line) to history
+```
+
+**Do not:**
+
+- Show raw `Ġ` or mojibake to the user
+- Put real sensor numbers back into conversation history
+- Skip truncation at `\nuser:` or first newline
+- Run temperature above 0.8 on this 12M model
+
+**Do:**
+
+- Run `clean_reply` on every decode path
+- Validate intent names against your handler allowlist
+- Trim history when approaching 256 tokens
+
+---
+
+## Related docs
+
+- [Intent reference](./intent-reference.md) — all 35 intents and slots
+- [Smartwatch integration](./smartwatch-integration.md) — device wiring

docs/intent-reference.md

@@ -0,0 +1,157 @@
+# Wrist Wearable Assistant — Intent Reference
+
+This document lists every intent the on-device model can emit. Your app parses these tags and runs the matching handler.
+
+## How intents work
+
+Bot lines in training data and at runtime use this format:
+
+```
+bot: <INTENT:GET_STEPS> You're at <STEPS_TODAY> of <STEP_GOAL> — keep going!
+```
+
+| Part | Role |
+|------|------|
+| `<INTENT:NAME>` | Tells the app **what to do** (fetch steps, start timer, etc.) |
+| `<SLOT>` placeholders | Names **which values** the app injects from sensors/settings |
+| Text after the tag | What the user **sees or hears** on the watch |
+
+The model never outputs real metric values — only intents and slot tokens. Your runtime replaces slots with live data.
+
+`NONE` is used when the bot only acknowledges or chats with no device action.
+
+---
+
+## Steps
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `GET_STEPS` | Read step metrics from the pedometer: today’s count, goals, streaks, week/month totals, comparisons. | `<STEPS_TODAY>`, `<STEPS_YESTERDAY>`, `<STEP_GOAL>`, `<STEPS_REMAINING>`, `<STEP_GOAL_PCT>`, `<STEP_STREAK_DAYS>`, `<STEPS_WEEK_TOTAL>`, `<STEPS_LAST_WEEK_TOTAL>`, `<STEPS_MONTH_TOTAL>`, `<TIME>`, `<DATE>`, `<PERCENT>` |
+| `SET_STEP_GOAL` | Change the user’s daily step target. | `<STEP_GOAL>` |
+
+---
+
+## Distance
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `GET_DISTANCE` | Read distance metrics: today’s total, last session, units, weekly progress, lifetime mileage, hourly average. | `<DISTANCE_TODAY>`, `<DISTANCE_LAST_SESSION>`, `<DISTANCE_UNIT>`, `<DISTANCE_WEEK_REMAINING>`, `<DISTANCE_WEEK_BEST>`, `<DISTANCE_LIFETIME>`, `<DISTANCE_HOURLY_AVG>`, `<TIME>`, `<DATE>` |
+
+---
+
+## Heart rate
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `GET_HEART_RATE` | Read heart rate data: current BPM, resting rate, overnight low, session average, daily peak, elevated minutes, effort zone. | `<HR_CURRENT_BPM>`, `<HR_RESTING_BPM>`, `<HR_RESTING_OVERNIGHT_LOW>`, `<HR_AVG_SESSION>`, `<HR_PEAK_TODAY>`, `<HR_ELEVATED_MINUTES>`, `<HR_ZONE>`, `<DURATION>` |
+| `MEASURE_HEART_RATE` | Trigger a live on-demand pulse reading from the optical sensor. | `<HR_CURRENT_BPM>` |
+
+---
+
+## Sleep
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `GET_SLEEP` | Read sleep data: duration last night, sleep/wake times, weekly average, awake minutes, goal gap, weekend total. | `<SLEEP_HOURS_LAST_NIGHT>`, `<SLEEP_START_TIME>`, `<SLEEP_WAKE_TIME>`, `<SLEEP_AVG_WEEK>`, `<SLEEP_AWAKE_MINUTES>`, `<SLEEP_GOAL_HOURS>`, `<SLEEP_WEEKEND_TOTAL>`, `<DATE>`, `<DURATION>` |
+| `LOG_NAP` | Record or confirm a short nap session. | `<DURATION>`, `<SLEEP_HOURS_LAST_NIGHT>`, `<TIME>` |
+
+---
+
+## Calories
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `GET_CALORIES` | Read calorie burn: daily total, active burn, remaining to goal, yesterday comparison, per-mile estimate. | `<CALORIES_TODAY>`, `<CALORIES_ACTIVE>`, `<CALORIES_REMAINING>`, `<CALORIE_GOAL>`, `<CALORIE_GOAL_PCT>`, `<CALORIES_YESTERDAY>`, `<CALORIES_PER_MILE>`, `<DISTANCE_UNIT>`, `<DURATION>` |
+| `SET_CALORIE_GOAL` | Change the user’s daily active calorie burn target. | `<CALORIE_GOAL>` |
+
+---
+
+## Active minutes
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `GET_ACTIVE_MINUTES` | Read movement time: today’s active minutes, remaining to goal, weekly total, most active day. | `<ACTIVE_MINUTES_TODAY>`, `<ACTIVE_MINUTES_REMAINING>`, `<ACTIVE_MINUTES_GOAL>`, `<ACTIVE_MINUTES_WEEK>`, `<MOST_ACTIVE_DAY>`, `<DURATION>`, `<TIME>`, `<DATE>` |
+| `SET_ACTIVE_GOAL` | Change the user’s daily active minutes target. | `<ACTIVE_MINUTES_GOAL>` |
+
+---
+
+## Battery
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `GET_BATTERY` | Read power state: percentage, estimated days left, days since charge, charge complete status. | `<BATTERY_PCT>`, `<BATTERY_DAYS_LEFT>`, `<DAYS_SINCE_CHARGE>`, `<CHARGE_COMPLETE>`, `<DURATION>`, `<DATE>` |
+| `ENABLE_POWER_SAVE` | Turn on low-power / battery saver mode on the device. | `<BATTERY_PCT>` |
+| `DISABLE_AOD` | Turn off always-on display to reduce power draw. | `<BATTERY_PCT>` |
+
+---
+
+## Alarms and reminders
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `SET_ALARM` | Create or update a wake alarm at a given time. | `<ALARM_TIME>`, `<ALARM_LABEL>`, `<DATE>`, `<TIME>` |
+| `LIST_ALARMS` | Show all alarms currently stored on the device. | `<ALARM_TIME>`, `<ALARM_LABEL>` |
+| `DELETE_ALARM` | Remove one or all alarms. | `<ALARM_LABEL>`, `<ALARM_TIME>` |
+| `SNOOZE_ALARM` | Snooze the currently firing alarm for a short period. | `<ALARM_TIME>`, `<DURATION>`, `<ALARM_LABEL>` |
+| `SET_REMINDER` | Schedule a recurring reminder (water, bedtime wind-down, move prompts). | `<ALARM_TIME>`, `<ALARM_LABEL>`, `<REMINDER_INTERVAL>`, `<TIME>` |
+| `MUTE_REMINDERS` | Disable hourly move reminders or similar nudges. | *(none)* |
+| `EXPLAIN_NUDGE` | Explain why the watch just vibrated (e.g. sedentary move nudge). | `<NUDGE_REASON>`, `<DURATION>`, `<TIME>` |
+
+---
+
+## Timers and stopwatch
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `START_TIMER` | Start a countdown timer (cooking, stretch, etc.). | `<DURATION>`, `<TIME>` |
+| `PAUSE_TIMER` | Pause an active countdown. | `<TIMER_REMAINING>`, `<DURATION>` |
+| `CANCEL_TIMER` | Stop and clear a running timer. | `<TIMER_REMAINING>`, `<DURATION>` |
+| `GET_TIMER_REMAINING` | Report time left on a timer or whether it has finished. | `<TIMER_REMAINING>`, `<DURATION>` |
+| `START_STOPWATCH` | Open or start the stopwatch app. | *(none)* |
+| `LAP_STOPWATCH` | Record a lap split on the stopwatch. | `<LAP_TIME>`, `<DURATION>` |
+| `RESET_STOPWATCH` | Reset stopwatch to zero. | `<DURATION>`, `<LAP_TIME>` |
+
+---
+
+## Workout toggles
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `START_WORKOUT` | Begin tracking a workout (walk, run, cycle, indoor cardio). | `<WORKOUT_TYPE>` |
+| `PAUSE_WORKOUT` | Pause the active workout session. | `<WORKOUT_ELAPSED>`, `<WORKOUT_STATE>`, `<WORKOUT_TYPE>` |
+| `RESUME_WORKOUT` | Resume a paused workout. | `<WORKOUT_ELAPSED>`, `<WORKOUT_STATE>`, `<WORKOUT_TYPE>` |
+| `STOP_WORKOUT` | End the workout and save the session. | `<WORKOUT_ELAPSED>`, `<WORKOUT_TYPE>`, `<WORKOUT_STATE>` |
+| `DISCARD_WORKOUT` | End the workout without saving data. | `<WORKOUT_TYPE>`, `<WORKOUT_STATE>` |
+| `GET_WORKOUT_STATUS` | Check if a workout is running, paused, or left on by mistake; report elapsed time. | `<WORKOUT_ELAPSED>`, `<WORKOUT_STATE>`, `<WORKOUT_TYPE>` |
+| `GET_WORKOUT_SUMMARY` | Show the post-workout summary screen (duration, distance, calories, HR). | `<WORKOUT_TYPE>`, `<WORKOUT_ELAPSED>`, `<DISTANCE_TODAY>`, `<DISTANCE_UNIT>`, `<CALORIES_ACTIVE>`, `<HR_AVG_SESSION>` |
+
+---
+
+## Non-action
+
+| Intent | What it does | Typical slots |
+|--------|--------------|---------------|
+| `NONE` | Pure conversational reply — thanks, encouragement, clarification — with no sensor fetch or device command. | *(none)* |
+
+---
+
+## Summary
+
+| Category | Intents | Count |
+|----------|---------|------:|
+| Steps | `GET_STEPS`, `SET_STEP_GOAL` | 2 |
+| Distance | `GET_DISTANCE` | 1 |
+| Heart rate | `GET_HEART_RATE`, `MEASURE_HEART_RATE` | 2 |
+| Sleep | `GET_SLEEP`, `LOG_NAP` | 2 |
+| Calories | `GET_CALORIES`, `SET_CALORIE_GOAL` | 2 |
+| Active minutes | `GET_ACTIVE_MINUTES`, `SET_ACTIVE_GOAL` | 2 |
+| Battery | `GET_BATTERY`, `ENABLE_POWER_SAVE`, `DISABLE_AOD` | 3 |
+| Alarms / reminders | `SET_ALARM`, `LIST_ALARMS`, `DELETE_ALARM`, `SNOOZE_ALARM`, `SET_REMINDER`, `MUTE_REMINDERS`, `EXPLAIN_NUDGE` | 7 |
+| Timers / stopwatch | `START_TIMER`, `PAUSE_TIMER`, `CANCEL_TIMER`, `GET_TIMER_REMAINING`, `START_STOPWATCH`, `LAP_STOPWATCH`, `RESET_STOPWATCH` | 7 |
+| Workout | `START_WORKOUT`, `PAUSE_WORKOUT`, `RESUME_WORKOUT`, `STOP_WORKOUT`, `DISCARD_WORKOUT`, `GET_WORKOUT_STATUS`, `GET_WORKOUT_SUMMARY` | 7 |
+| Non-action | `NONE` | 1 |
+| **Total** | | **35** |
+
+---
+
+Each wearable product may implement only a subset of these intents depending on hardware and firmware.

docs/smartwatch-integration.md

@@ -0,0 +1,195 @@
+# Smartwatch Integration Guide
+
+How to run **Smartwatch LM v0.1** on a wrist device and wire it to sensors, timers, and apps.
+
+The model is a **12M-parameter** GPT exported as ONNX. It does not execute device actions itself — it emits **intent tags** and **slot placeholders** that your firmware parses and handles.
+
+---
+
+## What you ship
+
+| File | Size (approx.) | Purpose |
+|------|----------------|---------|
+| `smartwatch_lm_merged.onnx` | ~52 MB | ONNX Runtime inference |
+| `tokenizer.json` | ~200 KB | Text ↔ token ids |
+| `tokenizer_config.json` | small | Tokenizer metadata |
+| `config.json` | small | Architecture and I/O names |
+| `reply_utils.py` | small | Cleanup, intent parse, slot fill |
+| `onnx_sample.py` | small | Reference ONNX generate loop |
+
+Optional on PC: `checkpoint.pt`, `chat.py`, `model.py` for PyTorch chat and fine-tuning.
+
+**ONNX I/O:**
+
+- Input: `input_ids` — int64, shape `[batch, seq]`, max seq **256**
+- Output: `logits` — float, shape `[batch, seq, vocab_size]` (vocab **3524**)
+
+Sample the **last position** logits autoregressively until EOS or max tokens.
+
+---
+
+## Architecture
+
+```
+User input (touch / voice-to-text)
+    → build_prompt()
+    → tokenizer.encode()
+    → ONNX generate loop
+    → tokenizer.decode (new tokens only)
+    → reply_utils.clean + extract_intent_reply
+    → intent router → sensor handlers
+    → fill_slots() → display / TTS
+```
+
+| Component | Role |
+|-----------|------|
+| LM | Intent + reply template with `<SLOT>` tokens |
+| Router | Map `<INTENT:NAME>` to a handler |
+| Handlers | Read/write device state |
+| Slot map | Live values for `<STEPS_TODAY>`, `<BATTERY_PCT>`, etc. |
+
+See [Intent reference](./intent-reference.md) for all 35 intents.
+
+---
+
+## Step 1 — Inference runtime
+
+| Platform | Runtime |
+|----------|---------|
+| Wear OS | ONNX Runtime Mobile (NNAPI / XNNPACK) |
+| watchOS | ORT Mobile or Core ML (consider INT8 quant) |
+| Companion phone | ORT on phone, BLE to watch for display |
+| Prototype | `python onnx_sample.py` on desktop |
+
+Budget ~52 MB weights + activations. Quantization or phone-side inference helps on tight RAM.
+
+---
+
+## Step 2 — Generation loop
+
+Recommended settings (see [Avoiding gibberish](./avoiding-gibberish.md)):
+
+```text
+temperature = 0.5
+top_k = 40
+max_new_tokens = 40
+block_size = 256
+eos_token_id = 0
+```
+
+Reference implementations:
+
+- ONNX: [`onnx_sample.py`](../onnx_sample.py)
+- PyTorch: [`model.py`](../model.py) `GPT.generate()` + [`chat.py`](../chat.py)
+
+---
+
+## Step 3 — Prompt and history
+
+```
+user: How many steps today?
+bot: <INTENT:GET_STEPS> You're at <STEPS_TODAY> of <STEP_GOAL> — keep going!
+user: Set a 10 minute timer
+bot:
+```
+
+Use `build_prompt()` from [`reply_utils.py`](../reply_utils.py).
+
+- History stores **raw** bot lines with unfilled slots
+- Never store display text with real numbers in history
+- Drop oldest turns when encoded length nears 256 tokens
+
+---
+
+## Step 4 — Cleanup and slots
+
+Always run the cleanup pipeline from [`reply_utils.py`](../reply_utils.py):
+
+1. `extract_bot_reply(prompt, generated)` — one line, no fake `\nuser:` tail
+2. `extract_intent_reply(raw)` → intent + template
+3. `fill_slots(template, slot_map)` → user-visible string
+
+Example slot map:
+
+```python
+{
+    "STEPS_TODAY": "4,231",
+    "STEP_GOAL": "10,000",
+    "STEPS_REMAINING": "5,769",
+    "BATTERY_PCT": "67%",
+    "TIME": "2:15 PM",
+}
+```
+
+Character cleanup (`Ġ`, mojibake, etc.) is documented in [Avoiding gibberish](./avoiding-gibberish.md).
+
+---
+
+## Step 5 — Intent router
+
+```python
+from reply_utils import fill_slots
+
+def dispatch(intent: str, template: str, slots: dict[str, str]) -> str:
+    filled = fill_slots(template, slots)
+    if intent == "GET_STEPS":
+        refresh_step_cache(slots)
+    elif intent == "START_TIMER":
+        timer.start(slots.get("DURATION", "5 minutes"))
+    elif intent == "NONE":
+        pass
+    else:
+        log.warning("unsupported intent: %s", intent)
+    return filled
+```
+
+Validate intents against your allowlist before side effects.
+
+---
+
+## Example trace
+
+**User:** “How many steps to hit my goal?”
+
+1. Prompt ends with `bot:`
+2. Model raw output: `<INTENT:GET_STEPS> You need <STEPS_REMAINING> more to reach <STEP_GOAL>.`
+3. Handler refreshes step slots from pedometer
+4. Display: `You need 5,769 more to reach 10,000.`
+5. History stores unfilled raw bot line
+
+---
+
+## Testing on desktop
+
+```bash
+pip install torch tokenizers
+python chat.py
+```
+
+```bash
+pip install numpy onnxruntime tokenizers
+python onnx_sample.py "How many steps today?"
+```
+
+```bash
+python reply_utils.py   # cleanup demo, no model
+```
+
+---
+
+## Production checklist
+
+- [ ] `smartwatch_lm_merged.onnx` + `tokenizer.json` bundled or downloaded once
+- [ ] Generation: temp 0.5, top_k 40, max 40 tokens
+- [ ] `reply_utils` cleanup on every decode path
+- [ ] History uses unfilled slot templates only
+- [ ] Intent allowlist matches implemented handlers
+- [ ] Slot map from real sensors before `fill_slots`
+- [ ] Fallback when parse fails or intent is unknown
+
+---
+
+## Related docs
+
+- [Avoiding gibberish](./avoiding-gibberish.md) — special characters, truncation, sample scripts
+- [Intent reference](./intent-reference.md) — all intents and slots

model.py

@@ -0,0 +1,182 @@
+"""GPT model definition and checkpoint loading for exported smartwatch LM."""
+
+from __future__ import annotations
+
+import math
+from pathlib import Path
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from tokenizers import Tokenizer
+
+import config as cfg
+
+
+class CausalSelfAttention(nn.Module):
+    def __init__(self, n_head: int, n_embd: int, block_size: int, dropout: float, bias: bool):
+        super().__init__()
+        assert n_embd % n_head == 0
+        self.n_head = n_head
+        self.n_embd = n_embd
+        self.head_dim = n_embd // n_head
+        self.c_attn = nn.Linear(n_embd, 3 * n_embd, bias=bias)
+        self.c_proj = nn.Linear(n_embd, n_embd, bias=bias)
+        self.attn_dropout = nn.Dropout(dropout)
+        self.resid_dropout = nn.Dropout(dropout)
+        self.register_buffer(
+            "bias",
+            torch.tril(torch.ones(block_size, block_size)).view(1, 1, block_size, block_size),
+        )
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        b, t, c = x.size()
+        q, k, v = self.c_attn(x).split(self.n_embd, dim=2)
+        k = k.view(b, t, self.n_head, self.head_dim).transpose(1, 2)
+        q = q.view(b, t, self.n_head, self.head_dim).transpose(1, 2)
+        v = v.view(b, t, self.n_head, self.head_dim).transpose(1, 2)
+        att = (q @ k.transpose(-2, -1)) * (1.0 / math.sqrt(self.head_dim))
+        att = att.masked_fill(self.bias[:, :, :t, :t] == 0, float("-inf"))
+        att = F.softmax(att, dim=-1)
+        att = self.attn_dropout(att)
+        y = att @ v
+        y = y.transpose(1, 2).contiguous().view(b, t, c)
+        return self.resid_dropout(self.c_proj(y))
+
+
+class MLP(nn.Module):
+    def __init__(self, n_embd: int, dropout: float, bias: bool):
+        super().__init__()
+        self.c_fc = nn.Linear(n_embd, 4 * n_embd, bias=bias)
+        self.gelu = nn.GELU()
+        self.c_proj = nn.Linear(4 * n_embd, n_embd, bias=bias)
+        self.dropout = nn.Dropout(dropout)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        return self.dropout(self.c_proj(self.gelu(self.c_fc(x))))
+
+
+class Block(nn.Module):
+    def __init__(self, n_head: int, n_embd: int, block_size: int, dropout: float, bias: bool):
+        super().__init__()
+        self.ln1 = nn.LayerNorm(n_embd)
+        self.attn = CausalSelfAttention(n_head, n_embd, block_size, dropout, bias)
+        self.ln2 = nn.LayerNorm(n_embd)
+        self.mlp = MLP(n_embd, dropout, bias)
+
+    def forward(self, x: torch.Tensor) -> torch.Tensor:
+        x = x + self.attn(self.ln1(x))
+        x = x + self.mlp(self.ln2(x))
+        return x
+
+
+class GPT(nn.Module):
+    def __init__(
+        self,
+        vocab_size: int,
+        n_layer: int,
+        n_head: int,
+        n_embd: int,
+        block_size: int,
+        dropout: float,
+        bias: bool,
+    ):
+        super().__init__()
+        self.block_size = block_size
+        self.transformer = nn.ModuleDict(
+            {
+                "wte": nn.Embedding(vocab_size, n_embd),
+                "wpe": nn.Embedding(block_size, n_embd),
+                "drop": nn.Dropout(dropout),
+                "h": nn.ModuleList(
+                    [Block(n_head, n_embd, block_size, dropout, bias) for _ in range(n_layer)]
+                ),
+                "ln_f": nn.LayerNorm(n_embd),
+            }
+        )
+        self.lm_head = nn.Linear(n_embd, vocab_size, bias=False)
+        self.transformer.wte.weight = self.lm_head.weight
+        self.apply(self._init_weights)
+
+    def _init_weights(self, module: nn.Module) -> None:
+        if isinstance(module, nn.Linear):
+            torch.nn.init.normal_(module.weight, mean=0.0, std=0.02)
+            if module.bias is not None:
+                torch.nn.init.zeros_(module.bias)
+        elif isinstance(module, nn.Embedding):
+            torch.nn.init.normal_(module.weight, mean=0.0, std=0.02)
+
+    def forward(self, idx: torch.Tensor, targets=None):
+        b, t = idx.size()
+        assert t <= self.block_size
+        pos = torch.arange(0, t, dtype=torch.long, device=idx.device)
+        x = self.transformer.drop(
+            self.transformer.wte(idx) + self.transformer.wpe(pos)
+        )
+        for block in self.transformer.h:
+            x = block(x)
+        x = self.transformer.ln_f(x)
+        logits = self.lm_head(x)
+        loss = None
+        if targets is not None:
+            loss = F.cross_entropy(logits.view(-1, logits.size(-1)), targets.view(-1))
+        return logits, loss
+
+    @torch.no_grad()
+    def generate(self, idx: torch.Tensor, max_new_tokens: int, temperature: float = 1.0, top_k=None):
+        for _ in range(max_new_tokens):
+            idx_cond = idx[:, -self.block_size :]
+            logits, _ = self(idx_cond)
+            logits = logits[:, -1, :] / max(temperature, 1e-8)
+            if top_k is not None:
+                v, _ = torch.topk(logits, min(top_k, logits.size(-1)))
+                logits[logits < v[:, [-1]]] = -float("Inf")
+            probs = F.softmax(logits, dim=-1)
+            idx_next = torch.multinomial(probs, num_samples=1)
+            idx = torch.cat((idx, idx_next), dim=1)
+        return idx
+
+
+def resolve_checkpoint_paths(
+    checkpoint_path: Path | None = None,
+    tokenizer_path: Path | None = None,
+) -> tuple[Path, Path]:
+    ckpt = checkpoint_path or cfg.OUTPUT_DIR / "checkpoint.pt"
+    tok = tokenizer_path or cfg.OUTPUT_DIR / "tokenizer.json"
+    if not ckpt.is_file():
+        raise FileNotFoundError(
+            f"Checkpoint not found at {ckpt}. Train first, then run collab-run-2/export_model.py."
+        )
+    if not tok.is_file():
+        raise FileNotFoundError(
+            f"Tokenizer not found at {tok}. Train first, then run collab-run-2/export_model.py."
+        )
+    return ckpt, tok
+
+
+def load_model(
+    checkpoint_path: Path | None = None,
+    tokenizer_path: Path | None = None,
+    device: str | None = None,
+) -> tuple[GPT, Tokenizer, str]:
+    ckpt_path, tok_path = resolve_checkpoint_paths(checkpoint_path, tokenizer_path)
+    dev = device or ("cuda" if torch.cuda.is_available() else "cpu")
+
+    tokenizer = Tokenizer.from_file(str(tok_path))
+    checkpoint = torch.load(ckpt_path, map_location=dev, weights_only=False)
+    model_config = checkpoint["model_config"]
+
+    model = GPT(
+        vocab_size=model_config["vocab_size"],
+        n_layer=model_config["n_layer"],
+        n_head=model_config["n_head"],
+        n_embd=model_config["n_embd"],
+        block_size=model_config["block_size"],
+        dropout=model_config["dropout"],
+        bias=model_config["bias"],
+    )
+    model.load_state_dict(checkpoint["model_state_dict"])
+    model.to(dev)
+    model.eval()
+
+    return model, tokenizer, dev

onnx_sample.py

@@ -0,0 +1,95 @@
+"""
+Minimal ONNX inference sample for the exported Smartwatch LM.
+
+Requirements:
+    pip install numpy onnxruntime tokenizers
+
+Usage:
+    python onnx_sample.py "How many steps today?"
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+import numpy as np
+import onnxruntime as ort
+from tokenizers import Tokenizer
+
+from reply_utils import build_prompt, process_model_output
+
+MODEL_DIR = Path(__file__).parent
+ONNX_PATH = MODEL_DIR / "smartwatch_lm_merged.onnx"
+TOKENIZER_PATH = MODEL_DIR / "tokenizer.json"
+
+BLOCK_SIZE = 256
+MAX_NEW_TOKENS = 40
+TEMPERATURE = 0.5
+TOP_K = 40
+EOS_TOKEN_ID = 0
+
+
+def sample_next_token(logits: np.ndarray, temperature: float, top_k: int) -> int:
+    scaled = logits / max(temperature, 1e-8)
+    scaled = scaled - scaled.max()
+    probs = np.exp(scaled)
+    probs /= probs.sum()
+    k = min(top_k, probs.size)
+    top_idx = np.argpartition(probs, -k)[-k:]
+    top_probs = probs[top_idx]
+    top_probs /= top_probs.sum()
+    return int(np.random.choice(top_idx, p=top_probs))
+
+
+def generate(session: ort.InferenceSession, tokenizer: Tokenizer, prompt: str) -> str:
+    ids = tokenizer.encode(prompt).ids
+    prompt_len = len(ids)
+
+    for step in range(MAX_NEW_TOKENS):
+        seq = ids[-BLOCK_SIZE:]
+        x = np.array([seq], dtype=np.int64)
+        logits = session.run(None, {"input_ids": x})[0]
+        next_logits = logits[0, -1, :]
+        next_id = sample_next_token(next_logits, TEMPERATURE, TOP_K)
+        ids.append(next_id)
+        if step > 2 and next_id == EOS_TOKEN_ID:
+            break
+
+    return tokenizer.decode(ids[prompt_len:])
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="ONNX chat sample")
+    parser.add_argument("message", nargs="?", default="How many steps today?")
+    args = parser.parse_args()
+
+    if not ONNX_PATH.is_file():
+        print(f"Missing {ONNX_PATH.name}", file=sys.stderr)
+        sys.exit(1)
+    if not TOKENIZER_PATH.is_file():
+        print(f"Missing {TOKENIZER_PATH.name}", file=sys.stderr)
+        sys.exit(1)
+
+    session = ort.InferenceSession(str(ONNX_PATH), providers=["CPUExecutionProvider"])
+    tokenizer = Tokenizer.from_file(str(TOKENIZER_PATH))
+
+    prompt = build_prompt([], args.message)
+    continuation = generate(session, tokenizer, prompt)
+
+    slot_data = {
+        "STEPS_TODAY": "4,231",
+        "STEP_GOAL": "10,000",
+        "STEPS_REMAINING": "5,769",
+    }
+    raw, parsed, display = process_model_output(prompt, continuation, slot_data)
+
+    print(f"user> {args.message}")
+    print(f"raw>  {raw}")
+    print(f"intent> {parsed.intent}")
+    print(f"bot>  {display}")
+
+
+if __name__ == "__main__":
+    main()

reply_utils.py

@@ -0,0 +1,124 @@
+"""
+Post-processing for Smartwatch LM chat replies (BPE gibberish removal).
+
+Use after tokenizer.decode() — same logic is embedded in colab_all_in_one.py.
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+
+_BPE_SPACE = "\u0120"  # Ġ
+_BPE_NEWLINE = "\u010a"  # Ċ
+
+_MOJIBAKE_REPLACEMENTS: tuple[tuple[str, str], ...] = (
+    ("âĢĶ", "—"),
+    ("âĢĻ", "'"),
+    ("âĢĺ", "'"),
+    ("’", "'"),
+    ("–", "—"),
+)
+
+
+def build_prompt(history: list[tuple[str, str]], user_message: str) -> str:
+    lines: list[str] = []
+    for user_text, bot_text in history:
+        lines.append(f"user: {user_text}")
+        lines.append(f"bot: {bot_text}")
+    lines.append(f"user: {user_message}")
+    lines.append("bot:")
+    return "\n".join(lines)
+
+
+def _compact_tag(match: re.Match[str]) -> str:
+    inner = re.sub(r"\s+", "", match.group(1))
+    return f"<{inner}>"
+
+
+def clean_reply(text: str) -> str:
+    """Remove ByteLevel BPE artifacts (Ġ Ċ) and fix broken punctuation."""
+    out = text.replace(_BPE_SPACE, " ").replace(_BPE_NEWLINE, "\n")
+    for bad, good in _MOJIBAKE_REPLACEMENTS:
+        out = out.replace(bad, good)
+    out = re.sub(r" +", " ", out)
+    out = re.sub(r"<\s*([^>]+?)\s*>", _compact_tag, out)
+    return out.replace(" '", "'").strip()
+
+
+def _first_bot_line(text: str) -> str:
+    """Keep only the first bot utterance; drop hallucinated user turns."""
+    text = clean_reply(text.lstrip())
+    if re.match(r"^\s*user\s*:", text, re.IGNORECASE):
+        match = re.search(r"bot\s*:\s*(.+)", text, re.IGNORECASE | re.DOTALL)
+        if match:
+            text = match.group(1)
+        else:
+            return ""
+    text = re.sub(r"^\s*bot\s*:\s*", "", text, count=1, flags=re.IGNORECASE)
+    text = re.split(r"\n\s*user\s*:", text, maxsplit=1, flags=re.IGNORECASE)[0]
+    if "\n\n" in text:
+        text = text.split("\n\n", 1)[0]
+    return clean_reply(text.split("\n", 1)[0].strip())
+
+
+def extract_bot_reply(prompt: str, generated: str) -> str:
+    """Strip prompt prefix and return one cleaned bot line."""
+    marker = prompt.rstrip() + " "
+    if generated.startswith(marker):
+        reply = generated[len(marker) :]
+    elif re.search(r"bot\s*:", generated, re.IGNORECASE):
+        reply = re.split(r"bot\s*:", generated, maxsplit=0, flags=re.IGNORECASE)[-1]
+    else:
+        reply = generated
+    return _first_bot_line(reply)
+
+
+def extract_bot_reply_from_continuation(continuation: str) -> str:
+    """Decode only new tokens, then extract the first bot line."""
+    return _first_bot_line(continuation)
+
+
+@dataclass
+class ParsedReply:
+    intent: str
+    template: str
+
+
+def extract_intent_reply(text: str) -> ParsedReply:
+    cleaned = clean_reply(text)
+    match = re.search(r"<\s*INTENT\s*:[^>]+>", cleaned, re.IGNORECASE)
+    if not match:
+        first = cleaned.split("\n", 1)[0].strip()
+        return ParsedReply(intent="NONE", template=first or cleaned)
+
+    rest = cleaned[match.start() :]
+    rest = re.split(r"\nuser\s*:", rest, maxsplit=1, flags=re.IGNORECASE)[0]
+    line = rest.split("\n", 1)[0].strip()
+
+    intent_match = re.match(r"^<INTENT:([A-Z_]+)>\s*(.*)", line, re.IGNORECASE | re.DOTALL)
+    if intent_match:
+        return ParsedReply(intent=intent_match.group(1), template=intent_match.group(2).strip())
+    return ParsedReply(intent="NONE", template=line)
+
+
+def fill_slots(text: str, data: dict[str, str]) -> str:
+    return re.sub(
+        r"<([A-Z_]+)>",
+        lambda m: data.get(m.group(1), m.group(0)),
+        text,
+    )
+
+
+def process_model_output(
+    prompt: str,
+    generated: str,
+    slot_data: dict[str, str] | None = None,
+) -> tuple[str, ParsedReply, str]:
+    """Raw continuation -> cleaned bot line -> intent parse -> slot-filled display."""
+    raw = extract_bot_reply_from_continuation(generated)
+    if not raw:
+        raw = extract_bot_reply(prompt, generated)
+    parsed = extract_intent_reply(raw)
+    display = fill_slots(parsed.template, slot_data or {})
+    return raw, parsed, display

smartwatch_lm_merged.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae31bdd84cf86fe027e19ce17677d04d9f9613d6a109acdfb01899cf431433c3
+size 60231131

tokenizer_config.json

@@ -0,0 +1,4 @@
+{
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "model_max_length": 256
+}