mlboydaisuke commited on
Commit
5b6b488
·
verified ·
1 Parent(s): 4664158

android/: the Android sample app (model picker, hold to talk, three clips, device check); card: the Sample app line with the Galaxy S26 numbers

Browse files
.gitattributes CHANGED
@@ -36,3 +36,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
36
  Qwen3-ASR-1.7B.litertlm filter=lfs diff=lfs merge=lfs -text
37
  assets/demo_s26.png filter=lfs diff=lfs merge=lfs -text
38
  assets/demo_s26.mp4 filter=lfs diff=lfs merge=lfs -text
 
 
 
 
36
  Qwen3-ASR-1.7B.litertlm filter=lfs diff=lfs merge=lfs -text
37
  assets/demo_s26.png filter=lfs diff=lfs merge=lfs -text
38
  assets/demo_s26.mp4 filter=lfs diff=lfs merge=lfs -text
39
+ android/app/src/main/res/raw/fleurs_cmn_1698.wav filter=lfs diff=lfs merge=lfs -text
40
+ android/app/src/main/res/raw/fleurs_en_1698.wav filter=lfs diff=lfs merge=lfs -text
41
+ android/app/src/main/res/raw/fleurs_ja_1698.wav filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -153,6 +153,8 @@ print(raw.partition("<asr_text>")[2].strip())
153
 
154
  For the Chinese example clip `raw` is `language Chinese<asr_text>开放时间:早上九点至下午五点。` and the script prints the text after the tag. `backend=GPU()` (from `litert_lm.interfaces`) puts the language model on the GPU. Keep `audio_backend=CPU()`: the audio encoder does not run on the GPU delegate (`create_conversation` fails). The Kotlin Conversation API takes the same calls. In the Galaxy S26 app above (`litertlm-android` 0.17.1, CPU), the three clips gave the same answers, rendered prompts and token counts as the Python API on the Mac (0.17.1, CPU) in all 5 app runs. The Swift API was not run for this file.
155
 
 
 
156
  On Android the S26 rows above came from the command-line tool: `litert_lm_advanced_main --backend=cpu --audio_backend=cpu --sampler_backend=cpu --num_cpu_threads=4 --model_path=Qwen3-ASR-1.7B.litertlm --cache_dir=cache --max_num_tokens=1024 --max_output_tokens=256 --benchmark --input_prompt='[audio:/path/clip.wav]'`.
157
 
158
  ## License
 
153
 
154
  For the Chinese example clip `raw` is `language Chinese<asr_text>开放时间:早上九点至下午五点。` and the script prints the text after the tag. `backend=GPU()` (from `litert_lm.interfaces`) puts the language model on the GPU. Keep `audio_backend=CPU()`: the audio encoder does not run on the GPU delegate (`create_conversation` fails). The Kotlin Conversation API takes the same calls. In the Galaxy S26 app above (`litertlm-android` 0.17.1, CPU), the three clips gave the same answers, rendered prompts and token counts as the Python API on the Mac (0.17.1, CPU) in all 5 app runs. The Swift API was not run for this file.
155
 
156
+ Sample app (Android): [`android/`](https://huggingface.co/litert-community/Qwen3-ASR-1.7B/tree/main/android) — one app for this file and two sibling bundles (Qwen3-ASR-1.7B, Fun-ASR-Nano-2512, Confucius4-R2T2): a model picker, hold-to-talk up to 30 s and three FLEURS clips, with a device check. Galaxy S26, language model and audio encoder on the CPU (4 threads), the 6.4 s English clip in 2.49 s and the 9.7 s Mandarin clip in 2.38 s (2026-10-09, CPU at 3.51 / 4.19 GHz); a 9 s microphone recording in 2.97 s with no character error, measured by the app's device check on 2026-10-09.
157
+
158
  On Android the S26 rows above came from the command-line tool: `litert_lm_advanced_main --backend=cpu --audio_backend=cpu --sampler_backend=cpu --num_cpu_threads=4 --model_path=Qwen3-ASR-1.7B.litertlm --cache_dir=cache --max_num_tokens=1024 --max_output_tokens=256 --benchmark --input_prompt='[audio:/path/clip.wav]'`.
159
 
160
  ## License
android/.gitignore ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ .gradle/
2
+ .kotlin/
3
+ build/
4
+ app/build/
5
+ local.properties
6
+ .gradle-user-home/
7
+ # The .litertlm bundles are pushed to the phone (see README), never committed.
8
+ *.litertlm
9
+ # hf download --local-dir in the README writes the bundles (and its own metadata) here.
10
+ bundles/
android/LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Daisuke Majima
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
android/README.md ADDED
@@ -0,0 +1,189 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ASR LiteRT-LM — speech to text on Android with three LiteRT-LM bundles
2
+
3
+ Hold the button, speak for up to 30 s, and the app writes what you said. On the CPU of a Galaxy S26
4
+ (4 threads, measured 2026-10-09), Qwen3-ASR-1.7B turned a 6.4 s English clip into text in 2.49 s and
5
+ Fun-ASR-Nano-2512 in 1.19 s. The app runs three speech recognition models in the LiteRT-LM `.litertlm`
6
+ format through one code path: pick a model, then talk or play one of three bundled clips. Everything
7
+ runs on the phone; the app asks for the microphone and nothing else (no network permission).
8
+
9
+ ## Models
10
+
11
+ | Model | Hugging Face | File | Size | License | Languages |
12
+ |---|---|---|---|---|---|
13
+ | Qwen3-ASR-1.7B | [litert-community/Qwen3-ASR-1.7B](https://huggingface.co/litert-community/Qwen3-ASR-1.7B) | `Qwen3-ASR-1.7B.litertlm` | 2,693,064,192 B | Apache-2.0 | 30 languages (the card scores English, Mandarin, Japanese) |
14
+ | Fun-ASR-Nano-2512 | [litert-community/Fun-ASR-Nano-2512](https://huggingface.co/litert-community/Fun-ASR-Nano-2512) | `Fun-ASR-Nano-2512.litertlm` | 1,255,894,736 B | Apache-2.0 | Chinese, English, Japanese |
15
+ | Confucius4-R2T2 | [mlboydaisuke/Confucius4-R2T2-LiteRT](https://huggingface.co/mlboydaisuke/Confucius4-R2T2-LiteRT) | `Confucius4-R2T2.litertlm` | 2,693,064,192 B | [NetEase Youdao Model Use License Agreement](https://huggingface.co/mlboydaisuke/Confucius4-R2T2-LiteRT/blob/main/MODEL_LICENSE) | Chinese, English; a fine-tune of Qwen3-ASR-1.7B |
16
+
17
+ Each file holds the audio encoder, the language model, the tokenizer and the prompt template. Read
18
+ the Confucius4-R2T2 agreement before you use that file: it has user-count and revenue thresholds
19
+ (clause 2.2) and limits on using the model to improve other models (clause 3.4).
20
+
21
+ ## Run it in 5 lines
22
+
23
+ From this directory, with the phone connected over adb:
24
+
25
+ ```bash
26
+ hf download litert-community/Qwen3-ASR-1.7B Qwen3-ASR-1.7B.litertlm --local-dir bundles
27
+ hf download litert-community/Fun-ASR-Nano-2512 Fun-ASR-Nano-2512.litertlm --local-dir bundles
28
+ hf download mlboydaisuke/Confucius4-R2T2-LiteRT Confucius4-R2T2.litertlm --local-dir bundles
29
+ ./gradlew :app:installDebug && adb shell am start -n com.asrlitertlm/.MainActivity
30
+ adb push bundles/*.litertlm /sdcard/Android/data/com.asrlitertlm/files/
31
+ ```
32
+
33
+ Reopen the app, tap a model in the picker and hold **Hold to talk**. The first launch creates the
34
+ folder the push writes to. One bundle is enough; the picker marks the others `missing`.
35
+
36
+ - Storage: the three files take 6.6 GB. The first load of each file writes a runtime cache into the
37
+ app's cache directory: 3.0 GB for each Qwen3-ASR file, 1.5 GB for Fun-ASR.
38
+ - Run `adb shell touch /sdcard/Android/data/com.asrlitertlm/files/*.litertlm` after the push. adb push
39
+ writes whole-second modification times, and with those a later launch can miss the runtime cache
40
+ and write it again ([LiteRT-LM #3772](https://github.com/google-ai-edge/LiteRT-LM/issues/3772)).
41
+
42
+ ## Galaxy S26 numbers
43
+
44
+ Galaxy S26 (SM-S942Q, Android 16), `litertlm-android` 0.17.1, language model and audio encoder on the
45
+ CPU with 4 threads, 2026-10-09. "Transcribe" is the wall time of `sendMessage()`, from sending the
46
+ clip to the full answer; RTF is that time over the clip's length. The three clips are the same sentence
47
+ in three languages (below).
48
+
49
+ | Model | zh, 9.7 s | en, 6.4 s | ja, 11.0 s | Microphone, 9 s | CPU limit at the start (big / prime cores) |
50
+ |---|---|---|---|---|---|
51
+ | Qwen3-ASR-1.7B | 2.38 s, RTF 0.25 | 2.49 s, RTF 0.39 | 3.56 s, RTF 0.32 | 2.97 s, CER 0 | 3.51 / 4.19 GHz |
52
+ | Confucius4-R2T2 | 3.48 s, RTF 0.36 | 2.59 s, RTF 0.40 | 3.54 s, RTF 0.32 | 3.17 s, CER 0 | 3.63 / 4.74 GHz (no limit) |
53
+ | Fun-ASR-Nano-2512 | 1.95 s, RTF 0.20 | 1.19 s, RTF 0.19 | 1.83 s, RTF 0.17 | 1.44 s, CER 0 | 3.63 / 4.74 GHz (no limit) |
54
+
55
+ | Model | Engine load, first (writes the cache) | Engine load, cache in place | Runtime cache | Peak resident memory, first load |
56
+ |---|---|---|---|---|
57
+ | Qwen3-ASR-1.7B | 7.35 s | 0.72 s | 3.0 GB | 5.6 GB |
58
+ | Confucius4-R2T2 | 4.27 s | 1.33 s | 3.0 GB | 5.7 GB |
59
+ | Fun-ASR-Nano-2512 | 1.39 s | 1.13 s | 1.5 GB | 3.5 GB |
60
+
61
+ - Every clip's text equals the Mac CPU answer of the same file (LiteRT-LM 0.17.1, Python) after
62
+ removing punctuation and spaces. The three models write the same sentences for these clips.
63
+ - The last column is the CPU limit at the start of the run (the maximum is 3.63 / 4.74 GHz; a warm phone
64
+ lowers it). The Confucius4-R2T2 and Fun-ASR rows are a second run of the check on the same day with no
65
+ limit (Confucius4-R2T2 at thermal status 0, Fun-ASR at status 1, each file pushed alone). The first run,
66
+ with the limit lowered by heat, gave Confucius4-R2T2 4.03 / 3.06 / 3.83 s at 2.75 / 2.67 GHz and
67
+ Fun-ASR 1.26 / 1.60 / 2.29 s at 2.23 / 2.23 GHz. Confucius4-R2T2 has the same architecture and file
68
+ size as Qwen3-ASR-1.7B. In a run that started at the maximum, Qwen3-ASR-1.7B took 2.13 / 2.14 / 3.19 s
69
+ for the three clips.
70
+ - "Engine load" is `Engine.initialize()` plus the first conversation, which creates the audio encoder.
71
+ The Qwen3-ASR-1.7B cached load is a new launch; the other two are a second load in the same process.
72
+ - Microphone: the phone plays clip en through its speaker and records it with its microphone
73
+ (media volume 3 of 15), then transcribes the recording. CER is the character error rate against
74
+ the clip's sentence.
75
+ - Language model on the GPU (launch extra `--es backend gpu`), Fun-ASR-Nano-2512: 1.69 / 1.68 / 2.29 s,
76
+ the same text. All nodes of the prefill and decode graphs ran on the GPU (LITERT_CL); the audio
77
+ encoder stays on the CPU (the model cards: the GPU delegate does not take it). Engine load 2.61 s.
78
+ It was not faster than the CPU here.
79
+
80
+ ## How the app calls a model
81
+
82
+ The three bundles share one path ([`AsrEngine.kt`](app/src/main/kotlin/com/asrlitertlm/AsrEngine.kt)):
83
+ one `Engine`, a new `Conversation` per clip, and one user message that holds only the audio file. Each
84
+ bundle's template adds the rest: Fun-ASR's default instruction `语音转写:`, Qwen3-ASR's audio markers.
85
+
86
+ ```kotlin
87
+ val engine = Engine(EngineConfig(
88
+ modelPath = bundle.path, // /sdcard/Android/data/com.asrlitertlm/files/<file>
89
+ backend = Backend.CPU(threadCount = 4), // the language model
90
+ audioBackend = Backend.CPU(threadCount = 4), // the audio encoder
91
+ cacheDir = context.cacheDir.path,
92
+ ))
93
+ engine.initialize()
94
+
95
+ val raw = engine.createConversation(ConversationConfig(
96
+ samplerConfig = SamplerConfig(topK = 1, topP = 1.0, temperature = 0.0),
97
+ maxOutputToken = 512,
98
+ )).use { conversation ->
99
+ conversation.sendMessage(Message.user(Contents.of(Content.AudioFile(wav.absolutePath)))).toString()
100
+ }
101
+ // Qwen3-ASR-1.7B and Confucius4-R2T2: "language English<asr_text>Everything in the universe ..."
102
+ // Fun-ASR-Nano-2512: "Everything in the universe ..."
103
+ ```
104
+
105
+ Only the answer differs: [`ModelProfile.kt`](app/src/main/kotlin/com/asrlitertlm/ModelProfile.kt)
106
+ splits the Qwen3-ASR answer at `<asr_text>` and shows the named language; Fun-ASR's answer is the text.
107
+
108
+ - One file is loaded at a time: picking another model closes the engine first. A first load of a
109
+ Qwen3-ASR file peaked at 5.6–5.7 GB resident (VmHWM, the memory-mapped file included) on this phone
110
+ with 11.4 GB of memory.
111
+ - `Engine.close()` throws when called twice, so the app drops its reference before closing; releasing
112
+ twice is harmless.
113
+ - All model calls run on one worker thread.
114
+
115
+ ## Microphone
116
+
117
+ Hold **Hold to talk** and speak; releasing the button ends the recording, and so does the 30 s limit.
118
+ Each model reads one window of about 30 s per message, so the app sends at most 30 s. The app records
119
+ 16 kHz mono 16-bit PCM (the `VOICE_RECOGNITION` source) and writes it as a WAV file for the runtime.
120
+ A recording quieter than −60 dBFS RMS is reported as "No speech heard" and is not sent to the model.
121
+ Without the microphone permission the app shows a sentence that says how to allow it; the clips still work.
122
+
123
+ ## The three clips
124
+
125
+ `app/src/main/res/raw/` holds FLEURS test sentence 1698 read in Mandarin, English and Japanese
126
+ ([google/fleurs](https://huggingface.co/datasets/google/fleurs) revision `70bb2e84`, test split, id 1698;
127
+ FLEURS, Conneau et al. 2022, licensed [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)). Changes:
128
+ converted to 16-bit PCM and given one linear gain each, so that all three have an active RMS of −20 dBFS.
129
+ The English reading is "Everything in the Universe is made of matter. All matter is made of tiny
130
+ particles called atoms."
131
+
132
+ ## Device check
133
+
134
+ [`AsrDeviceCheck.kt`](app/src/androidTest/kotlin/com/asrlitertlm/AsrDeviceCheck.kt) runs on a phone
135
+ with at least one bundle pushed (as above). Turn the media volume up first: the microphone step plays a
136
+ clip through the speaker.
137
+
138
+ ```bash
139
+ ./gradlew :app:installDebug :app:assembleDebugAndroidTest
140
+ adb install -r -t app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk
141
+ adb shell pm revoke com.asrlitertlm android.permission.RECORD_AUDIO
142
+ adb shell am instrument -w -e class com.asrlitertlm.AsrDeviceCheck#check com.asrlitertlm.test/androidx.test.runner.AndroidJUnitRunner
143
+ adb logcat -d -s asr-check | grep RESULT
144
+ ```
145
+
146
+ It prints one line per step, `RESULT step=<name> ok=<true|false> model=<id> ...`, then
147
+ `RESULT ok=<all>`. It starts with the microphone permission revoked and checks the sentence the app
148
+ shows; then it grants the permission. For each bundle on the phone: load, the three clips (text equal to
149
+ the expected sentence after removing punctuation and spaces), the microphone (CER ≤ 0.15), release
150
+ twice, and a second load. Then a 33.6 s clip (cut to 30 s), a silent recording and a file that is not
151
+ on the phone. With two or more bundles it also switches models without releasing. Add
152
+ `-e class com.asrlitertlm.AsrDeviceCheck#gpu -e gpu_model fun-asr-nano-2512` for the GPU run.
153
+
154
+ ## Launch extras
155
+
156
+ For recordings and tests; the app works without them.
157
+
158
+ | Extra | Effect |
159
+ |---|---|
160
+ | `--es model <id>` | `qwen3-asr-1.7b`, `fun-asr-nano-2512` or `confucius4-r2t2` (default: the model used last) |
161
+ | `--es backend gpu` | language model on the GPU; the audio encoder stays on the CPU |
162
+ | `--ez autoplay true` | after loading: clips zh, en, ja, each played and then transcribed (`--ei delay_ms`, `--ei gap_ms`) |
163
+ | `--ei mic_test_ms 9000` | after loading and `delay_ms`: record that long, then transcribe (`--ez mic_test_play true` plays clip en meanwhile) |
164
+
165
+ The app logs one `STATE <name>` line per screen state and `LOAD` / `TRANSCRIBED` lines with the times
166
+ under the tag `AsrLitertlm`.
167
+
168
+ ## Files
169
+
170
+ | File | Role |
171
+ |---|---|
172
+ | `app/src/main/kotlin/com/asrlitertlm/AsrEngine.kt` | Engine and one Conversation per clip: load, transcribe (cut at 30 s), release (safe twice) |
173
+ | `app/src/main/kotlin/com/asrlitertlm/ModelProfile.kt` | the three bundles: file name, Hugging Face id, answer format |
174
+ | `app/src/main/kotlin/com/asrlitertlm/AsrSession.kt` | the app's one engine on one worker thread, events for the screen, the no-speech check |
175
+ | `app/src/main/kotlin/com/asrlitertlm/MainActivity.kt` | picker, hold to talk, clip buttons, launch extras |
176
+ | `app/src/main/kotlin/com/asrlitertlm/MicRecorder.kt` | 16 kHz mono recording, up to 30 s |
177
+ | `app/src/main/kotlin/com/asrlitertlm/ClipPlayer.kt`, `PlayBar.kt` | clip playback through the speaker and its progress bar |
178
+ | `app/src/main/kotlin/com/asrlitertlm/Wav.kt`, `TextMatch.kt` | WAV read, write and cut; transcript comparison (normalization and CER) |
179
+ | `app/src/main/res/raw/fleurs_*_1698.wav` | the three clips |
180
+ | `app/src/androidTest/kotlin/com/asrlitertlm/AsrDeviceCheck.kt` | device check |
181
+ | `app/src/test/kotlin/com/asrlitertlm/AsrLogicTest.kt` | JVM tests: answer parsing, transcript comparison, WAV cut and levels |
182
+
183
+ Build: AGP 9.3.1 with Kotlin 2.4.0 (the `litertlm-android` 0.17.1 AAR carries Kotlin 2.4 metadata),
184
+ compileSdk 36, minSdk 31, arm64-v8a. The runtime version is pinned in `gradle.properties`.
185
+
186
+ ## License
187
+
188
+ The app code is under the repository's MIT license. The clips are CC BY 4.0 (FLEURS, see above). Each
189
+ model file has its own license (table above); none of them is in this directory.
android/app/build.gradle.kts ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // On-device speech recognition with three LiteRT-LM bundles (Qwen3-ASR-1.7B, Fun-ASR-Nano-2512, Confucius4-R2T2):
2
+ // one Engine at a time, a new Conversation per clip, the language model and the audio encoder on the CPU.
3
+ plugins { id("com.android.application") }
4
+
5
+ val litertlmVersion: String = providers.gradleProperty("litertlmVersion").get()
6
+
7
+ android {
8
+ namespace = "com.asrlitertlm"
9
+ compileSdk = 36
10
+
11
+ defaultConfig {
12
+ applicationId = "com.asrlitertlm"
13
+ minSdk = 31
14
+ targetSdk = 36
15
+ versionCode = 1
16
+ versionName = "1.0"
17
+ ndk { abiFilters += setOf("arm64-v8a") }
18
+ buildConfigField("String", "LITERTLM_VERSION", "\"$litertlmVersion\"")
19
+ testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
20
+ }
21
+
22
+ buildFeatures { buildConfig = true }
23
+
24
+ compileOptions {
25
+ sourceCompatibility = JavaVersion.VERSION_17
26
+ targetCompatibility = JavaVersion.VERSION_17
27
+ }
28
+ }
29
+
30
+ dependencies {
31
+ // Brings gson, kotlin-reflect and kotlinx-coroutines-android (its POM).
32
+ implementation("com.google.ai.edge.litertlm:litertlm-android:$litertlmVersion")
33
+
34
+ testImplementation("junit:junit:4.13.2")
35
+
36
+ // Device check (app/src/androidTest): README "Device check" runs it with am instrument.
37
+ androidTestImplementation("androidx.test:runner:1.6.2")
38
+ androidTestImplementation("androidx.test.ext:junit:1.2.1")
39
+ }
android/app/src/androidTest/kotlin/com/asrlitertlm/AsrDeviceCheck.kt ADDED
@@ -0,0 +1,417 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ import android.Manifest
4
+ import android.content.Intent
5
+ import android.content.pm.PackageManager
6
+ import android.util.Log
7
+ import androidx.test.ext.junit.runners.AndroidJUnit4
8
+ import androidx.test.platform.app.InstrumentationRegistry
9
+ import java.io.File
10
+ import java.util.Locale
11
+ import java.util.concurrent.CountDownLatch
12
+ import java.util.concurrent.ExecutionException
13
+ import java.util.concurrent.Future
14
+ import java.util.concurrent.TimeUnit
15
+ import java.util.concurrent.atomic.AtomicReference
16
+ import org.junit.Assert.assertTrue
17
+ import org.junit.Assume.assumeTrue
18
+ import org.junit.Test
19
+ import org.junit.runner.RunWith
20
+
21
+ /**
22
+ * Device check for the app's paths on a phone with the bundles pushed. One line per step under the log tag
23
+ * `asr-check`: `RESULT step=<name> ok=<bool> model=<id> ...`, then `RESULT ok=<all> scope=<test>`.
24
+ *
25
+ * [check] expects RECORD_AUDIO to be revoked when it starts (`adb shell pm revoke com.asrlitertlm
26
+ * android.permission.RECORD_AUDIO`): its first step is the microphone path without the permission; it then grants the
27
+ * permission itself. Per bundle on the phone: load, the three clips (the answer's text after normalization equals the
28
+ * expected sentence; the CER is printed when it does not), the microphone (clip en played through the speaker and
29
+ * picked up by the microphone, CER <= 0.15), release (a second release is harmless) and, when the load wrote the
30
+ * runtime cache, a second load. Then the failure paths: switching models, a clip longer than 30 s, a silent
31
+ * recording and a bundle that is not on the phone.
32
+ *
33
+ * [gpu] runs only with `-e gpu_model <id>`: the language model on the GPU, the audio encoder on the CPU.
34
+ */
35
+ @RunWith(AndroidJUnit4::class)
36
+ class AsrDeviceCheck {
37
+ private val instrumentation = InstrumentationRegistry.getInstrumentation()
38
+ private val context = instrumentation.targetContext
39
+ private val outcomes = mutableListOf<Boolean>()
40
+
41
+ @Test
42
+ fun check() {
43
+ AsrSession.init(context)
44
+ val present = ORDER.filter { AsrSession.isPresent(it) }
45
+ report(
46
+ "bundles",
47
+ present.isNotEmpty(),
48
+ null,
49
+ "present" to present.joinToString(",") { it.id }.ifEmpty { "none" },
50
+ "missing" to ORDER.filterNot { it in present }.joinToString(",") { it.id }.ifEmpty { "none" },
51
+ "dir" to AsrSession.modelDir.path,
52
+ )
53
+ if (present.isEmpty()) return finish("check")
54
+
55
+ val first = present.first()
56
+ val firstCache = cacheFiles(first)
57
+ val screen = launch(launchIntent(first))
58
+ try {
59
+ val firstLoad = awaitLoad(checkNotNull(AsrSession.lastLoad) { "the screen did not start a load" }, first)
60
+ val loads = mutableMapOf(first to (firstLoad to firstCache))
61
+
62
+ permissionStep(screen, first)
63
+ instrumentation.uiAutomation.grantRuntimePermission(context.packageName, Manifest.permission.RECORD_AUDIO)
64
+ report("grant", hasMicPermission(), null, "granted" to hasMicPermission())
65
+
66
+ for (profile in present) {
67
+ val load = loads[profile] ?: run {
68
+ val before = cacheFiles(profile)
69
+ awaitLoad(onScreen(screen) { it.selectModel(profile) }, profile)?.let { it to before }
70
+ }
71
+ if (load?.first == null) continue
72
+ reportLoad("load", profile, load.first!!, load.second)
73
+ clipSteps(profile, "clip")
74
+ micStep(screen, profile)
75
+ releaseStep(profile)
76
+ if (cacheState(profile, load.second) == "written") reloadStep(profile)
77
+ }
78
+ failurePaths(screen, present)
79
+ } finally {
80
+ close(screen)
81
+ }
82
+ finish("check")
83
+ }
84
+
85
+ @Test
86
+ fun gpu() {
87
+ val profile = ModelProfile.byId(InstrumentationRegistry.getArguments().getString("gpu_model"))
88
+ assumeTrue("run with -e gpu_model <id>", profile != null)
89
+ AsrSession.init(context)
90
+ val model = profile!!
91
+ if (!AsrSession.isPresent(model)) {
92
+ report("gpu-load", false, model, "reason" to "bundle not on the phone")
93
+ return finish("gpu")
94
+ }
95
+ val before = cacheFiles(model)
96
+ val screen = launch(launchIntent(model).putExtra(MainActivity.EXTRA_BACKEND, "gpu"))
97
+ try {
98
+ val info = awaitLoad(checkNotNull(AsrSession.lastLoad), model, step = "gpu-load")
99
+ if (info != null) {
100
+ reportLoad("gpu-load", model, info, before)
101
+ clipSteps(model, "gpu-clip")
102
+ releaseStep(model, step = "gpu-release")
103
+ val beforeReload = cacheFiles(model)
104
+ val reloaded = awaitLoad(AsrSession.load(model, LmBackend.GPU), model, step = "gpu-reload")
105
+ if (reloaded != null) reportLoad("gpu-reload", model, reloaded, beforeReload)
106
+ AsrSession.release().get(TIMEOUT_S, TimeUnit.SECONDS)
107
+ }
108
+ } finally {
109
+ close(screen)
110
+ }
111
+ finish("gpu")
112
+ }
113
+
114
+ // ---- steps --------------------------------------------------------------------------------------------------
115
+
116
+ /** The microphone path with RECORD_AUDIO revoked returns a sentence, records nothing, and a clip still works. */
117
+ private fun permissionStep(screen: MainActivity, profile: ModelProfile) {
118
+ val deniedAtStart = !hasMicPermission()
119
+ val result = micTest(screen, durationMs = 2000, play = false, waitS = 10)
120
+ val sentence = (result as? MicTestResult.Failed)?.message.orEmpty()
121
+ val clip = Clip.ALL[1]
122
+ val clipText = transcribeText(AsrSession.clipFile(clip), "clip:${clip.id}")
123
+ val clipOk = clipText != null && TextMatch.matches(EXPECTED.getValue(clip.id), clipText.answer.text)
124
+ report(
125
+ "permission",
126
+ deniedAtStart && sentence == MicRecorder.PERMISSION_MESSAGE && clipOk,
127
+ profile,
128
+ "denied_at_start" to deniedAtStart,
129
+ "message" to sentence.ifEmpty { result.toString() },
130
+ "clip_en_while_denied" to clipOk,
131
+ )
132
+ }
133
+
134
+ private fun clipSteps(profile: ModelProfile, step: String) {
135
+ for (clip in Clip.ALL) {
136
+ val expected = EXPECTED.getValue(clip.id)
137
+ val transcript = transcribeText(AsrSession.clipFile(clip), "clip:${clip.id}")
138
+ if (transcript == null) {
139
+ report(step, false, profile, "clip" to clip.label, "reason" to "no transcript")
140
+ continue
141
+ }
142
+ val text = transcript.answer.text
143
+ val match = TextMatch.matches(expected, text)
144
+ report(
145
+ step,
146
+ match,
147
+ profile,
148
+ "clip" to clip.label,
149
+ "backend" to transcript.lmBackend,
150
+ "audio_s" to fmt(transcript.sentSeconds),
151
+ "send_s" to fmt(transcript.sendMs / 1000),
152
+ "rtf" to fmt(transcript.rtf, digits = 3),
153
+ "conversation_s" to fmt(transcript.createConversationMs / 1000),
154
+ "language" to transcript.answer.language.ifEmpty { "-" },
155
+ "match" to match,
156
+ "cer" to fmt(TextMatch.cer(expected, text), digits = 4),
157
+ "text" to text,
158
+ )
159
+ }
160
+ }
161
+
162
+ /** Clip en through the speaker, picked up by the microphone (the mic_test launch extras' path). */
163
+ private fun micStep(screen: MainActivity, profile: ModelProfile) {
164
+ val result = micTest(screen, durationMs = MIC_TEST_MS, play = true, waitS = 90)
165
+ val outcome = (result as? MicTestResult.Finished)?.outcome
166
+ val transcript = (outcome as? TranscribeOutcome.Text)?.transcript
167
+ if (transcript == null) {
168
+ report("mic", false, profile, "result" to result.toString())
169
+ return
170
+ }
171
+ val expected = EXPECTED.getValue(Clip.ALL[1].id)
172
+ val cer = TextMatch.cer(expected, transcript.answer.text)
173
+ report(
174
+ "mic",
175
+ cer <= MIC_MAX_CER,
176
+ profile,
177
+ "rec_s" to fmt(transcript.audioSeconds),
178
+ "send_s" to fmt(transcript.sendMs / 1000),
179
+ "rtf" to fmt(transcript.rtf, digits = 3),
180
+ "cer" to fmt(cer, digits = 4),
181
+ "language" to transcript.answer.language.ifEmpty { "-" },
182
+ "text" to transcript.answer.text,
183
+ )
184
+ }
185
+
186
+ private fun releaseStep(profile: ModelProfile, step: String = "release") {
187
+ val first = runCatching { AsrSession.release().get(TIMEOUT_S, TimeUnit.SECONDS) }
188
+ val second = runCatching { AsrSession.release().get(TIMEOUT_S, TimeUnit.SECONDS) }
189
+ report(
190
+ step,
191
+ first.getOrNull() == true && second.getOrNull() == false && AsrSession.current == null,
192
+ profile,
193
+ "first" to (first.getOrNull() ?: first.exceptionOrNull()),
194
+ "second" to (second.getOrNull() ?: second.exceptionOrNull()),
195
+ "vmrss_mb" to statusMb("VmRSS"),
196
+ )
197
+ }
198
+
199
+ /** A second load, now that the first wrote the runtime cache: the load time a user sees from then on. */
200
+ private fun reloadStep(profile: ModelProfile) {
201
+ val before = cacheFiles(profile)
202
+ val info = awaitLoad(AsrSession.load(profile), profile, step = "reload") ?: return
203
+ reportLoad("reload", profile, info, before)
204
+ AsrSession.release().get(TIMEOUT_S, TimeUnit.SECONDS)
205
+ }
206
+
207
+ private fun failurePaths(screen: MainActivity, present: List<ModelProfile>) {
208
+ // Switching: load one bundle, then pick another without releasing; the screen's path closes the first.
209
+ val to = present.first()
210
+ val from = present.firstOrNull { it != to }
211
+ if (from == null) {
212
+ // Not a result: the phone holds one bundle (with its runtime cache) at a time when storage is short.
213
+ Log.i(TAG, "NOTE step=switch not_run=\"one bundle on the phone\" model=${to.id}")
214
+ } else {
215
+ awaitLoad(onScreen(screen) { it.selectModel(from) }, from, step = "switch")
216
+ val switched = awaitLoad(onScreen(screen) { it.selectModel(to) }, to, step = "switch")
217
+ val clip = Clip.ALL[1]
218
+ val text = transcribeText(AsrSession.clipFile(clip), "clip:${clip.id}")?.answer?.text
219
+ report(
220
+ "switch",
221
+ switched != null && AsrSession.current?.profile == to && text != null &&
222
+ TextMatch.matches(EXPECTED.getValue(clip.id), text),
223
+ to,
224
+ "from" to from.id,
225
+ "load_s" to (switched?.let { fmt(it.loadMs / 1000) } ?: "-"),
226
+ "vmrss_mb" to statusMb("VmRSS"),
227
+ "text" to (text ?: "-"),
228
+ )
229
+ }
230
+ if (AsrSession.current == null) awaitLoad(AsrSession.load(to), to, step = "long-clip")
231
+
232
+ // Longer than one window: the app sends the first 30 s and does not crash.
233
+ val long = File(context.cacheDir, "long-check.wav")
234
+ val pcm = (Clip.ALL + Clip.ALL[1]).map { Wav.pcm16(AsrSession.clipFile(it)).data }
235
+ Wav.writeMono16(long, pcm.fold(ByteArray(0)) { all, part -> all + part }, MicRecorder.SAMPLE_RATE)
236
+ val longText = transcribeText(long, "long-clip")
237
+ report(
238
+ "long-clip",
239
+ longText != null && longText.audioSeconds > 31.0 && kotlin.math.abs(longText.sentSeconds - 30.0) < 0.01 &&
240
+ longText.answer.text.isNotEmpty(),
241
+ AsrSession.current?.profile,
242
+ "audio_s" to fmt(longText?.audioSeconds ?: Wav.seconds(long)),
243
+ "sent_s" to (longText?.let { fmt(it.sentSeconds) } ?: "-"),
244
+ "send_s" to (longText?.let { fmt(it.sendMs / 1000) } ?: "-"),
245
+ "text" to (longText?.answer?.text ?: "-"),
246
+ )
247
+
248
+ // A silent recording: "no speech", and the model is not called.
249
+ val silent = File(context.cacheDir, "silence-check.wav")
250
+ Wav.writeMono16(silent, ByteArray(MicRecorder.SAMPLE_RATE * 2 * 3), MicRecorder.SAMPLE_RATE)
251
+ val silence = runCatching { AsrSession.transcribe(silent, "silence").get(TIMEOUT_S, TimeUnit.SECONDS) }
252
+ report(
253
+ "silence",
254
+ silence.getOrNull() is TranscribeOutcome.NoSpeech,
255
+ AsrSession.current?.profile,
256
+ "result" to (silence.getOrNull() ?: silence.exceptionOrNull()),
257
+ )
258
+
259
+ // A bundle that is not on the phone: a sentence naming the file and where to push it; the loaded model stays.
260
+ val loadedBefore = AsrSession.current?.profile
261
+ val ghost = ModelProfile.QWEN3_ASR_1_7B.copy(fileName = "not-on-this-phone.litertlm")
262
+ val missing =
263
+ runCatching { AsrSession.load(ghost, bundle = AsrSession.bundleFile(ghost)).get(TIMEOUT_S, TimeUnit.SECONDS) }
264
+ val message = ((missing.exceptionOrNull() as? ExecutionException)?.cause as? AsrException)?.message.orEmpty()
265
+ report(
266
+ "missing-bundle",
267
+ missing.isFailure && message.contains("not-on-this-phone.litertlm") &&
268
+ message.contains(AsrSession.modelDir.path) && AsrSession.current?.profile == loadedBefore,
269
+ loadedBefore,
270
+ "message" to message.ifEmpty { missing.toString() },
271
+ "still_loaded" to (AsrSession.current?.profile?.id ?: "none"),
272
+ )
273
+ releaseStep(loadedBefore ?: to, step = "release-final")
274
+ long.delete()
275
+ silent.delete()
276
+ }
277
+
278
+ // ---- helpers ------------------------------------------------------------------------------------------------
279
+
280
+ private fun launchIntent(profile: ModelProfile) =
281
+ Intent(context, MainActivity::class.java)
282
+ .putExtra(MainActivity.EXTRA_MODEL, profile.id)
283
+ .addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
284
+
285
+ /** Starts the screen in the foreground (the microphone records only for a visible app) and waits until it runs. */
286
+ private fun launch(intent: Intent): MainActivity = instrumentation.startActivitySync(intent) as MainActivity
287
+
288
+ private fun close(screen: MainActivity) {
289
+ instrumentation.runOnMainSync { screen.finish() }
290
+ instrumentation.waitForIdleSync()
291
+ }
292
+
293
+ private fun <T> onScreen(screen: MainActivity, block: (MainActivity) -> T): T {
294
+ val value = AtomicReference<T>()
295
+ instrumentation.runOnMainSync { value.set(block(screen)) }
296
+ return value.get()
297
+ }
298
+
299
+ private fun awaitLoad(future: Future<LoadInfo>?, profile: ModelProfile, step: String = "load"): LoadInfo? {
300
+ if (future == null) {
301
+ report(step, false, profile, "reason" to "the screen did not load it")
302
+ return null
303
+ }
304
+ return try {
305
+ future.get(LOAD_TIMEOUT_S, TimeUnit.SECONDS)
306
+ } catch (e: ExecutionException) {
307
+ report(step, false, profile, "message" to (e.cause?.message ?: e.toString()))
308
+ null
309
+ }
310
+ }
311
+
312
+ private fun transcribeText(wav: File, source: String): Transcript? =
313
+ try {
314
+ (AsrSession.transcribe(wav, source).get(TIMEOUT_S, TimeUnit.SECONDS) as? TranscribeOutcome.Text)?.transcript
315
+ } catch (e: ExecutionException) {
316
+ Log.e(TAG, "transcription failed: $source", e.cause)
317
+ null
318
+ }
319
+
320
+ private fun micTest(screen: MainActivity, durationMs: Long, play: Boolean, waitS: Long): MicTestResult? {
321
+ val done = CountDownLatch(1)
322
+ val result = AtomicReference<MicTestResult>()
323
+ instrumentation.runOnMainSync {
324
+ screen.runMicTest(durationMs, play) {
325
+ result.set(it)
326
+ done.countDown()
327
+ }
328
+ }
329
+ done.await(waitS, TimeUnit.SECONDS)
330
+ return result.get()
331
+ }
332
+
333
+ private fun reportLoad(step: String, profile: ModelProfile, info: LoadInfo, before: Map<String, CacheFile>) {
334
+ report(
335
+ step,
336
+ true,
337
+ profile,
338
+ "backend" to info.lmBackend,
339
+ "load_s" to fmt(info.loadMs / 1000),
340
+ "initialize_s" to fmt(info.initializeMs / 1000),
341
+ "first_conversation_s" to fmt(info.firstConversationMs / 1000),
342
+ "cache" to cacheState(profile, before),
343
+ "cache_mb" to cacheFiles(profile).values.sumOf { it.bytes } / 1_000_000,
344
+ "threads" to AsrEngine.CPU_THREADS,
345
+ "vmhwm_mb" to statusMb("VmHWM"),
346
+ "vmrss_mb" to statusMb("VmRSS"),
347
+ "mem_available_mb" to memAvailableMb(),
348
+ )
349
+ }
350
+
351
+ /** One runtime cache file: the runtime names them after the bundle file. */
352
+ private data class CacheFile(val bytes: Long, val modified: Long)
353
+
354
+ private fun cacheFiles(profile: ModelProfile): Map<String, CacheFile> =
355
+ context.cacheDir
356
+ .listFiles { file -> file.name.startsWith(profile.fileName) }
357
+ .orEmpty()
358
+ .associate { it.name to CacheFile(it.length(), it.lastModified()) }
359
+
360
+ /** "loaded" when the load left the cache files as they were, "written" when it created or rewrote them. */
361
+ private fun cacheState(profile: ModelProfile, before: Map<String, CacheFile>): String =
362
+ if (cacheFiles(profile) == before && before.isNotEmpty()) "loaded" else "written"
363
+
364
+ private fun hasMicPermission(): Boolean =
365
+ context.checkSelfPermission(Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED
366
+
367
+ private fun statusMb(key: String): Long =
368
+ File("/proc/self/status").readLines().firstOrNull { it.startsWith("$key:") }
369
+ ?.trim()?.split(Regex("\\s+"))?.getOrNull(1)?.toLongOrNull()?.div(1024) ?: -1
370
+
371
+ private fun memAvailableMb(): Long =
372
+ File("/proc/meminfo").readLines().firstOrNull { it.startsWith("MemAvailable:") }
373
+ ?.trim()?.split(Regex("\\s+"))?.getOrNull(1)?.toLongOrNull()?.div(1024) ?: -1
374
+
375
+ private fun report(step: String, ok: Boolean, profile: ModelProfile?, vararg fields: Pair<String, Any?>) {
376
+ outcomes += ok
377
+ val line = StringBuilder("RESULT step=$step ok=$ok model=${profile?.id ?: "-"}")
378
+ for ((key, value) in fields) line.append(' ').append(key).append('=').append(quote(value.toString()))
379
+ Log.i(TAG, line.toString())
380
+ }
381
+
382
+ private fun finish(scope: String) {
383
+ val all = outcomes.isNotEmpty() && outcomes.all { it }
384
+ Log.i(TAG, "RESULT ok=$all scope=$scope steps=${outcomes.size} failed=${outcomes.count { !it }}")
385
+ assertTrue("a device check step failed (logcat -s $TAG)", all)
386
+ }
387
+
388
+ private fun quote(value: String): String =
389
+ if (value.isNotEmpty() && value.none { it.isWhitespace() || it == '"' }) value
390
+ else "\"" + value.replace("\\", "\\\\").replace("\"", "\\\"").replace("\n", "\\n") + "\""
391
+
392
+ private fun fmt(value: Double, digits: Int = 2) = String.format(Locale.US, "%.${digits}f", value)
393
+
394
+ private companion object {
395
+ const val TAG = "asr-check"
396
+ const val LOAD_TIMEOUT_S = 300L
397
+ const val TIMEOUT_S = 120L
398
+ const val MIC_TEST_MS = 9000L
399
+ const val MIC_MAX_CER = 0.15
400
+
401
+ /** Bundles in the order the check loads them. */
402
+ val ORDER =
403
+ listOf(ModelProfile.QWEN3_ASR_1_7B, ModelProfile.CONFUCIUS4_R2T2, ModelProfile.FUN_ASR_NANO_2512)
404
+
405
+ /**
406
+ * The expected sentences: Qwen3-ASR-1.7B's answers on the Mac CPU (litert-lm-api 0.17.1), which the S26 app runs
407
+ * of 2026-10-03 matched. Fun-ASR-Nano-2512 and Confucius4-R2T2 give the same sentences on the Mac CPU.
408
+ */
409
+ val EXPECTED =
410
+ mapOf(
411
+ "fleurs_cmn_1698" to "宇宙中的一切都由物质构成,而所有的物质都由被称为原子的微小颗粒组成。",
412
+ "fleurs_en_1698" to
413
+ "Everything in the universe is made of matter. All matter is made of tiny particles called atoms.",
414
+ "fleurs_ja_1698" to "宇宙に存在するあらゆるものは物質でできています。あらゆる物質は原子と呼ばれる微小な粒子でできています。",
415
+ )
416
+ }
417
+ }
android/app/src/main/AndroidManifest.xml ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <?xml version="1.0" encoding="utf-8"?>
2
+ <manifest xmlns:android="http://schemas.android.com/apk/res/android">
3
+
4
+ <uses-permission android:name="android.permission.RECORD_AUDIO" />
5
+
6
+ <application
7
+ android:allowBackup="false"
8
+ android:label="@string/app_name"
9
+ android:largeHeap="true"
10
+ android:theme="@style/Theme.AsrLitertlm">
11
+
12
+ <!-- Only for the language model on the GPU (launch extra backend=gpu); the CPU path does not need it. -->
13
+ <uses-native-library
14
+ android:name="libOpenCL.so"
15
+ android:required="false" />
16
+
17
+ <activity
18
+ android:name=".MainActivity"
19
+ android:exported="true"
20
+ android:launchMode="singleTask"
21
+ android:screenOrientation="portrait">
22
+ <intent-filter>
23
+ <action android:name="android.intent.action.MAIN" />
24
+ <category android:name="android.intent.category.LAUNCHER" />
25
+ </intent-filter>
26
+ </activity>
27
+ </application>
28
+ </manifest>
android/app/src/main/kotlin/com/asrlitertlm/AsrEngine.kt ADDED
@@ -0,0 +1,181 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ import android.os.SystemClock
4
+ import com.google.ai.edge.litertlm.Backend
5
+ import com.google.ai.edge.litertlm.Content
6
+ import com.google.ai.edge.litertlm.Contents
7
+ import com.google.ai.edge.litertlm.ConversationConfig
8
+ import com.google.ai.edge.litertlm.Engine
9
+ import com.google.ai.edge.litertlm.EngineConfig
10
+ import com.google.ai.edge.litertlm.Message
11
+ import com.google.ai.edge.litertlm.SamplerConfig
12
+ import java.io.File
13
+
14
+ /** Which processor runs the language model. The audio encoder always runs on the CPU (the GPU delegate rejects it). */
15
+ enum class LmBackend {
16
+ CPU,
17
+ GPU,
18
+ }
19
+
20
+ /** A failure the app shows as a sentence instead of crashing. */
21
+ class AsrException(message: String, cause: Throwable? = null) : Exception(message, cause)
22
+
23
+ /** What one load cost. "Engine load" = Engine.initialize() plus the first conversation, which creates the encoder. */
24
+ data class LoadInfo(
25
+ val profile: ModelProfile,
26
+ val bundle: File,
27
+ val bundleBytes: Long,
28
+ val lmBackend: LmBackend,
29
+ val initializeMs: Double,
30
+ val firstConversationMs: Double,
31
+ ) {
32
+ val loadMs: Double
33
+ get() = initializeMs + firstConversationMs
34
+ }
35
+
36
+ /** One transcription. [sendMs] is the wall time of sendMessage(), from sending the clip to the full answer. */
37
+ data class Transcript(
38
+ val profile: ModelProfile,
39
+ val lmBackend: LmBackend,
40
+ val raw: String,
41
+ val answer: Answer,
42
+ /** Length of the audio the app received. */
43
+ val audioSeconds: Double,
44
+ /** Length of the audio sent to the model (at most [AsrEngine.MAX_AUDIO_SECONDS]). */
45
+ val sentSeconds: Double,
46
+ val sendMs: Double,
47
+ val createConversationMs: Double,
48
+ ) {
49
+ /** Real-time factor: transcription time over the length of the audio the model heard. */
50
+ val rtf: Double
51
+ get() = sendMs / 1000.0 / sentSeconds
52
+ }
53
+
54
+ /**
55
+ * One loaded `.litertlm` bundle and the calls every model in [ModelProfile.ALL] shares: an Engine with the language
56
+ * model on the CPU (or the GPU) and the audio encoder on the CPU, a new Conversation per clip, one user message that
57
+ * holds only the audio file, greedy sampling.
58
+ *
59
+ * Not thread-safe: the app calls it from one worker thread ([AsrSession]).
60
+ */
61
+ class AsrEngine(private val cacheDir: File) : AutoCloseable {
62
+ private var engine: Engine? = null
63
+
64
+ /** The bundle in memory, or null. */
65
+ var loaded: LoadInfo? = null
66
+ private set
67
+
68
+ /**
69
+ * Loads [profile] from [bundle], closing the engine loaded before. A missing file or a runtime failure becomes an
70
+ * [AsrException] with a sentence for the screen; a missing file leaves the loaded engine as it was.
71
+ */
72
+ fun load(profile: ModelProfile, bundle: File, lmBackend: LmBackend = LmBackend.CPU): LoadInfo {
73
+ if (!bundle.isFile) throw AsrException(missingBundleMessage(profile, bundle.parentFile))
74
+ // One engine at a time: two 2.7 GB bundles in memory at once would cross what the phone gives one app.
75
+ release()
76
+ val lm = if (lmBackend == LmBackend.GPU) Backend.GPU() else Backend.CPU(threadCount = CPU_THREADS)
77
+ val config =
78
+ EngineConfig(
79
+ modelPath = bundle.path,
80
+ backend = lm,
81
+ audioBackend = Backend.CPU(threadCount = CPU_THREADS),
82
+ cacheDir = cacheDir.path,
83
+ )
84
+ val newEngine = Engine(config)
85
+ val start = SystemClock.elapsedRealtimeNanos()
86
+ val initialized: Long
87
+ try {
88
+ newEngine.initialize()
89
+ initialized = SystemClock.elapsedRealtimeNanos()
90
+ // The runtime creates the audio encoder with the first conversation; do it here so it counts as loading.
91
+ newEngine.createConversation(conversationConfig(profile)).close()
92
+ } catch (t: Throwable) {
93
+ if (newEngine.isInitialized()) runCatching { newEngine.close() }
94
+ throw AsrException("Could not load ${profile.fileName} (${lmBackend.name}): ${t.message ?: t}", t)
95
+ }
96
+ val ready = SystemClock.elapsedRealtimeNanos()
97
+ engine = newEngine
98
+ return LoadInfo(
99
+ profile = profile,
100
+ bundle = bundle,
101
+ bundleBytes = bundle.length(),
102
+ lmBackend = lmBackend,
103
+ initializeMs = (initialized - start) / 1e6,
104
+ firstConversationMs = (ready - initialized) / 1e6,
105
+ )
106
+ .also { loaded = it }
107
+ }
108
+
109
+ /**
110
+ * Transcribes a 16 kHz mono 16-bit wav. Audio longer than [MAX_AUDIO_SECONDS] is cut there first ([scratch] holds
111
+ * the cut copy): each model reads one window of about 30 s per message.
112
+ */
113
+ fun transcribe(wav: File, scratch: File): Transcript {
114
+ val currentEngine = engine ?: throw AsrException("No model is loaded.")
115
+ val info = loaded ?: throw AsrException("No model is loaded.")
116
+ val audioSeconds = Wav.seconds(wav)
117
+ val sent = Wav.trimmed(wav, MAX_AUDIO_SECONDS, scratch)
118
+ val sentSeconds = if (sent === wav) audioSeconds else Wav.seconds(sent)
119
+ val created: Long
120
+ val sendStart: Long
121
+ val sendEnd: Long
122
+ val raw: String
123
+ val start = SystemClock.elapsedRealtimeNanos()
124
+ try {
125
+ currentEngine.createConversation(conversationConfig(info.profile)).use { conversation ->
126
+ created = SystemClock.elapsedRealtimeNanos()
127
+ val message = Message.user(Contents.of(Content.AudioFile(sent.absolutePath)))
128
+ sendStart = SystemClock.elapsedRealtimeNanos()
129
+ raw = conversation.sendMessage(message).toString()
130
+ sendEnd = SystemClock.elapsedRealtimeNanos()
131
+ }
132
+ } catch (t: Throwable) {
133
+ throw AsrException("${info.profile.displayName} could not transcribe this clip: ${t.message ?: t}", t)
134
+ }
135
+ return Transcript(
136
+ profile = info.profile,
137
+ lmBackend = info.lmBackend,
138
+ raw = raw,
139
+ answer = info.profile.parse(raw),
140
+ audioSeconds = audioSeconds,
141
+ sentSeconds = sentSeconds,
142
+ sendMs = (sendEnd - sendStart) / 1e6,
143
+ createConversationMs = (created - start) / 1e6,
144
+ )
145
+ }
146
+
147
+ /**
148
+ * Closes the engine and returns true, or returns false when none is loaded. Safe to call twice: the runtime's own
149
+ * Engine.close() throws on a second call, so the reference is dropped before closing.
150
+ */
151
+ fun release(): Boolean {
152
+ val current = engine ?: return false
153
+ engine = null
154
+ loaded = null
155
+ current.close()
156
+ return true
157
+ }
158
+
159
+ override fun close() {
160
+ release()
161
+ }
162
+
163
+ companion object {
164
+ /** CPU threads for the language model and the audio encoder (the model cards' Galaxy S26 rows). */
165
+ const val CPU_THREADS = 4
166
+
167
+ /** The longest audio sent in one message; longer audio is cut here. */
168
+ const val MAX_AUDIO_SECONDS = 30.0
169
+
170
+ fun conversationConfig(profile: ModelProfile) =
171
+ ConversationConfig(
172
+ samplerConfig = SamplerConfig(topK = 1, topP = 1.0, temperature = 0.0),
173
+ maxOutputToken = profile.maxOutputTokens,
174
+ )
175
+
176
+ /** The sentence the picker and a failed load show for a bundle that is not on the phone. */
177
+ fun missingBundleMessage(profile: ModelProfile, directory: File?): String =
178
+ "${profile.fileName} is not on this phone. Download it from huggingface.co/${profile.hubRepo} and push it " +
179
+ "to ${directory?.path ?: "the app's files directory"}/ with adb (see README)."
180
+ }
181
+ }
android/app/src/main/kotlin/com/asrlitertlm/AsrSession.kt ADDED
@@ -0,0 +1,248 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ import android.content.Context
4
+ import android.os.Handler
5
+ import android.os.Looper
6
+ import android.util.Log
7
+ import java.io.File
8
+ import java.io.IOException
9
+ import java.util.Locale
10
+ import java.util.concurrent.Callable
11
+ import java.util.concurrent.CopyOnWriteArrayList
12
+ import java.util.concurrent.ExecutorService
13
+ import java.util.concurrent.Executors
14
+ import java.util.concurrent.Future
15
+
16
+ /** One bundled clip: FLEURS test sentence 1698 read in one language (google/fleurs, CC BY 4.0). */
17
+ data class Clip(val id: String, val label: String, val rawRes: Int) {
18
+ companion object {
19
+ val ALL =
20
+ listOf(
21
+ Clip("fleurs_cmn_1698", "zh", R.raw.fleurs_cmn_1698),
22
+ Clip("fleurs_en_1698", "en", R.raw.fleurs_en_1698),
23
+ Clip("fleurs_ja_1698", "ja", R.raw.fleurs_ja_1698),
24
+ )
25
+ }
26
+ }
27
+
28
+ /** The end of one transcription request. */
29
+ sealed interface TranscribeOutcome {
30
+ data class Text(val transcript: Transcript) : TranscribeOutcome
31
+
32
+ /** The audio was close to digital silence, so the model was not called. */
33
+ data class NoSpeech(val levels: Wav.Levels, val seconds: Double) : TranscribeOutcome
34
+ }
35
+
36
+ /** What the session did, delivered to listeners on the main thread. */
37
+ sealed interface AsrEvent {
38
+ data class Loading(val profile: ModelProfile, val lmBackend: LmBackend) : AsrEvent
39
+
40
+ data class Loaded(val info: LoadInfo) : AsrEvent
41
+
42
+ data class LoadFailed(val profile: ModelProfile, val message: String) : AsrEvent
43
+
44
+ data class Transcribing(val source: String) : AsrEvent
45
+
46
+ data class Transcribed(val source: String, val outcome: TranscribeOutcome) : AsrEvent
47
+
48
+ data class TranscribeFailed(val source: String, val message: String) : AsrEvent
49
+
50
+ data class Released(val profile: ModelProfile) : AsrEvent
51
+ }
52
+
53
+ /**
54
+ * The app's one engine, shared by the screen and the device check. Every model call runs on one worker thread (an
55
+ * Engine is not called from two threads), and one bundle is in memory at a time. The methods return futures for
56
+ * callers that wait (the device check) and publish [AsrEvent]s for the screen.
57
+ */
58
+ object AsrSession {
59
+ private const val TAG = "AsrLitertlm"
60
+
61
+ /** Recordings quieter than this (RMS) are reported as no speech instead of being sent to the model. */
62
+ const val NO_SPEECH_RMS_DBFS = -60.0
63
+
64
+ private val worker: ExecutorService = Executors.newSingleThreadExecutor { Thread(it, "asr-worker") }
65
+ private val main = Handler(Looper.getMainLooper())
66
+ private val listeners = CopyOnWriteArrayList<(AsrEvent) -> Unit>()
67
+
68
+ @Volatile private var engine: AsrEngine? = null
69
+ private lateinit var modelDirectory: File
70
+ private lateinit var cacheDirectory: File
71
+ private lateinit var appContext: Context
72
+
73
+ /** The bundle in memory, or null. */
74
+ @Volatile
75
+ var current: LoadInfo? = null
76
+ private set
77
+
78
+ /** The last load request, so that a caller can wait for a load the screen started. */
79
+ @Volatile
80
+ var lastLoad: Future<LoadInfo>? = null
81
+ private set
82
+
83
+ /** Where the app looks for the bundles: `/sdcard/Android/data/com.asrlitertlm/files`. */
84
+ val modelDir: File
85
+ get() = modelDirectory
86
+
87
+ fun init(context: Context) {
88
+ synchronized(this) {
89
+ if (engine != null) return
90
+ appContext = context.applicationContext
91
+ modelDirectory = appContext.getExternalFilesDir(null) ?: appContext.filesDir
92
+ cacheDirectory = appContext.cacheDir
93
+ engine = AsrEngine(cacheDirectory)
94
+ }
95
+ }
96
+
97
+ fun bundleFile(profile: ModelProfile): File = File(modelDirectory, profile.fileName)
98
+
99
+ fun isPresent(profile: ModelProfile): Boolean = bundleFile(profile).isFile
100
+
101
+ /** The clip as a wav file the runtime can open (copied out of the APK once). */
102
+ fun clipFile(clip: Clip): File {
103
+ val file = File(cacheDirectory, "${clip.id}.wav")
104
+ if (!file.isFile) {
105
+ val partial = File(cacheDirectory, "${clip.id}.wav.part")
106
+ appContext.resources.openRawResource(clip.rawRes).use { input ->
107
+ partial.outputStream().use { input.copyTo(it) }
108
+ }
109
+ check(partial.renameTo(file)) { "could not write $file" }
110
+ }
111
+ return file
112
+ }
113
+
114
+ fun addListener(listener: (AsrEvent) -> Unit) {
115
+ listeners.add(listener)
116
+ }
117
+
118
+ fun removeListener(listener: (AsrEvent) -> Unit) {
119
+ listeners.remove(listener)
120
+ }
121
+
122
+ /** Loads [profile] (closing the bundle loaded before). [bundle] defaults to the file in [modelDir]. */
123
+ fun load(
124
+ profile: ModelProfile,
125
+ lmBackend: LmBackend = LmBackend.CPU,
126
+ bundle: File = bundleFile(profile),
127
+ ): Future<LoadInfo> =
128
+ worker
129
+ .submit(
130
+ Callable {
131
+ publish(AsrEvent.Loading(profile, lmBackend))
132
+ val asr = checkNotNull(engine) { "AsrSession.init() was not called" }
133
+ try {
134
+ asr.load(profile, bundle, lmBackend).also { info ->
135
+ current = info
136
+ Log.i(
137
+ TAG,
138
+ String.format(
139
+ Locale.US,
140
+ "LOAD model=%s backend=%s load_ms=%.1f initialize_ms=%.1f first_conversation_ms=%.1f bytes=%d",
141
+ profile.id,
142
+ lmBackend,
143
+ info.loadMs,
144
+ info.initializeMs,
145
+ info.firstConversationMs,
146
+ info.bundleBytes,
147
+ ),
148
+ )
149
+ publish(AsrEvent.Loaded(info))
150
+ }
151
+ } catch (e: AsrException) {
152
+ current = asr.loaded
153
+ Log.e(TAG, "LOAD_FAILED model=${profile.id} backend=$lmBackend: ${e.message}", e.cause)
154
+ publish(AsrEvent.LoadFailed(profile, e.message ?: e.toString()))
155
+ throw e
156
+ }
157
+ }
158
+ )
159
+ .also { lastLoad = it }
160
+
161
+ /**
162
+ * Transcribes a 16 kHz mono 16-bit wav with the loaded bundle; [source] names the input in events and logs
163
+ * ("clip:<id>", "mic"). Near-silent audio returns [TranscribeOutcome.NoSpeech] without calling the model.
164
+ */
165
+ fun transcribe(wav: File, source: String): Future<TranscribeOutcome> =
166
+ worker.submit(
167
+ Callable {
168
+ publish(AsrEvent.Transcribing(source))
169
+ val asr = checkNotNull(engine) { "AsrSession.init() was not called" }
170
+ try {
171
+ val pcm =
172
+ try {
173
+ Wav.pcm16(wav)
174
+ } catch (e: IOException) {
175
+ throw AsrException("Could not read ${wav.name}: ${e.message}", e)
176
+ } catch (e: RuntimeException) { // not a wav, no data chunk, not 16-bit
177
+ throw AsrException("Could not read ${wav.name}: ${e.message}", e)
178
+ }
179
+ val levels = Wav.levels(pcm.data)
180
+ val outcome =
181
+ if (levels.rmsDbfs < NO_SPEECH_RMS_DBFS) {
182
+ TranscribeOutcome.NoSpeech(levels, pcm.seconds)
183
+ } else {
184
+ TranscribeOutcome.Text(asr.transcribe(wav, File(cacheDirectory, "sent.wav")))
185
+ }
186
+ log(source, outcome)
187
+ publish(AsrEvent.Transcribed(source, outcome))
188
+ outcome
189
+ } catch (e: AsrException) {
190
+ Log.e(TAG, "TRANSCRIBE_FAILED source=$source: ${e.message}", e.cause)
191
+ publish(AsrEvent.TranscribeFailed(source, e.message ?: e.toString()))
192
+ throw e
193
+ }
194
+ }
195
+ )
196
+
197
+ /** Closes the loaded bundle; the future holds false when none was loaded (a second call is harmless). */
198
+ fun release(): Future<Boolean> =
199
+ worker.submit(
200
+ Callable {
201
+ val profile = current?.profile
202
+ val closed = checkNotNull(engine) { "AsrSession.init() was not called" }.release()
203
+ current = null
204
+ Log.i(TAG, "RELEASE model=${profile?.id} closed=$closed")
205
+ if (closed && profile != null) publish(AsrEvent.Released(profile))
206
+ closed
207
+ }
208
+ )
209
+
210
+ private fun log(source: String, outcome: TranscribeOutcome) {
211
+ when (outcome) {
212
+ is TranscribeOutcome.Text -> {
213
+ val transcript = outcome.transcript
214
+ Log.i(
215
+ TAG,
216
+ String.format(
217
+ Locale.US,
218
+ "TRANSCRIBED source=%s model=%s backend=%s audio_s=%.2f sent_s=%.2f send_ms=%.1f rtf=%.3f raw=%s",
219
+ source,
220
+ transcript.profile.id,
221
+ transcript.lmBackend,
222
+ transcript.audioSeconds,
223
+ transcript.sentSeconds,
224
+ transcript.sendMs,
225
+ transcript.rtf,
226
+ transcript.raw.replace('\n', ' '),
227
+ ),
228
+ )
229
+ }
230
+ is TranscribeOutcome.NoSpeech ->
231
+ Log.i(
232
+ TAG,
233
+ String.format(
234
+ Locale.US,
235
+ "NO_SPEECH source=%s seconds=%.2f rms_dbfs=%.1f peak_dbfs=%.1f",
236
+ source,
237
+ outcome.seconds,
238
+ outcome.levels.rmsDbfs,
239
+ outcome.levels.peakDbfs,
240
+ ),
241
+ )
242
+ }
243
+ }
244
+
245
+ private fun publish(event: AsrEvent) {
246
+ main.post { listeners.forEach { it(event) } }
247
+ }
248
+ }
android/app/src/main/kotlin/com/asrlitertlm/ClipPlayer.kt ADDED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ import android.media.AudioAttributes
4
+ import android.media.AudioFormat
5
+ import android.media.AudioTrack
6
+ import android.os.Handler
7
+ import android.os.Looper
8
+ import java.io.File
9
+
10
+ /**
11
+ * Plays one 16-bit wav through the phone's speaker. [onDone] runs on the main thread once the last sample has reached
12
+ * the mixer plus the output latency (about 0.2 s on the Galaxy S26), or with a sentence when playback failed.
13
+ */
14
+ class ClipPlayer {
15
+ private val main = Handler(Looper.getMainLooper())
16
+ private var track: AudioTrack? = null
17
+ private var finish: Runnable? = null
18
+
19
+ /** Starts [wav]; [onProgress] gets the played share (0..1) every 50 ms. Stops a clip that is still playing. */
20
+ fun play(wav: File, onProgress: (Float) -> Unit, onDone: (failure: String?) -> Unit) {
21
+ stop()
22
+ val newTrack: AudioTrack
23
+ val frames: Int
24
+ val rate: Int
25
+ try {
26
+ val pcm = Wav.pcm16(wav)
27
+ frames = pcm.frames
28
+ rate = pcm.sampleRate
29
+ newTrack =
30
+ AudioTrack.Builder()
31
+ .setAudioAttributes(
32
+ AudioAttributes.Builder()
33
+ .setUsage(AudioAttributes.USAGE_MEDIA)
34
+ .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
35
+ .build()
36
+ )
37
+ .setAudioFormat(
38
+ AudioFormat.Builder()
39
+ .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
40
+ .setSampleRate(pcm.sampleRate)
41
+ .setChannelMask(
42
+ if (pcm.channels == 1) AudioFormat.CHANNEL_OUT_MONO else AudioFormat.CHANNEL_OUT_STEREO
43
+ )
44
+ .build()
45
+ )
46
+ .setTransferMode(AudioTrack.MODE_STATIC)
47
+ .setBufferSizeInBytes(pcm.data.size)
48
+ .build()
49
+ check(newTrack.write(pcm.data, 0, pcm.data.size) == pcm.data.size) { "AudioTrack took a partial buffer" }
50
+ } catch (e: RuntimeException) {
51
+ onDone("Could not play ${wav.name}: ${e.message ?: e}")
52
+ return
53
+ }
54
+ val done = Runnable {
55
+ if (track === newTrack) {
56
+ stop()
57
+ onDone(null)
58
+ }
59
+ }
60
+ finish = done
61
+ track = newTrack
62
+ newTrack.positionNotificationPeriod = rate / PROGRESS_PER_SECOND
63
+ newTrack.notificationMarkerPosition = maxOf(1, frames - rate / MARKER_LEAD_PER_SECOND)
64
+ newTrack.setPlaybackPositionUpdateListener(
65
+ object : AudioTrack.OnPlaybackPositionUpdateListener {
66
+ override fun onMarkerReached(t: AudioTrack) {
67
+ main.removeCallbacks(done)
68
+ main.postDelayed(done, OUTPUT_LATENCY_MS)
69
+ }
70
+
71
+ override fun onPeriodicNotification(t: AudioTrack) {
72
+ if (track === t) onProgress((t.playbackHeadPosition.toFloat() / frames).coerceIn(0f, 1f))
73
+ }
74
+ },
75
+ main,
76
+ )
77
+ newTrack.play()
78
+ // A marker that never fires (a muted or rerouted output) still ends the clip.
79
+ main.postDelayed(done, frames * 1000L / rate + FALLBACK_SLACK_MS)
80
+ }
81
+
82
+ /** Stops and frees the current clip without calling its onDone. */
83
+ fun stop() {
84
+ finish?.let { main.removeCallbacks(it) }
85
+ finish = null
86
+ track?.let {
87
+ runCatching { it.stop() }
88
+ it.release()
89
+ }
90
+ track = null
91
+ }
92
+
93
+ private companion object {
94
+ const val PROGRESS_PER_SECOND = 20
95
+ const val MARKER_LEAD_PER_SECOND = 50 // the marker sits 20 ms before the last frame
96
+ const val OUTPUT_LATENCY_MS = 250L
97
+ const val FALLBACK_SLACK_MS = 3_000L
98
+ }
99
+ }
android/app/src/main/kotlin/com/asrlitertlm/MainActivity.kt ADDED
@@ -0,0 +1,712 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ import android.Manifest
4
+ import android.annotation.SuppressLint
5
+ import android.app.Activity
6
+ import android.content.Intent
7
+ import android.content.pm.PackageManager
8
+ import android.graphics.Typeface
9
+ import android.graphics.drawable.GradientDrawable
10
+ import android.os.Build
11
+ import android.os.Bundle
12
+ import android.os.Handler
13
+ import android.os.Looper
14
+ import android.text.SpannableStringBuilder
15
+ import android.text.Spanned
16
+ import android.text.style.ForegroundColorSpan
17
+ import android.text.style.StyleSpan
18
+ import android.util.Log
19
+ import android.util.TypedValue
20
+ import android.view.Gravity
21
+ import android.view.MotionEvent
22
+ import android.view.View
23
+ import android.view.ViewGroup
24
+ import android.view.WindowInsets
25
+ import android.view.WindowManager
26
+ import android.widget.LinearLayout
27
+ import android.widget.ScrollView
28
+ import android.widget.TextView
29
+ import java.io.File
30
+ import java.util.Locale
31
+ import java.util.concurrent.CompletableFuture
32
+ import java.util.concurrent.Future
33
+
34
+ /** How a microphone test ended (see [MainActivity.runMicTest]). */
35
+ sealed interface MicTestResult {
36
+ data class Finished(val outcome: TranscribeOutcome) : MicTestResult
37
+
38
+ data class Failed(val message: String) : MicTestResult
39
+ }
40
+
41
+ /**
42
+ * Speech to text on the phone with one of three LiteRT-LM bundles: pick a model, hold the microphone button and speak
43
+ * (up to 30 s), or play one of three bundled clips. The bundle's language model and audio encoder run on the CPU
44
+ * (4 threads); every number on screen is measured by this app: "engine load" = Engine.initialize() plus the first
45
+ * conversation, "transcribe" = the wall time of sendMessage(), RTF = that time over the clip length.
46
+ *
47
+ * The bundles are read from /sdcard/Android/data/com.asrlitertlm/files/<file name> (adb push, see README). Launch
48
+ * extras, for recordings and the device check (all optional):
49
+ * ```
50
+ * --es model <id> qwen3-asr-1.7b | fun-asr-nano-2512 | confucius4-r2t2 (default: the last one used)
51
+ * --es backend gpu language model on the GPU (audio encoder stays on the CPU)
52
+ * --ez autoplay true after READY: wait delay_ms, then clips zh -> en -> ja (each played, then transcribed)
53
+ * --ei delay_ms 3000 with gap_ms between one transcript and the next clip
54
+ * --ei gap_ms 2500
55
+ * --ei mic_test_ms 9000 after READY and delay_ms: record that long without touching the screen, then transcribe
56
+ * --ez mic_test_play true and play clip en through the speaker meanwhile (needs RECORD_AUDIO granted already)
57
+ * ```
58
+ * Logs go to the tag AsrLitertlm: one `STATE <name>` line per screen state, `LOAD` and `TRANSCRIBED` lines with the
59
+ * measured times.
60
+ */
61
+ class MainActivity : Activity() {
62
+ private val main = Handler(Looper.getMainLooper())
63
+ private val player = ClipPlayer()
64
+ private lateinit var recorder: MicRecorder
65
+ private val onEvent: (AsrEvent) -> Unit = { handle(it) }
66
+
67
+ private var busy = false
68
+ private var lmBackend = LmBackend.CPU
69
+ private var selected: ModelProfile? = null
70
+ private var micCallback: ((MicTestResult) -> Unit)? = null
71
+ private var extrasPending = false
72
+ private var autoplayIndex = -1
73
+ private var pendingNotice: String? = null
74
+
75
+ private lateinit var pill: TextView
76
+ private lateinit var engineLine: TextView
77
+ private lateinit var pickerRows: Map<ModelProfile, TextView>
78
+ private lateinit var sourceLine: TextView
79
+ private lateinit var bar: PlayBar
80
+ private lateinit var body: TextView
81
+ private lateinit var languageLine: TextView
82
+ private lateinit var timeLine: TextView
83
+ private lateinit var deviceLine: TextView
84
+ private lateinit var clipButtons: List<TextView>
85
+ private lateinit var micButton: TextView
86
+
87
+ private val deviceName: String
88
+ get() = if (Build.MODEL.startsWith("SM-S942")) "Galaxy S26" else Build.MODEL
89
+
90
+ override fun onCreate(savedInstanceState: Bundle?) {
91
+ super.onCreate(savedInstanceState)
92
+ // Show over the lock screen and keep the display on: a hidden activity's process moves to the background CPU
93
+ // set (little cores), where the same model runs about 10x slower.
94
+ setShowWhenLocked(true)
95
+ setTurnScreenOn(true)
96
+ window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
97
+ AsrSession.init(this)
98
+ recorder = MicRecorder(this)
99
+ setContentView(buildUi())
100
+ AsrSession.addListener(onEvent)
101
+ startFrom(intent)
102
+ }
103
+
104
+ override fun onNewIntent(intent: Intent) {
105
+ super.onNewIntent(intent)
106
+ setIntent(intent)
107
+ if (!busy) startFrom(intent)
108
+ }
109
+
110
+ override fun onResume() {
111
+ super.onResume()
112
+ refreshPicker()
113
+ }
114
+
115
+ override fun onDestroy() {
116
+ AsrSession.removeListener(onEvent)
117
+ recorder.stop()
118
+ player.stop()
119
+ main.removeCallbacksAndMessages(null)
120
+ // The engine holds gigabytes: free it when the screen goes away for good.
121
+ if (isFinishing) AsrSession.release()
122
+ super.onDestroy()
123
+ }
124
+
125
+ // ---- model --------------------------------------------------------------------------------------------------
126
+
127
+ /** Picks the model from the launch extras, the last one used, or the first on the phone, and loads it. */
128
+ private fun startFrom(launch: Intent) {
129
+ lmBackend = if (launch.getStringExtra(EXTRA_BACKEND) == "gpu") LmBackend.GPU else LmBackend.CPU
130
+ deviceLine.text = statusLine()
131
+ extrasPending = launch.getBooleanExtra(EXTRA_AUTOPLAY, false) || launch.getIntExtra(EXTRA_MIC_TEST_MS, 0) > 0
132
+ val requested = ModelProfile.byId(launch.getStringExtra(EXTRA_MODEL))
133
+ val saved = ModelProfile.byId(getPreferences(MODE_PRIVATE).getString(PREF_MODEL, null))
134
+ val choice =
135
+ listOfNotNull(requested, saved).firstOrNull { AsrSession.isPresent(it) }
136
+ ?: ModelProfile.ALL.firstOrNull { AsrSession.isPresent(it) }
137
+ when {
138
+ choice == null -> showNoModel()
139
+ requested != null && choice != requested -> {
140
+ // Shown once the model that is on the phone is ready (the load clears the screen).
141
+ pendingNotice = AsrEngine.missingBundleMessage(requested, AsrSession.modelDir)
142
+ selectModel(choice)
143
+ }
144
+ else -> selectModel(choice)
145
+ }
146
+ }
147
+
148
+ /**
149
+ * Loads [profile] (the picker's tap). Returns the load, or null when the bundle is not on the phone (the screen
150
+ * then says which file to push where). A model that is already loaded is not loaded again.
151
+ */
152
+ fun selectModel(profile: ModelProfile, backend: LmBackend = lmBackend): Future<LoadInfo>? {
153
+ if (!AsrSession.isPresent(profile)) {
154
+ refreshPicker()
155
+ showText(AsrEngine.missingBundleMessage(profile, AsrSession.modelDir))
156
+ return null
157
+ }
158
+ selected = profile
159
+ lmBackend = backend
160
+ getPreferences(MODE_PRIVATE).edit().putString(PREF_MODEL, profile.id).apply()
161
+ refreshPicker()
162
+ val current = AsrSession.current
163
+ if (current != null && current.profile == profile && current.lmBackend == backend) {
164
+ showReady(current)
165
+ return CompletableFuture.completedFuture(current)
166
+ }
167
+ setBusy(true)
168
+ return AsrSession.load(profile, backend)
169
+ }
170
+
171
+ private fun handle(event: AsrEvent) {
172
+ when (event) {
173
+ is AsrEvent.Loading -> {
174
+ setPill("LOADING", C_IDLE)
175
+ engineLine.text =
176
+ String.format(
177
+ Locale.US,
178
+ "loading %s (%.2f GB) · LM %s",
179
+ event.profile.displayName,
180
+ AsrSession.bundleFile(event.profile).length() / 1e9,
181
+ backendLabel(event.lmBackend),
182
+ )
183
+ clearResult()
184
+ }
185
+ is AsrEvent.Loaded -> {
186
+ setBusy(false)
187
+ showReady(event.info)
188
+ }
189
+ is AsrEvent.LoadFailed -> {
190
+ setBusy(false)
191
+ setPill("ERROR", C_LIVE)
192
+ // A missing file leaves the model loaded before in place.
193
+ engineLine.text = AsrSession.current?.let { "${it.profile.displayName} still loaded" } ?: "no model loaded"
194
+ showText(event.message)
195
+ }
196
+ is AsrEvent.Transcribing -> setPill("TRANSCRIBING", C_WORK)
197
+ is AsrEvent.Transcribed -> {
198
+ show(event.outcome)
199
+ setBusy(false)
200
+ if (event.source == SOURCE_MIC) finishMicTest(MicTestResult.Finished(event.outcome))
201
+ continueAutoplay(event.source)
202
+ }
203
+ is AsrEvent.TranscribeFailed -> {
204
+ setBusy(false)
205
+ setPill("ERROR", C_LIVE)
206
+ showText(event.message)
207
+ if (event.source == SOURCE_MIC) finishMicTest(MicTestResult.Failed(event.message))
208
+ autoplayIndex = -1
209
+ }
210
+ is AsrEvent.Released -> {
211
+ setPill("RELEASED", C_IDLE)
212
+ engineLine.text = "${event.profile.displayName} closed"
213
+ setBusy(busy)
214
+ }
215
+ }
216
+ }
217
+
218
+ private fun showReady(info: LoadInfo) {
219
+ selected = info.profile
220
+ setPill("READY", C_IDLE)
221
+ engineLine.text =
222
+ String.format(
223
+ Locale.US,
224
+ "engine load %.2f s · LM %s",
225
+ info.loadMs / 1000,
226
+ backendLabel(info.lmBackend),
227
+ )
228
+ deviceLine.text = statusLine()
229
+ refreshPicker()
230
+ pendingNotice?.let {
231
+ pendingNotice = null
232
+ showText(it)
233
+ }
234
+ if (extrasPending) {
235
+ extrasPending = false
236
+ runLaunchExtras()
237
+ }
238
+ }
239
+
240
+ private fun showNoModel() {
241
+ setPill("NO MODEL", C_LIVE)
242
+ engineLine.text = "no model on this phone"
243
+ val files = ModelProfile.ALL.joinToString("\n") { "· ${it.fileName} (${it.hubRepo})" }
244
+ showText("Push at least one bundle to ${AsrSession.modelDir.path}/ with adb, then reopen the app:\n$files")
245
+ refreshPicker()
246
+ }
247
+
248
+ // ---- clips --------------------------------------------------------------------------------------------------
249
+
250
+ /** A bundled clip: play it through the speaker, then transcribe it. */
251
+ private fun playClip(index: Int) {
252
+ if (busy || AsrSession.current == null) return
253
+ val clip = Clip.ALL[index]
254
+ val file = AsrSession.clipFile(clip)
255
+ setBusy(true)
256
+ clearResult()
257
+ setPill("● PLAYING", C_LIVE)
258
+ sourceLine.text = String.format(Locale.US, "clip %s · %.1f s", clip.label, Wav.seconds(file))
259
+ bar.set(0f, C_LIVE)
260
+ bar.visibility = View.VISIBLE
261
+ player.play(
262
+ file,
263
+ onProgress = { bar.fraction = it },
264
+ onDone = { failure ->
265
+ bar.set(1f, C_IDLE)
266
+ if (failure != null) {
267
+ setBusy(false)
268
+ setPill("ERROR", C_LIVE)
269
+ showText(failure)
270
+ autoplayIndex = -1
271
+ } else {
272
+ AsrSession.transcribe(file, SOURCE_CLIP + clip.id)
273
+ }
274
+ },
275
+ )
276
+ }
277
+
278
+ private fun runLaunchExtras() {
279
+ val delay = intent.getIntExtra(EXTRA_DELAY_MS, 3000).toLong()
280
+ val micTestMs = intent.getIntExtra(EXTRA_MIC_TEST_MS, 0)
281
+ if (intent.getBooleanExtra(EXTRA_AUTOPLAY, false)) {
282
+ Log.i(TAG, "AUTOPLAY_START delay_ms=$delay")
283
+ autoplayIndex = 0
284
+ main.postDelayed({ playClip(0) }, delay)
285
+ } else if (micTestMs > 0) {
286
+ val play = intent.getBooleanExtra(EXTRA_MIC_TEST_PLAY, false)
287
+ main.postDelayed({ runMicTest(micTestMs.toLong(), play) }, delay)
288
+ }
289
+ }
290
+
291
+ private fun continueAutoplay(source: String) {
292
+ if (autoplayIndex < 0 || !source.startsWith(SOURCE_CLIP)) return
293
+ autoplayIndex++
294
+ if (autoplayIndex >= Clip.ALL.size) {
295
+ autoplayIndex = -1
296
+ Log.i(TAG, "AUTOPLAY_DONE")
297
+ return
298
+ }
299
+ val next = autoplayIndex
300
+ main.postDelayed({ playClip(next) }, intent.getIntExtra(EXTRA_GAP_MS, 2500).toLong())
301
+ }
302
+
303
+ // ---- microphone ---------------------------------------------------------------------------------------------
304
+
305
+ /** Hold to talk: recording starts on press and ends on release (or at 30 s). */
306
+ @SuppressLint("ClickableViewAccessibility") // the button also takes a click: performClick() on release
307
+ private fun onMicTouch(view: View, event: MotionEvent): Boolean {
308
+ when (event.actionMasked) {
309
+ MotionEvent.ACTION_DOWN -> {
310
+ if (busy || AsrSession.current == null) return true
311
+ if (!recorder.hasPermission()) {
312
+ requestPermissions(arrayOf(Manifest.permission.RECORD_AUDIO), REQUEST_MIC)
313
+ return true
314
+ }
315
+ startRecording(playClip = false)
316
+ }
317
+ MotionEvent.ACTION_UP -> {
318
+ view.performClick()
319
+ recorder.stop()
320
+ }
321
+ MotionEvent.ACTION_CANCEL -> recorder.stop()
322
+ }
323
+ return true
324
+ }
325
+
326
+ override fun onRequestPermissionsResult(requestCode: Int, permissions: Array<out String>, grantResults: IntArray) {
327
+ super.onRequestPermissionsResult(requestCode, permissions, grantResults)
328
+ if (requestCode != REQUEST_MIC) return
329
+ if (grantResults.firstOrNull() == PackageManager.PERMISSION_GRANTED) {
330
+ showText("Microphone allowed. Hold the button and speak.")
331
+ } else {
332
+ setPill("NO MIC", C_LIVE)
333
+ showText(MicRecorder.PERMISSION_MESSAGE)
334
+ }
335
+ }
336
+
337
+ /**
338
+ * The microphone path without touching the screen: record for [durationMs] (playing clip en through the speaker
339
+ * when [playClip]), then transcribe like the button does. [onFinished] gets the outcome or the failure sentence;
340
+ * without the permission the sentence comes back at once and nothing is recorded.
341
+ */
342
+ fun runMicTest(durationMs: Long, playClip: Boolean, onFinished: ((MicTestResult) -> Unit)? = null) {
343
+ Log.i(TAG, "MIC_TEST ms=$durationMs play=$playClip")
344
+ micCallback = onFinished
345
+ if (!recorder.hasPermission()) {
346
+ setPill("NO MIC", C_LIVE)
347
+ showText(MicRecorder.PERMISSION_MESSAGE)
348
+ finishMicTest(MicTestResult.Failed(MicRecorder.PERMISSION_MESSAGE))
349
+ return
350
+ }
351
+ if (busy || AsrSession.current == null) {
352
+ finishMicTest(MicTestResult.Failed("The app is busy or no model is loaded."))
353
+ return
354
+ }
355
+ if (!startRecording(playClip)) return
356
+ main.postDelayed({ recorder.stop() }, durationMs)
357
+ }
358
+
359
+ private fun startRecording(playClip: Boolean): Boolean {
360
+ val wav = File(cacheDir, "mic.wav")
361
+ val failure =
362
+ recorder.start(
363
+ wav,
364
+ onProgress = { seconds -> main.post { showRecording(seconds) } },
365
+ onDone = { result -> main.post { onRecorded(result) } },
366
+ )
367
+ if (failure != null) {
368
+ setPill("ERROR", C_LIVE)
369
+ showText(failure)
370
+ finishMicTest(MicTestResult.Failed(failure))
371
+ return false
372
+ }
373
+ setBusy(true)
374
+ micButton.isEnabled = true
375
+ micButton.alpha = 1f
376
+ clearResult()
377
+ setPill("● LISTENING", C_LIVE)
378
+ showRecording(0.0)
379
+ bar.set(0f, C_LIVE)
380
+ bar.visibility = View.VISIBLE
381
+ if (playClip) {
382
+ val clip = AsrSession.clipFile(Clip.ALL[1])
383
+ main.postDelayed({ player.play(clip, onProgress = {}, onDone = {}) }, MIC_TEST_PLAY_DELAY_MS)
384
+ }
385
+ return true
386
+ }
387
+
388
+ private fun showRecording(seconds: Double) {
389
+ sourceLine.text = String.format(Locale.US, "mic · %.1f s of %d s", seconds, MicRecorder.MAX_SECONDS)
390
+ bar.fraction = (seconds / MicRecorder.MAX_SECONDS).toFloat()
391
+ micButton.text = String.format(Locale.US, "Release to transcribe · %.1f s", seconds)
392
+ }
393
+
394
+ private fun onRecorded(result: MicResult) {
395
+ micButton.text = MIC_LABEL
396
+ bar.set(1f, C_IDLE)
397
+ when (result) {
398
+ is MicResult.Failed -> {
399
+ setBusy(false)
400
+ setPill("ERROR", C_LIVE)
401
+ showText(result.message)
402
+ finishMicTest(MicTestResult.Failed(result.message))
403
+ }
404
+ is MicResult.Recorded -> {
405
+ Log.i(
406
+ TAG,
407
+ String.format(
408
+ Locale.US,
409
+ "MIC_RECORDED seconds=%.2f peak_dbfs=%.1f rms_dbfs=%.1f at_limit=%b",
410
+ result.seconds,
411
+ result.levels.peakDbfs,
412
+ result.levels.rmsDbfs,
413
+ result.stoppedAtLimit,
414
+ ),
415
+ )
416
+ sourceLine.text =
417
+ String.format(Locale.US, "mic · %.1f s", result.seconds) +
418
+ if (result.stoppedAtLimit) " · stopped at the ${MicRecorder.MAX_SECONDS} s limit" else ""
419
+ setBusy(true)
420
+ AsrSession.transcribe(result.wav, SOURCE_MIC)
421
+ }
422
+ }
423
+ }
424
+
425
+ private fun finishMicTest(result: MicTestResult) {
426
+ val callback = micCallback ?: return
427
+ micCallback = null
428
+ callback(result)
429
+ }
430
+
431
+ // ---- screen -------------------------------------------------------------------------------------------------
432
+
433
+ private fun show(outcome: TranscribeOutcome) {
434
+ when (outcome) {
435
+ is TranscribeOutcome.NoSpeech -> {
436
+ setPill("NO SPEECH", C_IDLE)
437
+ body.text = "No speech heard."
438
+ languageLine.text =
439
+ String.format(
440
+ Locale.US,
441
+ "The recording peaked at %.0f dBFS (RMS %.0f dBFS); the model was not called.",
442
+ outcome.levels.peakDbfs,
443
+ outcome.levels.rmsDbfs,
444
+ )
445
+ timeLine.text = ""
446
+ }
447
+ is TranscribeOutcome.Text -> {
448
+ val transcript = outcome.transcript
449
+ val answer = transcript.answer
450
+ setPill("DONE", C_DONE)
451
+ body.textLocale = localeFor(answer)
452
+ body.text = answer.text.ifEmpty { "(no speech in the answer)" }
453
+ languageLine.text =
454
+ when {
455
+ transcript.profile.answerFormat == AnswerFormat.PLAIN -> ""
456
+ answer.language.isEmpty() -> "detected language: none"
457
+ else -> "detected language: ${answer.language}"
458
+ }
459
+ val numbers =
460
+ String.format(
461
+ Locale.US,
462
+ "transcribe %.2f s / clip %.2f s = RTF %.2f",
463
+ transcript.sendMs / 1000,
464
+ transcript.sentSeconds,
465
+ transcript.rtf,
466
+ )
467
+ timeLine.text =
468
+ SpannableStringBuilder(numbers).apply {
469
+ setSpan(ForegroundColorSpan(C_TEXT), 0, numbers.length, Spanned.SPAN_EXCLUSIVE_EXCLUSIVE)
470
+ setSpan(StyleSpan(Typeface.BOLD), 0, numbers.length, Spanned.SPAN_EXCLUSIVE_EXCLUSIVE)
471
+ }
472
+ }
473
+ }
474
+ }
475
+
476
+ private fun showText(text: String) {
477
+ body.textLocale = Locale.US
478
+ body.text = text
479
+ languageLine.text = ""
480
+ timeLine.text = ""
481
+ }
482
+
483
+ private fun clearResult() {
484
+ sourceLine.text = ""
485
+ body.text = ""
486
+ languageLine.text = ""
487
+ timeLine.text = ""
488
+ bar.visibility = View.INVISIBLE
489
+ }
490
+
491
+ /** The glyph forms of Han characters follow the language the model named, or the script of the text. */
492
+ private fun localeFor(answer: Answer): Locale =
493
+ when {
494
+ answer.language == "Japanese" || answer.text.any { it in '぀'..'ヿ' } -> Locale.JAPANESE
495
+ answer.language == "Cantonese" -> Locale.TRADITIONAL_CHINESE
496
+ answer.language == "Chinese" ||
497
+ answer.text.any { Character.UnicodeScript.of(it.code) == Character.UnicodeScript.HAN } ->
498
+ Locale.SIMPLIFIED_CHINESE
499
+ else -> Locale.US
500
+ }
501
+
502
+ private fun refreshPicker() {
503
+ if (!::pickerRows.isInitialized) return
504
+ val loaded = AsrSession.current?.profile
505
+ for ((profile, row) in pickerRows) {
506
+ val present = AsrSession.isPresent(profile)
507
+ val bytes = if (present) AsrSession.bundleFile(profile).length() else profile.hubBytes
508
+ val mark = if (profile == selected) "● " else "○ "
509
+ val state =
510
+ when {
511
+ !present -> "missing"
512
+ profile == loaded -> "loaded"
513
+ else -> "on phone"
514
+ }
515
+ val head = mark + profile.displayName
516
+ val text = String.format(Locale.US, "%s %.2f GB · %s", head, bytes / 1e9, state)
517
+ row.text =
518
+ SpannableStringBuilder(text).apply {
519
+ setSpan(StyleSpan(Typeface.BOLD), 0, head.length, Spanned.SPAN_EXCLUSIVE_EXCLUSIVE)
520
+ }
521
+ row.setTextColor(if (present) C_TEXT else C_SUB)
522
+ row.background = rounded(if (profile == selected) C_SELECTED else C_BUTTON, C_BUTTON_EDGE)
523
+ row.isEnabled = !busy
524
+ }
525
+ }
526
+
527
+ private fun setBusy(on: Boolean) {
528
+ busy = on
529
+ val ready = !on && AsrSession.current != null
530
+ for (button in clipButtons + micButton) {
531
+ button.isEnabled = ready
532
+ button.alpha = if (ready) 1f else 0.4f
533
+ }
534
+ refreshPicker()
535
+ }
536
+
537
+ private fun setPill(label: String, color: Int) {
538
+ pill.text = label
539
+ pill.background = GradientDrawable().apply {
540
+ cornerRadius = dp(999f)
541
+ setColor(color)
542
+ }
543
+ Log.i(TAG, "STATE ${label.removePrefix("● ").replace(' ', '_')}")
544
+ }
545
+
546
+ private fun statusLine(): String =
547
+ String.format(
548
+ Locale.US,
549
+ "%s · LiteRT-LM %s · LM %s, audio CPU",
550
+ deviceName,
551
+ BuildConfig.LITERTLM_VERSION,
552
+ if (lmBackend == LmBackend.GPU) "GPU" else "CPU",
553
+ )
554
+
555
+ private fun backendLabel(backend: LmBackend): String =
556
+ if (backend == LmBackend.GPU) "GPU" else "CPU ${AsrEngine.CPU_THREADS} threads"
557
+
558
+ private fun buildUi(): View {
559
+ val side = dp(20f).toInt()
560
+ val root =
561
+ LinearLayout(this).apply {
562
+ orientation = LinearLayout.VERTICAL
563
+ setBackgroundColor(C_BG)
564
+ setOnApplyWindowInsetsListener { view, insets ->
565
+ val bars = insets.getInsets(WindowInsets.Type.systemBars() or WindowInsets.Type.displayCutout())
566
+ view.setPadding(side + bars.left, dp(14f).toInt() + bars.top, side + bars.right, dp(14f).toInt() + bars.bottom)
567
+ insets
568
+ }
569
+ }
570
+ val title = label(15f, C_TEXT, bold = true).apply { text = "Speech to text on the phone · LiteRT-LM" }
571
+ val picker = LinearLayout(this).apply { orientation = LinearLayout.VERTICAL }
572
+ pickerRows =
573
+ ModelProfile.ALL.associateWith { profile ->
574
+ label(14f, C_TEXT).apply {
575
+ setPadding(dp(14f).toInt(), dp(10f).toInt(), dp(14f).toInt(), dp(10f).toInt())
576
+ maxLines = 1
577
+ setOnClickListener { if (!busy) selectModel(profile) }
578
+ picker.addView(this, layoutParams(top = 8f))
579
+ }
580
+ }
581
+ pill =
582
+ label(13f, C_TEXT, bold = true).apply {
583
+ letterSpacing = 0.08f
584
+ gravity = Gravity.CENTER_VERTICAL
585
+ setPadding(dp(12f).toInt(), 0, dp(12f).toInt(), 0)
586
+ }
587
+ engineLine = label(14f, C_SUB)
588
+ sourceLine = label(17f, C_SUB)
589
+ bar = PlayBar(this, C_BUTTON_EDGE).apply { visibility = View.INVISIBLE }
590
+ body = label(24f, C_TEXT).apply { setLineSpacing(0f, 1.1f) }
591
+ languageLine = label(17f, C_TEXT)
592
+ timeLine = label(15f, C_SUB)
593
+ deviceLine = label(13f, C_SUB)
594
+ val attribution = label(13f, C_SUB).apply { text = ATTRIBUTION }
595
+
596
+ val result =
597
+ LinearLayout(this).apply {
598
+ orientation = LinearLayout.VERTICAL
599
+ addView(sourceLine, layoutParams(top = 14f))
600
+ addView(
601
+ bar,
602
+ LinearLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, dp(6f).toInt()).apply {
603
+ topMargin = dp(10f).toInt()
604
+ },
605
+ )
606
+ addView(body, layoutParams(top = 14f))
607
+ addView(languageLine, layoutParams(top = 14f))
608
+ }
609
+ val scroll =
610
+ ScrollView(this).apply {
611
+ isVerticalScrollBarEnabled = false
612
+ addView(result)
613
+ }
614
+ val clipRow = LinearLayout(this).apply { orientation = LinearLayout.HORIZONTAL }
615
+ clipButtons =
616
+ Clip.ALL.mapIndexed { index, clip ->
617
+ val seconds = Wav.seconds(AsrSession.clipFile(clip))
618
+ button(String.format(Locale.US, "%s · %.1f s", clip.label, seconds)) { playClip(index) }
619
+ .also {
620
+ clipRow.addView(
621
+ it,
622
+ LinearLayout.LayoutParams(0, dp(52f).toInt(), 1f).apply {
623
+ if (index > 0) marginStart = dp(10f).toInt()
624
+ },
625
+ )
626
+ }
627
+ }
628
+ micButton = button(MIC_LABEL) {}.apply { setOnTouchListener { view, event -> onMicTouch(view, event) } }
629
+ setBusy(false)
630
+
631
+ root.addView(title)
632
+ root.addView(picker, layoutParams(top = 4f))
633
+ root.addView(
634
+ pill,
635
+ LinearLayout.LayoutParams(ViewGroup.LayoutParams.WRAP_CONTENT, dp(28f).toInt()).apply {
636
+ topMargin = dp(14f).toInt()
637
+ },
638
+ )
639
+ root.addView(engineLine, layoutParams(top = 6f))
640
+ root.addView(scroll, LinearLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, 0, 1f))
641
+ root.addView(clipRow, layoutParams(top = 10f))
642
+ root.addView(
643
+ micButton,
644
+ LinearLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, dp(64f).toInt()).apply {
645
+ topMargin = dp(10f).toInt()
646
+ },
647
+ )
648
+ // Footer: the measured seconds, then device, runtime and backend, then the clips' source.
649
+ root.addView(timeLine, layoutParams(top = 12f))
650
+ root.addView(deviceLine, layoutParams(top = 4f))
651
+ root.addView(attribution, layoutParams(top = 2f))
652
+ return root
653
+ }
654
+
655
+ private fun layoutParams(top: Float) =
656
+ LinearLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.WRAP_CONTENT).apply {
657
+ topMargin = dp(top).toInt()
658
+ }
659
+
660
+ private fun label(sizeSp: Float, color: Int, bold: Boolean = false) =
661
+ TextView(this).apply {
662
+ setTextSize(TypedValue.COMPLEX_UNIT_SP, sizeSp)
663
+ setTextColor(color)
664
+ if (bold) typeface = Typeface.create(Typeface.DEFAULT, Typeface.BOLD)
665
+ }
666
+
667
+ private fun button(text: String, onClick: () -> Unit) =
668
+ label(16f, C_TEXT, bold = true).apply {
669
+ this.text = text
670
+ gravity = Gravity.CENTER
671
+ background = rounded(C_BUTTON, C_BUTTON_EDGE)
672
+ setOnClickListener { onClick() }
673
+ }
674
+
675
+ private fun rounded(fill: Int, edge: Int) =
676
+ GradientDrawable().apply {
677
+ cornerRadius = dp(14f)
678
+ setColor(fill)
679
+ setStroke(dp(1f).toInt(), edge)
680
+ }
681
+
682
+ private fun dp(value: Float) = TypedValue.applyDimension(TypedValue.COMPLEX_UNIT_DIP, value, resources.displayMetrics)
683
+
684
+ companion object {
685
+ private const val TAG = "AsrLitertlm"
686
+ const val EXTRA_MODEL = "model"
687
+ const val EXTRA_BACKEND = "backend"
688
+ const val EXTRA_AUTOPLAY = "autoplay"
689
+ const val EXTRA_DELAY_MS = "delay_ms"
690
+ const val EXTRA_GAP_MS = "gap_ms"
691
+ const val EXTRA_MIC_TEST_MS = "mic_test_ms"
692
+ const val EXTRA_MIC_TEST_PLAY = "mic_test_play"
693
+ const val SOURCE_MIC = "mic"
694
+ const val SOURCE_CLIP = "clip:"
695
+ private const val PREF_MODEL = "model"
696
+ private const val REQUEST_MIC = 1
697
+ private const val MIC_TEST_PLAY_DELAY_MS = 500L
698
+ private const val MIC_LABEL = "Hold to talk (up to 30 s)"
699
+ // The clips are FLEURS recordings with one linear gain each: CC BY 4.0 asks to say so.
700
+ private const val ATTRIBUTION = "clips: FLEURS #1698, CC BY 4.0, level-normalized"
701
+ private const val C_BG = 0xFF0E1116.toInt()
702
+ private const val C_IDLE = 0xFF5F6368.toInt()
703
+ private const val C_LIVE = 0xFFE53935.toInt()
704
+ private const val C_WORK = 0xFF1565C0.toInt()
705
+ private const val C_DONE = 0xFF2E7D32.toInt()
706
+ private const val C_TEXT = 0xFFE6E8EB.toInt()
707
+ private const val C_SUB = 0xFF8A919C.toInt()
708
+ private const val C_BUTTON = 0xFF1A1F27.toInt()
709
+ private const val C_SELECTED = 0xFF1E3A5F.toInt()
710
+ private const val C_BUTTON_EDGE = 0xFF2C333D.toInt()
711
+ }
712
+ }
android/app/src/main/kotlin/com/asrlitertlm/MicRecorder.kt ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ import android.Manifest
4
+ import android.annotation.SuppressLint
5
+ import android.content.Context
6
+ import android.content.pm.PackageManager
7
+ import android.media.AudioFormat
8
+ import android.media.AudioRecord
9
+ import android.media.MediaRecorder
10
+ import java.io.ByteArrayOutputStream
11
+ import java.io.File
12
+ import kotlin.concurrent.thread
13
+
14
+ /** The end of one recording. */
15
+ sealed interface MicResult {
16
+ /** A wav of 16 kHz mono 16-bit audio. [stoppedAtLimit] is true when the 30 s limit ended it. */
17
+ data class Recorded(val wav: File, val seconds: Double, val levels: Wav.Levels, val stoppedAtLimit: Boolean) :
18
+ MicResult
19
+
20
+ /** A sentence for the screen. */
21
+ data class Failed(val message: String) : MicResult
22
+ }
23
+
24
+ /**
25
+ * Records 16 kHz mono 16-bit audio (what the three models take) from the microphone, on its own thread, until
26
+ * [stop] is called or [MAX_SECONDS] have been recorded.
27
+ */
28
+ class MicRecorder(private val context: Context) {
29
+ @Volatile private var stopRequested = false
30
+
31
+ @Volatile
32
+ var isRecording = false
33
+ private set
34
+
35
+ fun hasPermission(): Boolean =
36
+ context.checkSelfPermission(Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED
37
+
38
+ /**
39
+ * Starts a recording into [output]. Returns a failure sentence at once (and records nothing) when the microphone
40
+ * permission is missing or a recording is running; otherwise returns null, reports the seconds recorded so far to
41
+ * [onProgress] and the result to [onDone], both on the recording thread.
42
+ */
43
+ fun start(output: File, onProgress: (Double) -> Unit, onDone: (MicResult) -> Unit): String? {
44
+ if (!hasPermission()) return PERMISSION_MESSAGE
45
+ if (isRecording) return "A recording is already running."
46
+ stopRequested = false
47
+ isRecording = true
48
+ thread(name = "mic-recorder") {
49
+ val result =
50
+ try {
51
+ record(output, onProgress)
52
+ } catch (e: SecurityException) {
53
+ MicResult.Failed(PERMISSION_MESSAGE)
54
+ } catch (e: RuntimeException) {
55
+ MicResult.Failed("Recording failed: ${e.message ?: e}")
56
+ }
57
+ isRecording = false
58
+ onDone(result)
59
+ }
60
+ return null
61
+ }
62
+
63
+ /** Ends the recording; [start]'s onDone follows. */
64
+ fun stop() {
65
+ stopRequested = true
66
+ }
67
+
68
+ @SuppressLint("MissingPermission") // checked in start(); a revocation in between ends in SecurityException
69
+ private fun record(output: File, onProgress: (Double) -> Unit): MicResult {
70
+ val minBuffer =
71
+ AudioRecord.getMinBufferSize(SAMPLE_RATE, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT)
72
+ if (minBuffer <= 0) return MicResult.Failed("This phone cannot record 16 kHz mono audio.")
73
+ val recorder =
74
+ AudioRecord(
75
+ MediaRecorder.AudioSource.VOICE_RECOGNITION,
76
+ SAMPLE_RATE,
77
+ AudioFormat.CHANNEL_IN_MONO,
78
+ AudioFormat.ENCODING_PCM_16BIT,
79
+ maxOf(minBuffer, SAMPLE_RATE),
80
+ )
81
+ if (recorder.state != AudioRecord.STATE_INITIALIZED) {
82
+ recorder.release()
83
+ return MicResult.Failed("The microphone could not be opened. Another app may be using it.")
84
+ }
85
+ val pcm = ByteArrayOutputStream()
86
+ val maxBytes = SAMPLE_RATE * BYTES_PER_SAMPLE * MAX_SECONDS
87
+ try {
88
+ recorder.startRecording()
89
+ if (recorder.recordingState != AudioRecord.RECORDSTATE_RECORDING) {
90
+ return MicResult.Failed("The microphone did not start. Another app may be using it.")
91
+ }
92
+ val buffer = ByteArray(SAMPLE_RATE / 10 * BYTES_PER_SAMPLE) // 100 ms
93
+ while (!stopRequested && pcm.size() < maxBytes) {
94
+ val count = recorder.read(buffer, 0, minOf(buffer.size, maxBytes - pcm.size()))
95
+ if (count < 0) return MicResult.Failed("Recording failed (AudioRecord error $count).")
96
+ pcm.write(buffer, 0, count)
97
+ onProgress(pcm.size().toDouble() / (SAMPLE_RATE * BYTES_PER_SAMPLE))
98
+ }
99
+ } finally {
100
+ runCatching { recorder.stop() }
101
+ recorder.release()
102
+ }
103
+ val bytes = pcm.toByteArray()
104
+ if (bytes.isEmpty()) return MicResult.Failed("No audio was recorded.")
105
+ Wav.writeMono16(output, bytes, SAMPLE_RATE)
106
+ return MicResult.Recorded(
107
+ wav = output,
108
+ seconds = bytes.size.toDouble() / (SAMPLE_RATE * BYTES_PER_SAMPLE),
109
+ levels = Wav.levels(bytes),
110
+ stoppedAtLimit = bytes.size >= maxBytes,
111
+ )
112
+ }
113
+
114
+ companion object {
115
+ const val SAMPLE_RATE = 16_000
116
+ const val MAX_SECONDS = 30
117
+ private const val BYTES_PER_SAMPLE = 2
118
+
119
+ const val PERMISSION_MESSAGE =
120
+ "Microphone permission is off, so the app cannot record. The three clips still work. To use the " +
121
+ "microphone, allow it in Settings > Apps > ASR LiteRT-LM > Permissions."
122
+ }
123
+ }
android/app/src/main/kotlin/com/asrlitertlm/ModelProfile.kt ADDED
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ /** How a model writes its answer. */
4
+ enum class AnswerFormat {
5
+ /** Qwen3-ASR and its fine-tunes: `language <name><asr_text><transcript>`. */
6
+ LANGUAGE_TAGGED,
7
+
8
+ /** Fun-ASR-Nano: the transcript alone. */
9
+ PLAIN,
10
+ }
11
+
12
+ /** A transcript, split into the language the model named (empty when it named none) and the text. */
13
+ data class Answer(val language: String, val text: String)
14
+
15
+ /**
16
+ * One bundle's contract with this app. All three bundles go through the same calls: one Engine, a new Conversation per
17
+ * clip, one user message holding only the audio (each bundle's own template adds the rest: Fun-ASR's default
18
+ * instruction `语音转写:`, Qwen3-ASR's audio markers), greedy sampling and [maxOutputTokens]. Only the shape of the
19
+ * answer differs ([answerFormat]).
20
+ */
21
+ data class ModelProfile(
22
+ /** Stable key for launch extras and the device check. */
23
+ val id: String,
24
+ val displayName: String,
25
+ /** The file name the app looks for in its external files directory. */
26
+ val fileName: String,
27
+ val hubRepo: String,
28
+ /** Size of the file on the Hub, shown before the file is on the phone. */
29
+ val hubBytes: Long,
30
+ val license: String,
31
+ val languages: String,
32
+ val answerFormat: AnswerFormat,
33
+ val maxOutputTokens: Int = 512,
34
+ ) {
35
+ /** Splits a raw answer into language and text. */
36
+ fun parse(raw: String): Answer =
37
+ when (answerFormat) {
38
+ AnswerFormat.PLAIN -> Answer(language = "", text = raw.trim())
39
+ AnswerFormat.LANGUAGE_TAGGED -> parseTagged(raw)
40
+ }
41
+
42
+ companion object {
43
+ private const val ASR_TEXT = "<asr_text>"
44
+ private const val LANGUAGE_PREFIX = "language "
45
+
46
+ val QWEN3_ASR_1_7B =
47
+ ModelProfile(
48
+ id = "qwen3-asr-1.7b",
49
+ displayName = "Qwen3-ASR-1.7B",
50
+ fileName = "Qwen3-ASR-1.7B.litertlm",
51
+ hubRepo = "litert-community/Qwen3-ASR-1.7B",
52
+ hubBytes = 2_693_064_192L,
53
+ license = "Apache-2.0",
54
+ languages = "30 languages",
55
+ answerFormat = AnswerFormat.LANGUAGE_TAGGED,
56
+ )
57
+
58
+ val FUN_ASR_NANO_2512 =
59
+ ModelProfile(
60
+ id = "fun-asr-nano-2512",
61
+ displayName = "Fun-ASR-Nano-2512",
62
+ fileName = "Fun-ASR-Nano-2512.litertlm",
63
+ hubRepo = "litert-community/Fun-ASR-Nano-2512",
64
+ hubBytes = 1_255_894_736L,
65
+ license = "Apache-2.0",
66
+ languages = "zh / en / ja",
67
+ answerFormat = AnswerFormat.PLAIN,
68
+ )
69
+
70
+ val CONFUCIUS4_R2T2 =
71
+ ModelProfile(
72
+ id = "confucius4-r2t2",
73
+ displayName = "Confucius4-R2T2",
74
+ fileName = "Confucius4-R2T2.litertlm",
75
+ hubRepo = "mlboydaisuke/Confucius4-R2T2-LiteRT",
76
+ hubBytes = 2_693_064_192L,
77
+ license = "NetEase Youdao Model Use License",
78
+ languages = "zh / en (Qwen3-ASR fine-tune)",
79
+ answerFormat = AnswerFormat.LANGUAGE_TAGGED,
80
+ )
81
+
82
+ /** The picker's rows, in screen order. */
83
+ val ALL = listOf(QWEN3_ASR_1_7B, FUN_ASR_NANO_2512, CONFUCIUS4_R2T2)
84
+
85
+ fun byId(id: String?): ModelProfile? = ALL.firstOrNull { it.id == id }
86
+
87
+ /**
88
+ * `language <name><asr_text><text>` -> (name, text), as qwen-asr's parse_asr_output does without a forced
89
+ * language; "language None" gives an empty name, and an answer without the tag is all text.
90
+ */
91
+ private fun parseTagged(raw: String): Answer {
92
+ val answer = raw.trim()
93
+ val tag = answer.indexOf(ASR_TEXT)
94
+ if (tag < 0) return Answer(language = "", text = answer)
95
+ val meta = answer.substring(0, tag)
96
+ val name =
97
+ meta
98
+ .lineSequence()
99
+ .map { it.trim() }
100
+ .firstOrNull { it.lowercase().startsWith(LANGUAGE_PREFIX) }
101
+ ?.substring(LANGUAGE_PREFIX.length)
102
+ ?.trim()
103
+ .orEmpty()
104
+ val language = if (name.equals("None", ignoreCase = true)) "" else name
105
+ return Answer(language = language, text = answer.substring(tag + ASR_TEXT.length).trim())
106
+ }
107
+ }
108
+ }
android/app/src/main/kotlin/com/asrlitertlm/PlayBar.kt ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ import android.content.Context
4
+ import android.graphics.Canvas
5
+ import android.graphics.Paint
6
+ import android.view.View
7
+
8
+ /** A thin rounded bar: the share of a clip played, or of the 30 s recording limit used. */
9
+ class PlayBar(context: Context, trackColor: Int) : View(context) {
10
+ private val trackPaint = Paint(Paint.ANTI_ALIAS_FLAG).apply { color = trackColor }
11
+ private val fillPaint = Paint(Paint.ANTI_ALIAS_FLAG)
12
+
13
+ var fraction = 0f
14
+ set(value) {
15
+ if (value != field) {
16
+ field = value
17
+ invalidate()
18
+ }
19
+ }
20
+
21
+ fun set(fraction: Float, fillColor: Int) {
22
+ fillPaint.color = fillColor
23
+ this.fraction = fraction
24
+ invalidate()
25
+ }
26
+
27
+ override fun onDraw(canvas: Canvas) {
28
+ val height = height.toFloat()
29
+ val radius = height / 2
30
+ canvas.drawRoundRect(0f, 0f, width.toFloat(), height, radius, radius, trackPaint)
31
+ if (fraction > 0f) {
32
+ canvas.drawRoundRect(0f, 0f, maxOf(height, width * fraction), height, radius, radius, fillPaint)
33
+ }
34
+ }
35
+ }
android/app/src/main/kotlin/com/asrlitertlm/TextMatch.kt ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ import java.text.Normalizer
4
+
5
+ /**
6
+ * Transcript comparison for the device check, the way the model cards score: NFKC, lowercase, every punctuation,
7
+ * separator and whitespace character removed, then the character error rate against the expected text.
8
+ */
9
+ object TextMatch {
10
+ fun normalize(text: String): String {
11
+ val folded = Normalizer.normalize(text, Normalizer.Form.NFKC).lowercase()
12
+ val out = StringBuilder()
13
+ var i = 0
14
+ while (i < folded.length) {
15
+ val codePoint = folded.codePointAt(i)
16
+ if (!isDropped(codePoint)) out.appendCodePoint(codePoint)
17
+ i += Character.charCount(codePoint)
18
+ }
19
+ return out.toString()
20
+ }
21
+
22
+ /** Edit distance between the normalized texts over the normalized [expected] length (code points). */
23
+ fun cer(expected: String, actual: String): Double {
24
+ val reference = normalize(expected).codePoints().toArray()
25
+ val hypothesis = normalize(actual).codePoints().toArray()
26
+ if (reference.isEmpty()) return if (hypothesis.isEmpty()) 0.0 else 1.0
27
+ return editDistance(reference, hypothesis).toDouble() / reference.size
28
+ }
29
+
30
+ /** True when the two texts are equal after [normalize]. */
31
+ fun matches(expected: String, actual: String): Boolean = normalize(expected) == normalize(actual)
32
+
33
+ private fun isDropped(codePoint: Int): Boolean =
34
+ Character.isWhitespace(codePoint) ||
35
+ when (Character.getType(codePoint).toByte()) {
36
+ Character.CONNECTOR_PUNCTUATION,
37
+ Character.DASH_PUNCTUATION,
38
+ Character.START_PUNCTUATION,
39
+ Character.END_PUNCTUATION,
40
+ Character.INITIAL_QUOTE_PUNCTUATION,
41
+ Character.FINAL_QUOTE_PUNCTUATION,
42
+ Character.OTHER_PUNCTUATION,
43
+ Character.SPACE_SEPARATOR,
44
+ Character.LINE_SEPARATOR,
45
+ Character.PARAGRAPH_SEPARATOR -> true
46
+ else -> false
47
+ }
48
+
49
+ private fun editDistance(a: IntArray, b: IntArray): Int {
50
+ var previous = IntArray(b.size + 1) { it }
51
+ var current = IntArray(b.size + 1)
52
+ for (i in 1..a.size) {
53
+ current[0] = i
54
+ for (j in 1..b.size) {
55
+ val substitution = previous[j - 1] + if (a[i - 1] == b[j - 1]) 0 else 1
56
+ current[j] = minOf(substitution, previous[j] + 1, current[j - 1] + 1)
57
+ }
58
+ val swap = previous
59
+ previous = current
60
+ current = swap
61
+ }
62
+ return previous[b.size]
63
+ }
64
+ }
android/app/src/main/kotlin/com/asrlitertlm/Wav.kt ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ import java.io.File
4
+ import java.io.RandomAccessFile
5
+ import java.nio.ByteBuffer
6
+ import java.nio.ByteOrder
7
+ import kotlin.math.abs
8
+ import kotlin.math.log10
9
+ import kotlin.math.max
10
+
11
+ /**
12
+ * Minimal RIFF/WAVE helpers: the length and the samples of a wav (walks the chunks; the bundled clips carry a LIST
13
+ * chunk), a 16-bit mono writer for the microphone, a trimmed copy for clips longer than one model window, and the
14
+ * level of a recording.
15
+ */
16
+ object Wav {
17
+ /** 16-bit PCM samples of a wav, interleaved when [channels] > 1. */
18
+ class Pcm(val data: ByteArray, val sampleRate: Int, val channels: Int) {
19
+ /** Frames (samples per channel). */
20
+ val frames: Int
21
+ get() = data.size / (2 * channels)
22
+
23
+ val seconds: Double
24
+ get() = frames.toDouble() / sampleRate
25
+ }
26
+
27
+ /** Peak and RMS of a recording in dBFS (-120 for digital silence). */
28
+ data class Levels(val peakDbfs: Double, val rmsDbfs: Double)
29
+
30
+ private class Info(
31
+ val sampleRate: Int,
32
+ val channels: Int,
33
+ val bits: Int,
34
+ val byteRate: Int,
35
+ val dataSize: Long,
36
+ val data: ByteArray?,
37
+ )
38
+
39
+ private fun read(file: File, withData: Boolean): Info {
40
+ RandomAccessFile(file, "r").use { f ->
41
+ val head = ByteArray(12).also { f.readFully(it) }
42
+ require(String(head, 0, 4) == "RIFF" && String(head, 8, 4) == "WAVE") { "not a wav: $file" }
43
+ var fmt: ByteBuffer? = null
44
+ while (f.filePointer + 8 <= f.length()) {
45
+ val header = ByteArray(8).also { f.readFully(it) }
46
+ val id = String(header, 0, 4)
47
+ val size = ByteBuffer.wrap(header, 4, 4).order(ByteOrder.LITTLE_ENDIAN).int.toLong() and 0xffffffffL
48
+ when (id) {
49
+ "fmt " -> {
50
+ fmt = ByteBuffer.wrap(ByteArray(size.toInt()).also { f.readFully(it) }).order(ByteOrder.LITTLE_ENDIAN)
51
+ if (size % 2 == 1L) f.skipBytes(1)
52
+ }
53
+ "data" -> {
54
+ val format = requireNotNull(fmt) { "data before fmt in $file" }
55
+ // A recorder that never patched the header reports 0 or 0xFFFFFFFF: take what the file holds.
56
+ val available = f.length() - f.filePointer
57
+ val length = if (size == 0L || size > available) available else size
58
+ val data = if (withData) ByteArray(length.toInt()).also { f.readFully(it) } else null
59
+ return Info(
60
+ sampleRate = format.getInt(4),
61
+ channels = format.getShort(2).toInt(),
62
+ bits = format.getShort(14).toInt(),
63
+ byteRate = format.getInt(8),
64
+ dataSize = length,
65
+ data = data,
66
+ )
67
+ }
68
+ else -> f.seek(f.filePointer + size + (size % 2))
69
+ }
70
+ }
71
+ }
72
+ error("no data chunk in $file")
73
+ }
74
+
75
+ /** The length of a wav in seconds, from its header. */
76
+ fun seconds(file: File): Double = read(file, withData = false).let { it.dataSize.toDouble() / it.byteRate }
77
+
78
+ /** The samples of a 16-bit PCM wav. */
79
+ fun pcm16(file: File): Pcm =
80
+ read(file, withData = true).let {
81
+ require(it.bits == 16) { "not 16-bit PCM: $file" }
82
+ Pcm(it.data!!, it.sampleRate, it.channels)
83
+ }
84
+
85
+ /** Writes 16-bit mono PCM as a canonical 44-byte-header wav. */
86
+ fun writeMono16(file: File, pcm: ByteArray, sampleRate: Int) {
87
+ val header =
88
+ ByteBuffer.allocate(44).order(ByteOrder.LITTLE_ENDIAN).apply {
89
+ put("RIFF".toByteArray())
90
+ putInt(36 + pcm.size)
91
+ put("WAVE".toByteArray())
92
+ put("fmt ".toByteArray())
93
+ putInt(16)
94
+ putShort(1)
95
+ putShort(1)
96
+ putInt(sampleRate)
97
+ putInt(sampleRate * 2)
98
+ putShort(2)
99
+ putShort(16)
100
+ put("data".toByteArray())
101
+ putInt(pcm.size)
102
+ }
103
+ file.outputStream().use {
104
+ it.write(header.array())
105
+ it.write(pcm)
106
+ }
107
+ }
108
+
109
+ /**
110
+ * Returns [source] when it is at most [maxSeconds] long; otherwise writes its first [maxSeconds] (mono, 16-bit)
111
+ * to [target] and returns that. The models read one window of about 30 s per message.
112
+ */
113
+ fun trimmed(source: File, maxSeconds: Double, target: File): File {
114
+ if (seconds(source) <= maxSeconds) return source
115
+ val pcm = pcm16(source)
116
+ require(pcm.channels == 1) { "only mono clips are trimmed: $source" }
117
+ val keepBytes = (maxSeconds * pcm.sampleRate).toInt() * 2
118
+ writeMono16(target, pcm.data.copyOf(keepBytes), pcm.sampleRate)
119
+ return target
120
+ }
121
+
122
+ /** Peak and RMS of 16-bit little-endian PCM. */
123
+ fun levels(pcm: ByteArray): Levels {
124
+ val samples = ByteBuffer.wrap(pcm).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer()
125
+ var peak = 0
126
+ var sumOfSquares = 0.0
127
+ for (k in 0 until samples.limit()) {
128
+ val value = samples.get(k).toInt()
129
+ peak = max(peak, abs(value))
130
+ sumOfSquares += value.toDouble() * value
131
+ }
132
+ val count = max(samples.limit(), 1)
133
+ return Levels(peakDbfs = toDbfs(peak.toDouble()), rmsDbfs = toDbfs(Math.sqrt(sumOfSquares / count)))
134
+ }
135
+
136
+ private fun toDbfs(amplitude: Double): Double =
137
+ if (amplitude <= 0.0) SILENCE_DBFS else max(SILENCE_DBFS, 20 * log10(amplitude / 32768.0))
138
+
139
+ private const val SILENCE_DBFS = -120.0
140
+ }
android/app/src/main/res/raw/fleurs_cmn_1698.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f72a74b2b83aee568941fe6bd174309a5a54ac9d2dc0b698f6587a8b29aeffd2
3
+ size 311084
android/app/src/main/res/raw/fleurs_en_1698.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a696132b9f79f7936d470a6dc6fdb19ce009b4701165afe9097fa970923dff76
3
+ size 205484
android/app/src/main/res/raw/fleurs_ja_1698.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1235eb34a774a5b52e2032e0a344765cbc7decadad943d8ae77872c482599ba7
3
+ size 353324
android/app/src/main/res/values/strings.xml ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ <?xml version="1.0" encoding="utf-8"?>
2
+ <resources>
3
+ <string name="app_name">ASR LiteRT-LM</string>
4
+ </resources>
android/app/src/main/res/values/themes.xml ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <?xml version="1.0" encoding="utf-8"?>
2
+ <resources>
3
+ <style name="Theme.AsrLitertlm" parent="@android:style/Theme.DeviceDefault.NoActionBar">
4
+ <item name="android:windowBackground">@color/bg</item>
5
+ <item name="android:colorBackground">@color/bg</item>
6
+ <item name="android:statusBarColor">@color/bg</item>
7
+ <item name="android:navigationBarColor">@color/bg</item>
8
+ <item name="android:windowLightStatusBar">false</item>
9
+ <!-- A plain dark launch screen: no default launcher icon on the system splash. -->
10
+ <item name="android:windowSplashScreenBackground">@color/bg</item>
11
+ <item name="android:windowSplashScreenAnimatedIcon">@android:color/transparent</item>
12
+ </style>
13
+
14
+ <color name="bg">#0E1116</color>
15
+ </resources>
android/app/src/test/kotlin/com/asrlitertlm/AsrLogicTest.kt ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ package com.asrlitertlm
2
+
3
+ import java.io.File
4
+ import java.nio.ByteBuffer
5
+ import java.nio.ByteOrder
6
+ import kotlin.math.sin
7
+ import org.junit.Assert.assertEquals
8
+ import org.junit.Assert.assertFalse
9
+ import org.junit.Assert.assertSame
10
+ import org.junit.Assert.assertTrue
11
+ import org.junit.Rule
12
+ import org.junit.Test
13
+ import org.junit.rules.TemporaryFolder
14
+
15
+ /** The parts of the app that run without a phone: answer parsing, transcript matching and the wav helpers. */
16
+ class AsrLogicTest {
17
+ @get:Rule val folder = TemporaryFolder()
18
+
19
+ @Test
20
+ fun taggedAnswerSplitsLanguageAndText() {
21
+ val answer =
22
+ ModelProfile.QWEN3_ASR_1_7B.parse(
23
+ "language English<asr_text>Everything in the universe is made of matter."
24
+ )
25
+ assertEquals(Answer("English", "Everything in the universe is made of matter."), answer)
26
+ }
27
+
28
+ @Test
29
+ fun taggedAnswerWithLanguageNoneHasNoLanguage() {
30
+ assertEquals(Answer("", "hello"), ModelProfile.CONFUCIUS4_R2T2.parse("language None<asr_text>hello"))
31
+ }
32
+
33
+ @Test
34
+ fun answerWithoutTagIsAllText() {
35
+ assertEquals(Answer("", "宇宙中的一切"), ModelProfile.QWEN3_ASR_1_7B.parse(" 宇宙中的一切 "))
36
+ }
37
+
38
+ @Test
39
+ fun plainAnswerKeepsTheTagLikeText() {
40
+ // Fun-ASR writes the transcript alone; nothing in it is read as a language tag.
41
+ assertEquals(
42
+ Answer("", "language English<asr_text>x"),
43
+ ModelProfile.FUN_ASR_NANO_2512.parse("language English<asr_text>x"),
44
+ )
45
+ }
46
+
47
+ @Test
48
+ fun normalizationDropsPunctuationSpaceAndCase() {
49
+ assertTrue(
50
+ TextMatch.matches(
51
+ "Everything in the universe is made of matter. All matter is made of tiny particles called atoms.",
52
+ "everything in the Universe is made of matter, all matter is made of tiny particles called atoms",
53
+ )
54
+ )
55
+ assertTrue(TextMatch.matches("宇宙中的一切都由物质构成,而所有的物质", "宇宙中的一切都由物质构成 而所有的物质。"))
56
+ assertTrue(TextMatch.matches("ABC123", "abc123")) // NFKC folds full-width forms
57
+ }
58
+
59
+ @Test
60
+ fun cerCountsCharacterEdits() {
61
+ // 微小粒子 -> 微小颗粒: two substitutions over 32 characters (punctuation removed).
62
+ val reference = "宇宙中的一切都由物质构成,而所有的物质都由被称为原子的微小粒子组成。"
63
+ val hypothesis = "宇宙中的一切都由物质构成,而所有的物质都由被称为原子的微小颗粒组成。"
64
+ assertEquals(2.0 / 32.0, TextMatch.cer(reference, hypothesis), 1e-9)
65
+ assertEquals(0.0, TextMatch.cer(reference, reference), 0.0)
66
+ assertEquals(1.0, TextMatch.cer(reference, ""), 0.0)
67
+ }
68
+
69
+ @Test
70
+ fun shortWavIsSentAsIs() {
71
+ val wav = tone(seconds = 2.0)
72
+ assertSame(wav, Wav.trimmed(wav, 30.0, folder.newFile("cut.wav")))
73
+ }
74
+
75
+ @Test
76
+ fun longWavIsCutAtThirtySeconds() {
77
+ val wav = tone(seconds = 33.6)
78
+ assertEquals(33.6, Wav.seconds(wav), 1e-6)
79
+ val cut = Wav.trimmed(wav, 30.0, folder.newFile("cut.wav"))
80
+ assertEquals(30.0, Wav.seconds(cut), 1e-6)
81
+ val original = Wav.pcm16(wav).data
82
+ val kept = Wav.pcm16(cut).data
83
+ assertTrue(kept.contentEquals(original.copyOf(kept.size)))
84
+ }
85
+
86
+ @Test
87
+ fun silenceAndToneLevels() {
88
+ val silence = Wav.levels(ByteArray(32_000))
89
+ assertEquals(-120.0, silence.rmsDbfs, 0.0)
90
+ assertTrue(silence.rmsDbfs < AsrSession.NO_SPEECH_RMS_DBFS)
91
+ val tone = Wav.levels(Wav.pcm16(tone(seconds = 1.0)).data)
92
+ // A sine at half scale: peak about -6 dBFS, RMS about -9 dBFS.
93
+ assertEquals(-6.0, tone.peakDbfs, 0.1)
94
+ assertEquals(-9.0, tone.rmsDbfs, 0.1)
95
+ assertFalse(tone.rmsDbfs < AsrSession.NO_SPEECH_RMS_DBFS)
96
+ }
97
+
98
+ private fun tone(seconds: Double, rate: Int = 16_000): File {
99
+ val samples = (seconds * rate).toInt()
100
+ val pcm = ByteBuffer.allocate(samples * 2).order(ByteOrder.LITTLE_ENDIAN)
101
+ for (i in 0 until samples) pcm.putShort((16384 * sin(2 * Math.PI * 440 * i / rate)).toInt().toShort())
102
+ return folder.newFile().also { Wav.writeMono16(it, pcm.array(), rate) }
103
+ }
104
+ }
android/build.gradle.kts ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ // AGP 9 has Kotlin built in; do not apply org.jetbrains.kotlin.android.
2
+ // litertlm-android 0.17.1 ships Kotlin 2.4 metadata. The Kotlin Gradle plugin that AGP 9.3.1 brings (2.2.10) reads
3
+ // metadata up to 2.3 only, so the build classpath gets KGP 2.4.0, which the built-in Kotlin then uses.
4
+ buildscript {
5
+ repositories { mavenCentral() }
6
+ dependencies { classpath("org.jetbrains.kotlin:kotlin-gradle-plugin:2.4.0") }
7
+ }
8
+
9
+ plugins { id("com.android.application") version "9.3.1" apply false }
android/gradle.properties ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ org.gradle.jvmargs=-Xmx4g -Dfile.encoding=UTF-8
2
+ android.useAndroidX=true
3
+ kotlin.code.style=official
4
+ # In-process Kotlin compilation: the daemon writes under ~/Library, which sandboxed builds cannot.
5
+ kotlin.compiler.execution.strategy=in-process
6
+ # The LiteRT-LM runtime the three model cards' Kotlin examples were run with; shown on screen as the runtime version.
7
+ litertlmVersion=0.17.1
android/gradle/wrapper/gradle-wrapper.jar ADDED
Binary file (47.5 kB). View file
 
android/gradle/wrapper/gradle-wrapper.properties ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ distributionBase=GRADLE_USER_HOME
2
+ distributionPath=wrapper/dists
3
+ distributionUrl=https\://services.gradle.org/distributions/gradle-9.7.0-bin.zip
4
+ networkTimeout=10000
5
+ validateDistributionUrl=true
6
+ zipStoreBase=GRADLE_USER_HOME
7
+ zipStorePath=wrapper/dists
android/gradlew ADDED
@@ -0,0 +1,248 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/sh
2
+
3
+ #
4
+ # Copyright © 2015 the original authors.
5
+ #
6
+ # Licensed under the Apache License, Version 2.0 (the "License");
7
+ # you may not use this file except in compliance with the License.
8
+ # You may obtain a copy of the License at
9
+ #
10
+ # https://www.apache.org/licenses/LICENSE-2.0
11
+ #
12
+ # Unless required by applicable law or agreed to in writing, software
13
+ # distributed under the License is distributed on an "AS IS" BASIS,
14
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15
+ # See the License for the specific language governing permissions and
16
+ # limitations under the License.
17
+ #
18
+ # SPDX-License-Identifier: Apache-2.0
19
+ #
20
+
21
+ ##############################################################################
22
+ #
23
+ # gradlew start up script for POSIX generated by Gradle.
24
+ #
25
+ # Important for running:
26
+ #
27
+ # (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
28
+ # noncompliant, but you have some other compliant shell such as ksh or
29
+ # bash, then to run this script, type that shell name before the whole
30
+ # command line, like:
31
+ #
32
+ # ksh gradlew
33
+ #
34
+ # Busybox and similar reduced shells will NOT work, because this script
35
+ # requires all of these POSIX shell features:
36
+ # * functions;
37
+ # * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
38
+ # «${var#prefix}», «${var%suffix}», and «$( cmd )»;
39
+ # * compound commands having a testable exit status, especially «case»;
40
+ # * various built-in commands including «command», «set», and «ulimit».
41
+ #
42
+ # Important for patching:
43
+ #
44
+ # (2) This script targets any POSIX shell, so it avoids extensions provided
45
+ # by Bash, Ksh, etc; in particular arrays are avoided.
46
+ #
47
+ # The "traditional" practice of packing multiple parameters into a
48
+ # space-separated string is a well documented source of bugs and security
49
+ # problems, so this is (mostly) avoided, by progressively accumulating
50
+ # options in "$@", and eventually passing that to Java.
51
+ #
52
+ # Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
53
+ # and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
54
+ # see the in-line comments for details.
55
+ #
56
+ # There are tweaks for specific operating systems such as AIX, CygWin,
57
+ # Darwin, MinGW, and NonStop.
58
+ #
59
+ # (3) This script is generated from the Groovy template
60
+ # https://github.com/gradle/gradle/blob/3d91ce3b8caaf77ad09f381f43615b715b53f72c/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
61
+ # within the Gradle project.
62
+ #
63
+ # You can find Gradle at https://github.com/gradle/gradle/.
64
+ #
65
+ ##############################################################################
66
+
67
+ # Attempt to set APP_HOME
68
+
69
+ # Resolve links: $0 may be a link
70
+ app_path=$0
71
+
72
+ # Need this for daisy-chained symlinks.
73
+ while
74
+ APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
75
+ [ -h "$app_path" ]
76
+ do
77
+ ls=$( ls -ld "$app_path" )
78
+ link=${ls#*' -> '}
79
+ case $link in #(
80
+ /*) app_path=$link ;; #(
81
+ *) app_path=$APP_HOME$link ;;
82
+ esac
83
+ done
84
+
85
+ # This is normally unused
86
+ # shellcheck disable=SC2034
87
+ APP_BASE_NAME=${0##*/}
88
+ # Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
89
+ APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit
90
+
91
+ # Use the maximum available, or set MAX_FD != -1 to use that value.
92
+ MAX_FD=maximum
93
+
94
+ warn () {
95
+ echo "$*"
96
+ } >&2
97
+
98
+ die () {
99
+ echo
100
+ echo "$*"
101
+ echo
102
+ exit 1
103
+ } >&2
104
+
105
+ # OS specific support (must be 'true' or 'false').
106
+ cygwin=false
107
+ msys=false
108
+ darwin=false
109
+ nonstop=false
110
+ case "$( uname )" in #(
111
+ CYGWIN* ) cygwin=true ;; #(
112
+ Darwin* ) darwin=true ;; #(
113
+ MSYS* | MINGW* ) msys=true ;; #(
114
+ NONSTOP* ) nonstop=true ;;
115
+ esac
116
+
117
+
118
+
119
+ # Determine the Java command to use to start the JVM.
120
+ if [ -n "$JAVA_HOME" ] ; then
121
+ if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
122
+ # IBM's JDK on AIX uses strange locations for the executables
123
+ JAVACMD=$JAVA_HOME/jre/sh/java
124
+ else
125
+ JAVACMD=$JAVA_HOME/bin/java
126
+ fi
127
+ if [ ! -x "$JAVACMD" ] ; then
128
+ die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
129
+
130
+ Please set the JAVA_HOME variable in your environment to match the
131
+ location of your Java installation."
132
+ fi
133
+ else
134
+ JAVACMD=java
135
+ if ! command -v java >/dev/null 2>&1
136
+ then
137
+ die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
138
+
139
+ Please set the JAVA_HOME variable in your environment to match the
140
+ location of your Java installation."
141
+ fi
142
+ fi
143
+
144
+ # Increase the maximum file descriptors if we can.
145
+ if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
146
+ case $MAX_FD in #(
147
+ max*)
148
+ # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
149
+ # shellcheck disable=SC2039,SC3045
150
+ MAX_FD=$( ulimit -H -n ) ||
151
+ warn "Could not query maximum file descriptor limit"
152
+ esac
153
+ case $MAX_FD in #(
154
+ '' | soft) :;; #(
155
+ *)
156
+ # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
157
+ # shellcheck disable=SC2039,SC3045
158
+ ulimit -n "$MAX_FD" ||
159
+ warn "Could not set maximum file descriptor limit to $MAX_FD"
160
+ esac
161
+ fi
162
+
163
+ # Collect all arguments for the java command, stacking in reverse order:
164
+ # * args from the command line
165
+ # * the main class name
166
+ # * -classpath
167
+ # * -D...appname settings
168
+ # * --module-path (only if needed)
169
+ # * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
170
+
171
+ # For Cygwin or MSYS, switch paths to Windows format before running java
172
+ if "$cygwin" || "$msys" ; then
173
+ APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
174
+
175
+ JAVACMD=$( cygpath --unix "$JAVACMD" )
176
+
177
+ # Now convert the arguments - kludge to limit ourselves to /bin/sh
178
+ for arg do
179
+ if
180
+ case $arg in #(
181
+ -*) false ;; # don't mess with options #(
182
+ /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
183
+ [ -e "$t" ] ;; #(
184
+ *) false ;;
185
+ esac
186
+ then
187
+ arg=$( cygpath --path --ignore --mixed "$arg" )
188
+ fi
189
+ # Roll the args list around exactly as many times as the number of
190
+ # args, so each arg winds up back in the position where it started, but
191
+ # possibly modified.
192
+ #
193
+ # NB: a `for` loop captures its iteration list before it begins, so
194
+ # changing the positional parameters here affects neither the number of
195
+ # iterations, nor the values presented in `arg`.
196
+ shift # remove old arg
197
+ set -- "$@" "$arg" # push replacement arg
198
+ done
199
+ fi
200
+
201
+
202
+ # Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
203
+ DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
204
+
205
+ # Collect all arguments for the java command:
206
+ # * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
207
+ # and any embedded shellness will be escaped.
208
+ # * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
209
+ # treated as '${Hostname}' itself on the command line.
210
+
211
+ set -- \
212
+ "-Dorg.gradle.appname=$APP_BASE_NAME" \
213
+ -jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \
214
+ "$@"
215
+
216
+ # Stop when "xargs" is not available.
217
+ if ! command -v xargs >/dev/null 2>&1
218
+ then
219
+ die "xargs is not available"
220
+ fi
221
+
222
+ # Use "xargs" to parse quoted args.
223
+ #
224
+ # With -n1 it outputs one arg per line, with the quotes and backslashes removed.
225
+ #
226
+ # In Bash we could simply go:
227
+ #
228
+ # readarray ARGS < <( xargs -n1 <<<"$var" ) &&
229
+ # set -- "${ARGS[@]}" "$@"
230
+ #
231
+ # but POSIX shell has neither arrays nor command substitution, so instead we
232
+ # post-process each arg (as a line of input to sed) to backslash-escape any
233
+ # character that might be a shell metacharacter, then use eval to reverse
234
+ # that process (while maintaining the separation between arguments), and wrap
235
+ # the whole thing up as a single "set" statement.
236
+ #
237
+ # This will of course break if any of these variables contains a newline or
238
+ # an unmatched quote.
239
+ #
240
+
241
+ eval "set -- $(
242
+ printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
243
+ xargs -n1 |
244
+ sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
245
+ tr '\n' ' '
246
+ )" '"$@"'
247
+
248
+ exec "$JAVACMD" "$@"
android/settings.gradle.kts ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ pluginManagement {
2
+ repositories {
3
+ google()
4
+ mavenCentral()
5
+ gradlePluginPortal()
6
+ }
7
+ }
8
+
9
+ dependencyResolutionManagement {
10
+ repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
11
+ // The LiteRT-LM AARs live on Google Maven.
12
+ repositories {
13
+ google()
14
+ mavenCentral()
15
+ }
16
+ }
17
+
18
+ rootProject.name = "asr-litertlm"
19
+
20
+ include(":app")