Upload folder using huggingface_hub
Browse files- .gitattributes +2 -0
- .gitignore +6 -0
- README.md +68 -0
- config.json +1 -0
- models/model.axmodel +3 -0
- models/model_meta.json +46 -0
- python/README.md +41 -0
- python/demo.py +35 -0
- python/gcrn_sdk/__init__.py +6 -0
- python/gcrn_sdk/audio.py +103 -0
- python/gcrn_sdk/inference.py +86 -0
- python/gcrn_sdk/runtime.py +114 -0
- python/requirements.txt +6 -0
- reports/compile_report.md +14 -0
- reports/export_report.md +8 -0
- reports/performance_report.md +41 -0
- reports/runonboard_report.md +20 -0
- reports/simulate_report.md +8 -0
- run.sh +12 -0
- setup.sh +25 -0
- test_audio/mix.wav +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
models/model.axmodel filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
test_audio/mix.wav filter=lfs diff=lfs merge=lfs -text
|
.gitignore
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
__pycache__/
|
| 2 |
+
*.pyc
|
| 3 |
+
build/
|
| 4 |
+
CMakeFiles/
|
| 5 |
+
CMakeCache.txt
|
| 6 |
+
*.egg-info/
|
README.md
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
pipeline_tag: audio-to-audio
|
| 4 |
+
tags:
|
| 5 |
+
- axmodel
|
| 6 |
+
- axera
|
| 7 |
+
- gcrn
|
| 8 |
+
- speech-enhancement
|
| 9 |
+
---
|
| 10 |
+
# GCRN 语音增强(AX650 NPU3)
|
| 11 |
+
|
| 12 |
+
GCRN(Graph Convolutional Recurrent Network)单声道语音增强模型,输入 16 kHz 噪声语音,
|
| 13 |
+
输出同长增强语音。本包为 AX650 / NPU3 量化部署版:
|
| 14 |
+
|
| 15 |
+
- 精度:AXMODEL 与 ONNX 参考余弦相似度 **0.9992**(增强波形余弦 0.9997)
|
| 16 |
+
- 速度:AX650 板端 **RTF 0.029**(9.77 秒音频约 0.28 秒处理完)
|
| 17 |
+
- 大小:AXMODEL **27.6 MB**(ONNX 38 MB)
|
| 18 |
+
|
| 19 |
+
## 快速开始(两步)
|
| 20 |
+
|
| 21 |
+
### 1. 安装环境
|
| 22 |
+
|
| 23 |
+
```bash
|
| 24 |
+
bash setup.sh
|
| 25 |
+
```
|
| 26 |
+
|
| 27 |
+
### 2. 跑推理
|
| 28 |
+
|
| 29 |
+
```bash
|
| 30 |
+
bash run.sh
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
脚本会自动选择后端:板端(有 axengine)跑 NPU 模型,普通电脑自动用 ONNX 模型演示。
|
| 34 |
+
输出为 `enhanced.wav`,增强前后的对比音频可以自己听一下。
|
| 35 |
+
|
| 36 |
+
## 目录说明
|
| 37 |
+
|
| 38 |
+
| 目录 | 用途 |
|
| 39 |
+
|------|------|
|
| 40 |
+
| `models/` | AXMODEL 模型 + `model_meta.json` |
|
| 41 |
+
| `python/` | Python SDK(axengine / onnxruntime 双后端)|
|
| 42 |
+
| `cpp/` | C++ SDK(AX Engine runtime,aarch64 交叉编译)|
|
| 43 |
+
| `model_convert/` | 导出、校准、Pulsar2 编译全套可复现脚本 |
|
| 44 |
+
| `reports/` | 导出/编译/仿真/板端验证报告 |
|
| 45 |
+
| `test_audio/` | 自带 9.77 秒测试音频 |
|
| 46 |
+
|
| 47 |
+
## 在自己的代码里用
|
| 48 |
+
|
| 49 |
+
```python
|
| 50 |
+
from gcrn_sdk import GCRNDenoiser
|
| 51 |
+
|
| 52 |
+
denoiser = GCRNDenoiser("models/model.axmodel")
|
| 53 |
+
report = denoiser.enhance_file("test_audio/mix.wav", "enhanced.wav")
|
| 54 |
+
print(report) # 含 RTF 等性能指标
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
## 常见问题
|
| 58 |
+
|
| 59 |
+
**Q: import 报错找不到 axengine?**
|
| 60 |
+
A: 说明当前不在 AX 板端。代码会自动用 ONNX 模型演示;要在 NPU 上跑,需在 AX650 板端
|
| 61 |
+
安装匹配的 axengine wheel(见 `setup.sh` 输出提示)。
|
| 62 |
+
|
| 63 |
+
**Q: 想自己重新编译 AXMODEL?**
|
| 64 |
+
A: 进入 `model_convert/` 按 README 操作,Pulsar2 会从 ONNX 重新量化编译。
|
| 65 |
+
原始编译使用 Docker 镜像 `pulsar2:7.0-lite`,配置为 NPU3 + U16 高精度模式。
|
| 66 |
+
|
| 67 |
+
**Q: 输入音频有什么要求?**
|
| 68 |
+
A: 16 kHz、单声道、16-bit PCM WAV;更长音频会自动按 4 秒分块处理并拼接。
|
config.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{}
|
models/model.axmodel
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:02e401e25dcf6a70e3d3dc31e154d7f6001278ec76b4c2c7f5a59aea88710aa6
|
| 3 |
+
size 28943407
|
models/model_meta.json
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_name": "gcrn_fixed_4sec",
|
| 3 |
+
"framework": "onnx",
|
| 4 |
+
"source": "gcrn_fixed_4sec.onnx",
|
| 5 |
+
"source_sha256": "9e8e0acd13126c9d5d19aae6f83f230a16c8914f100744153c78c58920d1ff4a",
|
| 6 |
+
"inputs": [
|
| 7 |
+
{
|
| 8 |
+
"name": "stft_input",
|
| 9 |
+
"shape": [
|
| 10 |
+
1,
|
| 11 |
+
2,
|
| 12 |
+
401,
|
| 13 |
+
161
|
| 14 |
+
],
|
| 15 |
+
"dtype": "float32",
|
| 16 |
+
"layout": "NCHW"
|
| 17 |
+
}
|
| 18 |
+
],
|
| 19 |
+
"outputs": [
|
| 20 |
+
{
|
| 21 |
+
"name": "stft_output",
|
| 22 |
+
"shape": [
|
| 23 |
+
1,
|
| 24 |
+
2,
|
| 25 |
+
401,
|
| 26 |
+
161
|
| 27 |
+
],
|
| 28 |
+
"dtype": "float32"
|
| 29 |
+
}
|
| 30 |
+
],
|
| 31 |
+
"opset": 12,
|
| 32 |
+
"onnxsim_cosine_min": 1.0,
|
| 33 |
+
"preprocess": {
|
| 34 |
+
"description": "STFT of 16 kHz mono waveform -> [1,2,401,161] (batch, real/imag, time, freq)",
|
| 35 |
+
"sample_rate": 16000,
|
| 36 |
+
"duration_sec": 4.0,
|
| 37 |
+
"n_fft": 320,
|
| 38 |
+
"hop_length": 160,
|
| 39 |
+
"win_length": 320,
|
| 40 |
+
"center": true,
|
| 41 |
+
"window": "hamming(sym=True)"
|
| 42 |
+
},
|
| 43 |
+
"notes": [
|
| 44 |
+
"\u8f93\u5165\u4e3a STFT \u590d\u6570\u8c31\uff08\u5b9e\u90e8/\u865a\u90e8\uff09\uff0c\u65e0 mean/std \u5f52\u4e00\u5316"
|
| 45 |
+
]
|
| 46 |
+
}
|
python/README.md
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# GCRN Python SDK
|
| 2 |
+
|
| 3 |
+
GCRN 语音增强 Python 推理 SDK。模型输入 16 kHz 单声道噪声语音的 STFT 谱
|
| 4 |
+
`[1, 2, 401, 161]`(4 秒分块),输出增强谱,再还原为增强波形。
|
| 5 |
+
|
| 6 |
+
## 安装
|
| 7 |
+
|
| 8 |
+
```bash
|
| 9 |
+
pip install -r requirements.txt
|
| 10 |
+
```
|
| 11 |
+
|
| 12 |
+
`axengine` 需要 AX650 板端(或带 AX runtime 的主机);普通电脑可以装 `onnxruntime`
|
| 13 |
+
后用 ONNX 模型调试。
|
| 14 |
+
|
| 15 |
+
## 快速使用
|
| 16 |
+
|
| 17 |
+
```bash
|
| 18 |
+
python3 demo.py --model ../models/model.axmodel --input ../test_audio/mix.wav --output enhanced.wav
|
| 19 |
+
```
|
| 20 |
+
|
| 21 |
+
## API
|
| 22 |
+
|
| 23 |
+
```python
|
| 24 |
+
from gcrn_sdk import GCRNDenoiser
|
| 25 |
+
|
| 26 |
+
denoiser = GCRNDenoiser("models/model.axmodel") # .onnx 自动走 onnxruntime
|
| 27 |
+
report = denoiser.enhance_file("noisy.wav", "enhanced.wav")
|
| 28 |
+
print(report["real_time_factor"])
|
| 29 |
+
|
| 30 |
+
# 直接处理 int16 PCM:
|
| 31 |
+
pcm = ... # np.int16
|
| 32 |
+
enhanced = denoiser.enhance(pcm)
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
## 预处理说明
|
| 36 |
+
|
| 37 |
+
- 输入要求:16 kHz、单声道、16-bit PCM WAV;
|
| 38 |
+
- 分块:4 秒(64000 样本),不足补零,逐块增强后拼接;
|
| 39 |
+
- STFT:`n_fft=320, hop=160, win=320`,hamming(sym)窗,与模型训练端一致;
|
| 40 |
+
- 模型边界:`stft_input [1,2,401,161] float32` -> `stft_output [1,2,401,161] float32`
|
| 41 |
+
(布局为 batch, real/imag, time, freq)。
|
python/demo.py
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
# coding: utf-8
|
| 3 |
+
"""GCRN AX650 语音增强示例:noisy.wav -> enhanced.wav"""
|
| 4 |
+
|
| 5 |
+
import argparse
|
| 6 |
+
import sys
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
|
| 9 |
+
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
| 10 |
+
|
| 11 |
+
from gcrn_sdk import GCRNDenoiser
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
def main() -> None:
|
| 15 |
+
ap = argparse.ArgumentParser(description="GCRN speech enhancement on AX650")
|
| 16 |
+
ap.add_argument("--model", default="models/model.axmodel", help="axmodel (或 onnx)")
|
| 17 |
+
ap.add_argument("--input", default="test_audio/mix.wav", help="16kHz mono noisy wav")
|
| 18 |
+
ap.add_argument("--output", default="enhanced.wav", help="输出 wav")
|
| 19 |
+
ap.add_argument("--report", default="gcrn_report.json", help="性能报告输出")
|
| 20 |
+
args = ap.parse_args()
|
| 21 |
+
|
| 22 |
+
denoiser = GCRNDenoiser(args.model)
|
| 23 |
+
report = denoiser.enhance_file(args.input, args.output)
|
| 24 |
+
denoiser.save_report(report, args.report)
|
| 25 |
+
print(f"input={args.input}")
|
| 26 |
+
print(f"output={args.output}")
|
| 27 |
+
print(
|
| 28 |
+
f"elapsed={report['elapsed_seconds']:.3f}s "
|
| 29 |
+
f"audio={report['audio_seconds']:.3f}s "
|
| 30 |
+
f"rtf={report['real_time_factor']:.4f}"
|
| 31 |
+
)
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
if __name__ == "__main__":
|
| 35 |
+
main()
|
python/gcrn_sdk/__init__.py
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# coding: utf-8
|
| 2 |
+
"""GCRN 语音增强 AX SDK。"""
|
| 3 |
+
|
| 4 |
+
from .inference import GCRNDenoiser
|
| 5 |
+
|
| 6 |
+
__all__ = ["GCRNDenoiser"]
|
python/gcrn_sdk/audio.py
ADDED
|
@@ -0,0 +1,103 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# coding: utf-8
|
| 2 |
+
"""GCRN 依赖极简的音频预处理:16kHz 单声道 WAV <-> STFT [1,2,401,161]。
|
| 3 |
+
|
| 4 |
+
与 GCRN.AXERA 的 board_inference/audio.py 保持一致(hamming sym 窗 + numpy rfft)。
|
| 5 |
+
"""
|
| 6 |
+
from __future__ import annotations
|
| 7 |
+
|
| 8 |
+
import wave
|
| 9 |
+
from pathlib import Path
|
| 10 |
+
|
| 11 |
+
import numpy as np
|
| 12 |
+
|
| 13 |
+
SAMPLE_RATE = 16000
|
| 14 |
+
CHUNK_SAMPLES = 4 * SAMPLE_RATE
|
| 15 |
+
N_FFT = 320
|
| 16 |
+
HOP_LENGTH = 160
|
| 17 |
+
WIN_LENGTH = 320
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def read_wav(path: str | Path) -> tuple[np.ndarray, int]:
|
| 21 |
+
with wave.open(str(path), "rb") as source:
|
| 22 |
+
if source.getnchannels() != 1 or source.getsampwidth() != 2:
|
| 23 |
+
raise ValueError(f"expected mono 16-bit PCM WAV: {path}")
|
| 24 |
+
sample_rate = source.getframerate()
|
| 25 |
+
samples = np.frombuffer(
|
| 26 |
+
source.readframes(source.getnframes()), dtype=np.int16
|
| 27 |
+
).copy()
|
| 28 |
+
return samples, sample_rate
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def write_wav(path: str | Path, samples: np.ndarray, sample_rate: int) -> None:
|
| 32 |
+
values = np.asarray(samples, dtype=np.float32).reshape(-1)
|
| 33 |
+
pcm = np.clip(values, -1.0, 1.0)
|
| 34 |
+
pcm = np.rint(pcm * 32767.0).astype(np.int16)
|
| 35 |
+
path = Path(path)
|
| 36 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 37 |
+
with wave.open(str(path), "wb") as output:
|
| 38 |
+
output.setnchannels(1)
|
| 39 |
+
output.setsampwidth(2)
|
| 40 |
+
output.setframerate(sample_rate)
|
| 41 |
+
output.writeframes(pcm.tobytes())
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
def _symmetric_hamming(size: int) -> np.ndarray:
|
| 45 |
+
index = np.arange(size, dtype=np.float32)
|
| 46 |
+
return (0.54 - 0.46 * np.cos(2.0 * np.pi * index / (size - 1))).astype(
|
| 47 |
+
np.float32
|
| 48 |
+
)
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def chunks(samples: np.ndarray) -> list[tuple[np.ndarray, int]]:
|
| 52 |
+
"""int16 PCM -> 4 秒 float32 chunk 序列 [(chunk, valid_samples)]。"""
|
| 53 |
+
values = np.asarray(samples, dtype=np.float32).reshape(-1) / 32768.0
|
| 54 |
+
if values.size == 0:
|
| 55 |
+
raise ValueError("input WAV is empty")
|
| 56 |
+
result = []
|
| 57 |
+
for start in range(0, values.size, CHUNK_SAMPLES):
|
| 58 |
+
end = min(start + CHUNK_SAMPLES, values.size)
|
| 59 |
+
valid = end - start
|
| 60 |
+
chunk = np.zeros(CHUNK_SAMPLES, dtype=np.float32)
|
| 61 |
+
chunk[:valid] = values[start:end]
|
| 62 |
+
result.append((chunk, valid))
|
| 63 |
+
return result
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
def stft(chunk: np.ndarray) -> np.ndarray:
|
| 67 |
+
"""4 秒波形 -> [1, 2, 401, 161](batch, real/imag, time, freq)。"""
|
| 68 |
+
signal = np.asarray(chunk, dtype=np.float32).reshape(-1)
|
| 69 |
+
if signal.size != CHUNK_SAMPLES:
|
| 70 |
+
raise ValueError(f"expected {CHUNK_SAMPLES} samples, got {signal.size}")
|
| 71 |
+
pad = N_FFT // 2
|
| 72 |
+
signal = np.pad(signal, (pad, pad), mode="constant")
|
| 73 |
+
frame_count = 1 + (signal.size - WIN_LENGTH) // HOP_LENGTH
|
| 74 |
+
frames = np.lib.stride_tricks.sliding_window_view(signal, WIN_LENGTH)[
|
| 75 |
+
::HOP_LENGTH
|
| 76 |
+
][:frame_count]
|
| 77 |
+
spectrum = np.fft.rfft(frames * _symmetric_hamming(WIN_LENGTH)[None, :], axis=1)
|
| 78 |
+
value = np.stack((spectrum.real, spectrum.imag), axis=0)
|
| 79 |
+
return value.astype(np.float32)[None, ...]
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
def istft(value: np.ndarray) -> np.ndarray:
|
| 83 |
+
"""[1,2,401,161] 增强谱 -> 64000 个 float32 样本。"""
|
| 84 |
+
output = np.asarray(value, dtype=np.float32)
|
| 85 |
+
if output.shape != (1, 2, 401, 161):
|
| 86 |
+
raise ValueError(f"unexpected GCRN output shape: {output.shape}")
|
| 87 |
+
value = output[0]
|
| 88 |
+
spectrum = value[0].astype(np.float64) + 1j * value[1].astype(np.float64)
|
| 89 |
+
frames = np.fft.irfft(spectrum, n=N_FFT, axis=1)
|
| 90 |
+
window = _symmetric_hamming(WIN_LENGTH).astype(np.float64)
|
| 91 |
+
frames *= window[None, :]
|
| 92 |
+
|
| 93 |
+
output_length = N_FFT + HOP_LENGTH * (frames.shape[0] - 1)
|
| 94 |
+
waveform = np.zeros(output_length, dtype=np.float64)
|
| 95 |
+
window_sum = np.zeros(output_length, dtype=np.float64)
|
| 96 |
+
for index, frame in enumerate(frames):
|
| 97 |
+
start = index * HOP_LENGTH
|
| 98 |
+
waveform[start : start + N_FFT] += frame
|
| 99 |
+
window_sum[start : start + N_FFT] += window * window
|
| 100 |
+
valid = window_sum > np.finfo(np.float64).eps
|
| 101 |
+
waveform[valid] /= window_sum[valid]
|
| 102 |
+
pad = N_FFT // 2
|
| 103 |
+
return waveform[pad:-pad].astype(np.float32)
|
python/gcrn_sdk/inference.py
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# coding: utf-8
|
| 2 |
+
"""GCRN 语音增强推理:4 秒 chunk STFT -> axmodel -> ISTFT -> 拼接输出。"""
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import json
|
| 6 |
+
import time
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
|
| 9 |
+
import numpy as np
|
| 10 |
+
|
| 11 |
+
from .audio import (
|
| 12 |
+
SAMPLE_RATE,
|
| 13 |
+
chunks,
|
| 14 |
+
istft,
|
| 15 |
+
read_wav,
|
| 16 |
+
stft,
|
| 17 |
+
write_wav,
|
| 18 |
+
)
|
| 19 |
+
from .runtime import InferenceSession, first_output
|
| 20 |
+
|
| 21 |
+
INPUT_NAME = "stft_input"
|
| 22 |
+
OUTPUT_NAME = "stft_output"
|
| 23 |
+
INPUT_SHAPE = (1, 2, 401, 161)
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
class GCRNDenoiser:
|
| 27 |
+
"""AX650 GCRN 语音增强器。默认 .axmodel + axengine。"""
|
| 28 |
+
|
| 29 |
+
def __init__(self, model_path: str | Path, backend: str | None = None):
|
| 30 |
+
self.session = InferenceSession(model_path, backend)
|
| 31 |
+
if len(self.session.inputs) != 1 or self.session.inputs[0].shape != INPUT_SHAPE:
|
| 32 |
+
raise RuntimeError(f"unexpected GCRN input metadata: {self.session.inputs}")
|
| 33 |
+
if len(self.session.outputs) != 1 or self.session.outputs[0].shape != INPUT_SHAPE:
|
| 34 |
+
raise RuntimeError(f"unexpected GCRN output metadata: {self.session.outputs}")
|
| 35 |
+
self.input_name = self.session.inputs[0].name
|
| 36 |
+
|
| 37 |
+
def _warmup(self) -> None:
|
| 38 |
+
feed = stft(np.zeros(64000, dtype=np.float32))
|
| 39 |
+
first_output(self.session, self.session.run({self.input_name: feed}))
|
| 40 |
+
|
| 41 |
+
def enhance_chunk(self, chunk: np.ndarray) -> np.ndarray:
|
| 42 |
+
feed = stft(chunk)
|
| 43 |
+
outputs = self.session.run({self.input_name: feed})
|
| 44 |
+
output = np.asarray(first_output(self.session, outputs), dtype=np.float32)
|
| 45 |
+
return istft(output)
|
| 46 |
+
|
| 47 |
+
def enhance(self, pcm: np.ndarray) -> np.ndarray:
|
| 48 |
+
"""int16 PCM -> 增强后 float32 波形(与输入等长)。"""
|
| 49 |
+
self._warmup()
|
| 50 |
+
enhanced = [
|
| 51 |
+
self.enhance_chunk(chunk)
|
| 52 |
+
for chunk, _valid in chunks(pcm)
|
| 53 |
+
]
|
| 54 |
+
return np.concatenate(enhanced)[: pcm.size]
|
| 55 |
+
|
| 56 |
+
def enhance_file(
|
| 57 |
+
self, input_path: str | Path, output_path: str | Path
|
| 58 |
+
) -> dict:
|
| 59 |
+
"""输入 16kHz 单声道 WAV,输出增强 WAV + 性能报告 dict。"""
|
| 60 |
+
samples, sample_rate = read_wav(input_path)
|
| 61 |
+
if sample_rate != SAMPLE_RATE:
|
| 62 |
+
raise ValueError(
|
| 63 |
+
f"GCRN requires {SAMPLE_RATE} Hz audio, got {sample_rate}"
|
| 64 |
+
)
|
| 65 |
+
start = time.perf_counter()
|
| 66 |
+
enhanced = self.enhance(samples)
|
| 67 |
+
elapsed = time.perf_counter() - start
|
| 68 |
+
write_wav(output_path, enhanced, sample_rate)
|
| 69 |
+
audio_seconds = samples.size / sample_rate
|
| 70 |
+
report = {
|
| 71 |
+
"backend": self.session.backend,
|
| 72 |
+
"model": str(self.session.path),
|
| 73 |
+
"input": str(input_path),
|
| 74 |
+
"output": str(output_path),
|
| 75 |
+
"input_samples": int(samples.size),
|
| 76 |
+
"output_samples": int(enhanced.size),
|
| 77 |
+
"elapsed_seconds": round(elapsed, 4),
|
| 78 |
+
"audio_seconds": round(audio_seconds, 4),
|
| 79 |
+
"real_time_factor": round(elapsed / audio_seconds, 4),
|
| 80 |
+
}
|
| 81 |
+
return report
|
| 82 |
+
|
| 83 |
+
def save_report(self, report: dict, path: str | Path) -> None:
|
| 84 |
+
path = Path(path)
|
| 85 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 86 |
+
path.write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")
|
python/gcrn_sdk/runtime.py
ADDED
|
@@ -0,0 +1,114 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# coding: utf-8
|
| 2 |
+
"""axengine / onnxruntime 推理会话封装。"""
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from dataclasses import dataclass
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
from typing import Any
|
| 8 |
+
|
| 9 |
+
import numpy as np
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
@dataclass(frozen=True)
|
| 13 |
+
class TensorInfo:
|
| 14 |
+
name: str
|
| 15 |
+
shape: tuple[int, ...]
|
| 16 |
+
dtype: np.dtype
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
def _numpy_dtype(value: Any) -> np.dtype:
|
| 20 |
+
text = str(value).lower()
|
| 21 |
+
mapping = (
|
| 22 |
+
(("tensor(float)", "float32", "fp32", "f32"), np.float32),
|
| 23 |
+
(("tensor(float16)", "float16", "fp16", "f16"), np.float16),
|
| 24 |
+
(("tensor(int64)", "int64", "s64"), np.int64),
|
| 25 |
+
(("tensor(int32)", "int32", "s32"), np.int32),
|
| 26 |
+
(("tensor(uint16)", "uint16", "u16"), np.uint16),
|
| 27 |
+
(("tensor(uint8)", "uint8", "u8"), np.uint8),
|
| 28 |
+
)
|
| 29 |
+
for aliases, dtype in mapping:
|
| 30 |
+
if any(alias in text for alias in aliases):
|
| 31 |
+
return np.dtype(dtype)
|
| 32 |
+
raise ValueError(f"unsupported runtime tensor dtype: {value}")
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def _tensor_info(value: Any) -> TensorInfo:
|
| 36 |
+
shape = getattr(value, "shape", None)
|
| 37 |
+
if shape is None:
|
| 38 |
+
shape = getattr(value, "dims", None)
|
| 39 |
+
if shape is None or any(dim is None for dim in shape):
|
| 40 |
+
raise ValueError(f"dynamic or missing tensor shape for {value.name}: {shape}")
|
| 41 |
+
dtype = getattr(value, "dtype", None)
|
| 42 |
+
if dtype is None:
|
| 43 |
+
dtype = getattr(value, "type", None)
|
| 44 |
+
return TensorInfo(
|
| 45 |
+
name=value.name,
|
| 46 |
+
shape=tuple(int(dim) for dim in shape),
|
| 47 |
+
dtype=_numpy_dtype(dtype),
|
| 48 |
+
)
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
class InferenceSession:
|
| 52 |
+
""".axmodel -> axengine;.onnx -> onnxruntime CPU。"""
|
| 53 |
+
|
| 54 |
+
def __init__(self, model_path: str | Path, backend: str | None = None):
|
| 55 |
+
self.path = Path(model_path)
|
| 56 |
+
if not self.path.is_file():
|
| 57 |
+
raise FileNotFoundError(self.path)
|
| 58 |
+
if backend is None:
|
| 59 |
+
backend = "axengine" if self.path.suffix == ".axmodel" else "onnx"
|
| 60 |
+
self.backend = backend
|
| 61 |
+
if backend == "axengine":
|
| 62 |
+
try:
|
| 63 |
+
import axengine
|
| 64 |
+
except ImportError as error:
|
| 65 |
+
raise RuntimeError(
|
| 66 |
+
"axengine is unavailable; run this backend on an AXERA board"
|
| 67 |
+
) from error
|
| 68 |
+
self._session = axengine.InferenceSession(str(self.path))
|
| 69 |
+
elif backend == "onnx":
|
| 70 |
+
try:
|
| 71 |
+
import onnxruntime as ort
|
| 72 |
+
except ImportError as error:
|
| 73 |
+
raise RuntimeError("onnxruntime is required for --backend onnx") from error
|
| 74 |
+
options = ort.SessionOptions()
|
| 75 |
+
options.inter_op_num_threads = 1
|
| 76 |
+
options.intra_op_num_threads = 1
|
| 77 |
+
self._session = ort.InferenceSession(
|
| 78 |
+
str(self.path),
|
| 79 |
+
sess_options=options,
|
| 80 |
+
providers=["CPUExecutionProvider"],
|
| 81 |
+
)
|
| 82 |
+
else:
|
| 83 |
+
raise ValueError(f"unsupported backend: {backend}")
|
| 84 |
+
|
| 85 |
+
self.inputs = [_tensor_info(value) for value in self._session.get_inputs()]
|
| 86 |
+
self.outputs = [_tensor_info(value) for value in self._session.get_outputs()]
|
| 87 |
+
self.input_by_name = {value.name: value for value in self.inputs}
|
| 88 |
+
|
| 89 |
+
def run(self, feed: dict[str, np.ndarray]) -> dict[str, np.ndarray]:
|
| 90 |
+
missing = [value.name for value in self.inputs if value.name not in feed]
|
| 91 |
+
if missing:
|
| 92 |
+
raise KeyError(f"missing inputs for {self.path.name}: {missing}")
|
| 93 |
+
prepared = {
|
| 94 |
+
name: np.ascontiguousarray(np.asarray(feed[name], dtype=meta.dtype))
|
| 95 |
+
for name, meta in self.input_by_name.items()
|
| 96 |
+
}
|
| 97 |
+
values = self._session.run(None, prepared)
|
| 98 |
+
if isinstance(values, dict):
|
| 99 |
+
return {name: np.asarray(value) for name, value in values.items()}
|
| 100 |
+
if not isinstance(values, (list, tuple)):
|
| 101 |
+
values = [values]
|
| 102 |
+
if len(values) != len(self.outputs):
|
| 103 |
+
raise RuntimeError(
|
| 104 |
+
f"unexpected output count from {self.path.name}: "
|
| 105 |
+
f"{len(values)} != {len(self.outputs)}"
|
| 106 |
+
)
|
| 107 |
+
return {
|
| 108 |
+
meta.name: np.asarray(value)
|
| 109 |
+
for meta, value in zip(self.outputs, values)
|
| 110 |
+
}
|
| 111 |
+
|
| 112 |
+
|
| 113 |
+
def first_output(session: InferenceSession, outputs: dict[str, np.ndarray]) -> np.ndarray:
|
| 114 |
+
return outputs[session.outputs[0].name]
|
python/requirements.txt
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# GCRN AX Python SDK 依赖
|
| 2 |
+
# 运行时(板端 / 带 AX runtime 主机):
|
| 3 |
+
axengine>=0.1.3 # pyaxengine:.axmodel NPU 推理
|
| 4 |
+
numpy>=1.24
|
| 5 |
+
# 仅主机调试(无 NPU 时用 onnx 模型 + onnxruntime):
|
| 6 |
+
onnxruntime>=1.16
|
reports/compile_report.md
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Compile Report
|
| 2 |
+
|
| 3 |
+
- image: pulsar2:7.0-lite
|
| 4 |
+
- target: AX650 (AX650N_CHIP)
|
| 5 |
+
- npu_mode: NPU3
|
| 6 |
+
- input: stft_input:1x2x401x161 (FP32)
|
| 7 |
+
- model: model.onnx -> model.axmodel
|
| 8 |
+
- quant: default U16, Pow/Sqrt/Div/LSTM FP32, conv_bias FP32, MinMax, 25 calib samples
|
| 9 |
+
- macs: 9,799,365,320 (~9.8 GMACs)
|
| 10 |
+
- max_cycle: 38,038,900
|
| 11 |
+
- end-to-end quant cosine (Pulsar2 precision table): 0.99963 (stft_output)
|
| 12 |
+
- axmodel size: 28 MB (source onnx 38 MB, compression 0.74x)
|
| 13 |
+
- compile time: ~1h51m (LSTM unrolled to ~300k ops; host heavily loaded)
|
| 14 |
+
- highest_mix_precision: false
|
reports/export_report.md
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Export Report
|
| 2 |
+
|
| 3 |
+
- ONNX: model.onnx (simplified from gcrn_fixed_4sec.onnx)
|
| 4 |
+
- Input: stft_input [1, 2, 401, 161] float32
|
| 5 |
+
- Output: stft_output [1, 2, 401, 161] float32
|
| 6 |
+
- Opset: 12
|
| 7 |
+
- onnxsim cosine (vs source, 25 windows): min=1.000000 mean=1.000000
|
| 8 |
+
- Calibration data: stft_input.tar.gz (25 samples)
|
reports/performance_report.md
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# GCRN AX650 性能报告
|
| 2 |
+
|
| 3 |
+
## 模型概况
|
| 4 |
+
|
| 5 |
+
| 项目 | 数值 |
|
| 6 |
+
|------|------|
|
| 7 |
+
| 模型 | GCRN 语音增强(4 秒固定输入)|
|
| 8 |
+
| 输入/输出 | `[1, 2, 401, 161]` float32 |
|
| 9 |
+
| 计算量 | 9.80 GMACs |
|
| 10 |
+
| 量化 | U16(LSTM/Pow/Sqrt/Div/conv_bias FP32)|
|
| 11 |
+
| ONNX 大小 | 38 MB |
|
| 12 |
+
| AXMODEL 大小 | 27.6 MB |
|
| 13 |
+
|
| 14 |
+
## 精度(ONNX vs AXMODEL,8 个样本)
|
| 15 |
+
|
| 16 |
+
| 指标 | mean ± std |
|
| 17 |
+
|------|-----------|
|
| 18 |
+
| cosine_similarity | 0.99922 ± 0.00039(min 0.99850)|
|
| 19 |
+
| MAE | 0.00888 ± 0.00066 |
|
| 20 |
+
| max_abs_diff | 1.37 ± 0.87 |
|
| 21 |
+
| 增强波形 cosine(首样本)| 0.99974 |
|
| 22 |
+
|
| 23 |
+
## 板端延迟(AX650N,NPU3 三核)
|
| 24 |
+
|
| 25 |
+
测试音频 9.77 秒(156302 样本,4 个 4 秒分块 + 尾部)。
|
| 26 |
+
|
| 27 |
+
| 语言 | 处理总耗时 | RTF |
|
| 28 |
+
|------|-----------|-----|
|
| 29 |
+
| Python(axengine)| 0.28 s | 0.0287 |
|
| 30 |
+
| C++(axengine)| ~28 s(含直接 DFT 预处理)| 2.9 |
|
| 31 |
+
|
| 32 |
+
说明:
|
| 33 |
+
- Python RTF 0.029 为完整链路(STFT + NPU + ISTFT + 拼接),NPU 单模型推理约 0.064 s/块;
|
| 34 |
+
- C++ 耗时主要花在 320 点直接 DFT(每块 401×161×320 复数乘加),若接入 FFT 库可降到亚秒级;
|
| 35 |
+
- 板端内存占用:模型 + 双 SDK 运行峰值 < 512 MB(板总内存 3.4 GB)。
|
| 36 |
+
|
| 37 |
+
## 编译信息
|
| 38 |
+
|
| 39 |
+
- Pulsar2 镜像:`pulsar2:7.0-lite`
|
| 40 |
+
- 目标:AX650 / NPU3;`highest_mix_precision=false`
|
| 41 |
+
- 编译耗时:约 1 小时 51 分(LSTM 展开约 30 万算子)
|
reports/runonboard_report.md
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# RunOnBoard Report
|
| 2 |
+
|
| 3 |
+
- board: AX650N_CHIP(Engine 2.12.0s, Compiler 7.0 6b1bcdf8)
|
| 4 |
+
- model: model.axmodel (NPU3, triple core)
|
| 5 |
+
|
| 6 |
+
## Python SDK (gcrn_sdk, axengine backend)
|
| 7 |
+
- input: test_audio/mix.wav (16 kHz mono, 9.77s, 156302 samples)
|
| 8 |
+
- output: enhanced_board.wav
|
| 9 |
+
- elapsed: 0.280s audio: 9.769s RTF: 0.0287
|
| 10 |
+
- board-vs-host ONNX enhanced waveform cosine: 0.999674, MAE 0.001599
|
| 11 |
+
- import gcrn_sdk: OK; model input/output [1,2,401,161] float32
|
| 12 |
+
|
| 13 |
+
## C++ SDK (gcrn_example, cross-compiled aarch64 + BSP)
|
| 14 |
+
- fixed: AX_SYS_Init before AX_ENGINE_Init; WAV 16-bit read/write; [c,t,f] row-major layout; braced-init vector size
|
| 15 |
+
- output: enhanced_cpp.wav (156302 samples)
|
| 16 |
+
- cpp-vs-python board waveform cosine: 0.999997, MAE 0.000063
|
| 17 |
+
- runtime: ~28s (direct DFT 320-point pre/post in C++, no FFT lib on board)
|
| 18 |
+
|
| 19 |
+
## 结论
|
| 20 |
+
- Python/C++ 双 SDK 在 AX650 板端跑通,输出一致,端到端音频质量与 ONNX 参考几乎一致。
|
reports/simulate_report.md
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Simulate Report
|
| 2 |
+
|
| 3 |
+
- Method: board axengine(AX650N 板端实测)
|
| 4 |
+
- Samples: 8
|
| 5 |
+
- cosine_similarity: mean=0.999222 std=0.000389 min=0.998504
|
| 6 |
+
- mae: mean=0.008880 std=0.000658 min=0.007749
|
| 7 |
+
- max_abs_diff: mean=1.373579 std=0.870593 min=0.490195
|
| 8 |
+
- audio_level: {"enhanced_waveform_cosine": 0.9997435882461786, "enhanced_waveform_mae": 0.0013857042649760842}
|
run.sh
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
set -euo pipefail
|
| 3 |
+
|
| 4 |
+
echo "=== 运行 GCRN 语音增强推理 ==="
|
| 5 |
+
if python3 -c "import axengine" 2>/dev/null; then
|
| 6 |
+
python3 python/demo.py --model models/model.axmodel --input test_audio/mix.wav --output enhanced.wav
|
| 7 |
+
else
|
| 8 |
+
echo "未检测到 axengine(本机无 NPU),自动使用 ONNX 模型在 CPU 上演示:"
|
| 9 |
+
python3 python/demo.py --model model_convert/model.onnx --input test_audio/mix.wav --output enhanced.wav
|
| 10 |
+
fi
|
| 11 |
+
|
| 12 |
+
echo "输出音频: enhanced.wav"
|
setup.sh
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
set -euo pipefail
|
| 3 |
+
|
| 4 |
+
echo "=== 安装 Python 依赖 ==="
|
| 5 |
+
pip install numpy>=1.24 onnxruntime>=1.16
|
| 6 |
+
|
| 7 |
+
echo "=== 尝试安装 axengine(NPU 推理;仅 AX650 板端可用)==="
|
| 8 |
+
if python3 -c "import axengine" 2>/dev/null; then
|
| 9 |
+
echo "axengine 已就绪"
|
| 10 |
+
else
|
| 11 |
+
if pip install axengine 2>/dev/null; then
|
| 12 |
+
echo "axengine 安装成功"
|
| 13 |
+
else
|
| 14 |
+
echo "⚠ 本机无法安装 axengine(通常因为不是 AX 板端环境)。"
|
| 15 |
+
echo " 板端请先安装匹配的 axengine wheel,例如:"
|
| 16 |
+
echo " pip3 install axengine-<版本>-py3-none-any.whl"
|
| 17 |
+
fi
|
| 18 |
+
fi
|
| 19 |
+
|
| 20 |
+
echo "=== C++ SDK(可选)==="
|
| 21 |
+
echo " 需要 AX650 BSP SDK(含 ax_engine 头文件与库),然后:"
|
| 22 |
+
echo " mkdir -p cpp/build && cd cpp/build"
|
| 23 |
+
echo " cmake .. -DAX650_SDK_ROOT=/path/to/bsp/out && make -j$(nproc)"
|
| 24 |
+
|
| 25 |
+
echo "✅ 环境准备完成"
|
test_audio/mix.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8d47e1d03eeb457c2549be79c8ec33a349ccd79f21c3add05f946f8f760c5a99
|
| 3 |
+
size 312648
|