inoryQwQ commited on
Commit
5e23710
·
verified ·
1 Parent(s): 00ee7dc

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ models/model.axmodel filter=lfs diff=lfs merge=lfs -text
37
+ test_audio/mix.wav filter=lfs diff=lfs merge=lfs -text
.gitignore ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ __pycache__/
2
+ *.pyc
3
+ build/
4
+ CMakeFiles/
5
+ CMakeCache.txt
6
+ *.egg-info/
README.md ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: audio-to-audio
4
+ tags:
5
+ - axmodel
6
+ - axera
7
+ - gcrn
8
+ - speech-enhancement
9
+ ---
10
+ # GCRN 语音增强(AX650 NPU3)
11
+
12
+ GCRN(Graph Convolutional Recurrent Network)单声道语音增强模型,输入 16 kHz 噪声语音,
13
+ 输出同长增强语音。本包为 AX650 / NPU3 量化部署版:
14
+
15
+ - 精度:AXMODEL 与 ONNX 参考余弦相似度 **0.9992**(增强波形余弦 0.9997)
16
+ - 速度:AX650 板端 **RTF 0.029**(9.77 秒音频约 0.28 秒处理完)
17
+ - 大小:AXMODEL **27.6 MB**(ONNX 38 MB)
18
+
19
+ ## 快速开始(两步)
20
+
21
+ ### 1. 安装环境
22
+
23
+ ```bash
24
+ bash setup.sh
25
+ ```
26
+
27
+ ### 2. 跑推理
28
+
29
+ ```bash
30
+ bash run.sh
31
+ ```
32
+
33
+ 脚本会自动选择后端:板端(有 axengine)跑 NPU 模型,普通电脑自动用 ONNX 模型演示。
34
+ 输出为 `enhanced.wav`,增强前后的对比音频可以自己听一下。
35
+
36
+ ## 目录说明
37
+
38
+ | 目录 | 用途 |
39
+ |------|------|
40
+ | `models/` | AXMODEL 模型 + `model_meta.json` |
41
+ | `python/` | Python SDK(axengine / onnxruntime 双后端)|
42
+ | `cpp/` | C++ SDK(AX Engine runtime,aarch64 交叉编译)|
43
+ | `model_convert/` | 导出、校准、Pulsar2 编译全套可复现脚本 |
44
+ | `reports/` | 导出/编译/仿真/板端验证报告 |
45
+ | `test_audio/` | 自带 9.77 秒测试音频 |
46
+
47
+ ## 在自己的代码里用
48
+
49
+ ```python
50
+ from gcrn_sdk import GCRNDenoiser
51
+
52
+ denoiser = GCRNDenoiser("models/model.axmodel")
53
+ report = denoiser.enhance_file("test_audio/mix.wav", "enhanced.wav")
54
+ print(report) # 含 RTF 等性能指标
55
+ ```
56
+
57
+ ## 常见问题
58
+
59
+ **Q: import 报错找不到 axengine?**
60
+ A: 说明当前不在 AX 板端。代码会自动用 ONNX 模型演示;要在 NPU 上跑,需在 AX650 板端
61
+ 安装匹配的 axengine wheel(见 `setup.sh` 输出提示)。
62
+
63
+ **Q: 想自己重新编译 AXMODEL?**
64
+ A: 进入 `model_convert/` 按 README 操作,Pulsar2 会从 ONNX 重新量化编译。
65
+ 原始编译使用 Docker 镜像 `pulsar2:7.0-lite`,配置为 NPU3 + U16 高精度模式。
66
+
67
+ **Q: 输入音频有什么要求?**
68
+ A: 16 kHz、单声道、16-bit PCM WAV;更长音频会自动按 4 秒分块处理并拼接。
config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {}
models/model.axmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:02e401e25dcf6a70e3d3dc31e154d7f6001278ec76b4c2c7f5a59aea88710aa6
3
+ size 28943407
models/model_meta.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_name": "gcrn_fixed_4sec",
3
+ "framework": "onnx",
4
+ "source": "gcrn_fixed_4sec.onnx",
5
+ "source_sha256": "9e8e0acd13126c9d5d19aae6f83f230a16c8914f100744153c78c58920d1ff4a",
6
+ "inputs": [
7
+ {
8
+ "name": "stft_input",
9
+ "shape": [
10
+ 1,
11
+ 2,
12
+ 401,
13
+ 161
14
+ ],
15
+ "dtype": "float32",
16
+ "layout": "NCHW"
17
+ }
18
+ ],
19
+ "outputs": [
20
+ {
21
+ "name": "stft_output",
22
+ "shape": [
23
+ 1,
24
+ 2,
25
+ 401,
26
+ 161
27
+ ],
28
+ "dtype": "float32"
29
+ }
30
+ ],
31
+ "opset": 12,
32
+ "onnxsim_cosine_min": 1.0,
33
+ "preprocess": {
34
+ "description": "STFT of 16 kHz mono waveform -> [1,2,401,161] (batch, real/imag, time, freq)",
35
+ "sample_rate": 16000,
36
+ "duration_sec": 4.0,
37
+ "n_fft": 320,
38
+ "hop_length": 160,
39
+ "win_length": 320,
40
+ "center": true,
41
+ "window": "hamming(sym=True)"
42
+ },
43
+ "notes": [
44
+ "\u8f93\u5165\u4e3a STFT \u590d\u6570\u8c31\uff08\u5b9e\u90e8/\u865a\u90e8\uff09\uff0c\u65e0 mean/std \u5f52\u4e00\u5316"
45
+ ]
46
+ }
python/README.md ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # GCRN Python SDK
2
+
3
+ GCRN 语音增强 Python 推理 SDK。模型输入 16 kHz 单声道噪声语音的 STFT 谱
4
+ `[1, 2, 401, 161]`(4 秒分块),输出增强谱,再还原为增强波形。
5
+
6
+ ## 安装
7
+
8
+ ```bash
9
+ pip install -r requirements.txt
10
+ ```
11
+
12
+ `axengine` 需要 AX650 板端(或带 AX runtime 的主机);普通电脑可以装 `onnxruntime`
13
+ 后用 ONNX 模型调试。
14
+
15
+ ## 快速使用
16
+
17
+ ```bash
18
+ python3 demo.py --model ../models/model.axmodel --input ../test_audio/mix.wav --output enhanced.wav
19
+ ```
20
+
21
+ ## API
22
+
23
+ ```python
24
+ from gcrn_sdk import GCRNDenoiser
25
+
26
+ denoiser = GCRNDenoiser("models/model.axmodel") # .onnx 自动走 onnxruntime
27
+ report = denoiser.enhance_file("noisy.wav", "enhanced.wav")
28
+ print(report["real_time_factor"])
29
+
30
+ # 直接处理 int16 PCM:
31
+ pcm = ... # np.int16
32
+ enhanced = denoiser.enhance(pcm)
33
+ ```
34
+
35
+ ## 预处理说明
36
+
37
+ - 输入要求:16 kHz、单声道、16-bit PCM WAV;
38
+ - 分块:4 秒(64000 样本),不足补零,逐块增强后拼接;
39
+ - STFT:`n_fft=320, hop=160, win=320`,hamming(sym)窗,与模型训练端一致;
40
+ - 模型边界:`stft_input [1,2,401,161] float32` -> `stft_output [1,2,401,161] float32`
41
+ (布局为 batch, real/imag, time, freq)。
python/demo.py ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ # coding: utf-8
3
+ """GCRN AX650 语音增强示例:noisy.wav -> enhanced.wav"""
4
+
5
+ import argparse
6
+ import sys
7
+ from pathlib import Path
8
+
9
+ sys.path.insert(0, str(Path(__file__).resolve().parent))
10
+
11
+ from gcrn_sdk import GCRNDenoiser
12
+
13
+
14
+ def main() -> None:
15
+ ap = argparse.ArgumentParser(description="GCRN speech enhancement on AX650")
16
+ ap.add_argument("--model", default="models/model.axmodel", help="axmodel (或 onnx)")
17
+ ap.add_argument("--input", default="test_audio/mix.wav", help="16kHz mono noisy wav")
18
+ ap.add_argument("--output", default="enhanced.wav", help="输出 wav")
19
+ ap.add_argument("--report", default="gcrn_report.json", help="性能报告输出")
20
+ args = ap.parse_args()
21
+
22
+ denoiser = GCRNDenoiser(args.model)
23
+ report = denoiser.enhance_file(args.input, args.output)
24
+ denoiser.save_report(report, args.report)
25
+ print(f"input={args.input}")
26
+ print(f"output={args.output}")
27
+ print(
28
+ f"elapsed={report['elapsed_seconds']:.3f}s "
29
+ f"audio={report['audio_seconds']:.3f}s "
30
+ f"rtf={report['real_time_factor']:.4f}"
31
+ )
32
+
33
+
34
+ if __name__ == "__main__":
35
+ main()
python/gcrn_sdk/__init__.py ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ # coding: utf-8
2
+ """GCRN 语音增强 AX SDK。"""
3
+
4
+ from .inference import GCRNDenoiser
5
+
6
+ __all__ = ["GCRNDenoiser"]
python/gcrn_sdk/audio.py ADDED
@@ -0,0 +1,103 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # coding: utf-8
2
+ """GCRN 依赖极简的音频预处理:16kHz 单声道 WAV <-> STFT [1,2,401,161]。
3
+
4
+ 与 GCRN.AXERA 的 board_inference/audio.py 保持一致(hamming sym 窗 + numpy rfft)。
5
+ """
6
+ from __future__ import annotations
7
+
8
+ import wave
9
+ from pathlib import Path
10
+
11
+ import numpy as np
12
+
13
+ SAMPLE_RATE = 16000
14
+ CHUNK_SAMPLES = 4 * SAMPLE_RATE
15
+ N_FFT = 320
16
+ HOP_LENGTH = 160
17
+ WIN_LENGTH = 320
18
+
19
+
20
+ def read_wav(path: str | Path) -> tuple[np.ndarray, int]:
21
+ with wave.open(str(path), "rb") as source:
22
+ if source.getnchannels() != 1 or source.getsampwidth() != 2:
23
+ raise ValueError(f"expected mono 16-bit PCM WAV: {path}")
24
+ sample_rate = source.getframerate()
25
+ samples = np.frombuffer(
26
+ source.readframes(source.getnframes()), dtype=np.int16
27
+ ).copy()
28
+ return samples, sample_rate
29
+
30
+
31
+ def write_wav(path: str | Path, samples: np.ndarray, sample_rate: int) -> None:
32
+ values = np.asarray(samples, dtype=np.float32).reshape(-1)
33
+ pcm = np.clip(values, -1.0, 1.0)
34
+ pcm = np.rint(pcm * 32767.0).astype(np.int16)
35
+ path = Path(path)
36
+ path.parent.mkdir(parents=True, exist_ok=True)
37
+ with wave.open(str(path), "wb") as output:
38
+ output.setnchannels(1)
39
+ output.setsampwidth(2)
40
+ output.setframerate(sample_rate)
41
+ output.writeframes(pcm.tobytes())
42
+
43
+
44
+ def _symmetric_hamming(size: int) -> np.ndarray:
45
+ index = np.arange(size, dtype=np.float32)
46
+ return (0.54 - 0.46 * np.cos(2.0 * np.pi * index / (size - 1))).astype(
47
+ np.float32
48
+ )
49
+
50
+
51
+ def chunks(samples: np.ndarray) -> list[tuple[np.ndarray, int]]:
52
+ """int16 PCM -> 4 秒 float32 chunk 序列 [(chunk, valid_samples)]。"""
53
+ values = np.asarray(samples, dtype=np.float32).reshape(-1) / 32768.0
54
+ if values.size == 0:
55
+ raise ValueError("input WAV is empty")
56
+ result = []
57
+ for start in range(0, values.size, CHUNK_SAMPLES):
58
+ end = min(start + CHUNK_SAMPLES, values.size)
59
+ valid = end - start
60
+ chunk = np.zeros(CHUNK_SAMPLES, dtype=np.float32)
61
+ chunk[:valid] = values[start:end]
62
+ result.append((chunk, valid))
63
+ return result
64
+
65
+
66
+ def stft(chunk: np.ndarray) -> np.ndarray:
67
+ """4 秒波形 -> [1, 2, 401, 161](batch, real/imag, time, freq)。"""
68
+ signal = np.asarray(chunk, dtype=np.float32).reshape(-1)
69
+ if signal.size != CHUNK_SAMPLES:
70
+ raise ValueError(f"expected {CHUNK_SAMPLES} samples, got {signal.size}")
71
+ pad = N_FFT // 2
72
+ signal = np.pad(signal, (pad, pad), mode="constant")
73
+ frame_count = 1 + (signal.size - WIN_LENGTH) // HOP_LENGTH
74
+ frames = np.lib.stride_tricks.sliding_window_view(signal, WIN_LENGTH)[
75
+ ::HOP_LENGTH
76
+ ][:frame_count]
77
+ spectrum = np.fft.rfft(frames * _symmetric_hamming(WIN_LENGTH)[None, :], axis=1)
78
+ value = np.stack((spectrum.real, spectrum.imag), axis=0)
79
+ return value.astype(np.float32)[None, ...]
80
+
81
+
82
+ def istft(value: np.ndarray) -> np.ndarray:
83
+ """[1,2,401,161] 增强谱 -> 64000 个 float32 样本。"""
84
+ output = np.asarray(value, dtype=np.float32)
85
+ if output.shape != (1, 2, 401, 161):
86
+ raise ValueError(f"unexpected GCRN output shape: {output.shape}")
87
+ value = output[0]
88
+ spectrum = value[0].astype(np.float64) + 1j * value[1].astype(np.float64)
89
+ frames = np.fft.irfft(spectrum, n=N_FFT, axis=1)
90
+ window = _symmetric_hamming(WIN_LENGTH).astype(np.float64)
91
+ frames *= window[None, :]
92
+
93
+ output_length = N_FFT + HOP_LENGTH * (frames.shape[0] - 1)
94
+ waveform = np.zeros(output_length, dtype=np.float64)
95
+ window_sum = np.zeros(output_length, dtype=np.float64)
96
+ for index, frame in enumerate(frames):
97
+ start = index * HOP_LENGTH
98
+ waveform[start : start + N_FFT] += frame
99
+ window_sum[start : start + N_FFT] += window * window
100
+ valid = window_sum > np.finfo(np.float64).eps
101
+ waveform[valid] /= window_sum[valid]
102
+ pad = N_FFT // 2
103
+ return waveform[pad:-pad].astype(np.float32)
python/gcrn_sdk/inference.py ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # coding: utf-8
2
+ """GCRN 语音增强推理:4 秒 chunk STFT -> axmodel -> ISTFT -> 拼接输出。"""
3
+ from __future__ import annotations
4
+
5
+ import json
6
+ import time
7
+ from pathlib import Path
8
+
9
+ import numpy as np
10
+
11
+ from .audio import (
12
+ SAMPLE_RATE,
13
+ chunks,
14
+ istft,
15
+ read_wav,
16
+ stft,
17
+ write_wav,
18
+ )
19
+ from .runtime import InferenceSession, first_output
20
+
21
+ INPUT_NAME = "stft_input"
22
+ OUTPUT_NAME = "stft_output"
23
+ INPUT_SHAPE = (1, 2, 401, 161)
24
+
25
+
26
+ class GCRNDenoiser:
27
+ """AX650 GCRN 语音增强器。默认 .axmodel + axengine。"""
28
+
29
+ def __init__(self, model_path: str | Path, backend: str | None = None):
30
+ self.session = InferenceSession(model_path, backend)
31
+ if len(self.session.inputs) != 1 or self.session.inputs[0].shape != INPUT_SHAPE:
32
+ raise RuntimeError(f"unexpected GCRN input metadata: {self.session.inputs}")
33
+ if len(self.session.outputs) != 1 or self.session.outputs[0].shape != INPUT_SHAPE:
34
+ raise RuntimeError(f"unexpected GCRN output metadata: {self.session.outputs}")
35
+ self.input_name = self.session.inputs[0].name
36
+
37
+ def _warmup(self) -> None:
38
+ feed = stft(np.zeros(64000, dtype=np.float32))
39
+ first_output(self.session, self.session.run({self.input_name: feed}))
40
+
41
+ def enhance_chunk(self, chunk: np.ndarray) -> np.ndarray:
42
+ feed = stft(chunk)
43
+ outputs = self.session.run({self.input_name: feed})
44
+ output = np.asarray(first_output(self.session, outputs), dtype=np.float32)
45
+ return istft(output)
46
+
47
+ def enhance(self, pcm: np.ndarray) -> np.ndarray:
48
+ """int16 PCM -> 增强后 float32 波形(与输入等长)。"""
49
+ self._warmup()
50
+ enhanced = [
51
+ self.enhance_chunk(chunk)
52
+ for chunk, _valid in chunks(pcm)
53
+ ]
54
+ return np.concatenate(enhanced)[: pcm.size]
55
+
56
+ def enhance_file(
57
+ self, input_path: str | Path, output_path: str | Path
58
+ ) -> dict:
59
+ """输入 16kHz 单声道 WAV,输出增强 WAV + 性能报告 dict。"""
60
+ samples, sample_rate = read_wav(input_path)
61
+ if sample_rate != SAMPLE_RATE:
62
+ raise ValueError(
63
+ f"GCRN requires {SAMPLE_RATE} Hz audio, got {sample_rate}"
64
+ )
65
+ start = time.perf_counter()
66
+ enhanced = self.enhance(samples)
67
+ elapsed = time.perf_counter() - start
68
+ write_wav(output_path, enhanced, sample_rate)
69
+ audio_seconds = samples.size / sample_rate
70
+ report = {
71
+ "backend": self.session.backend,
72
+ "model": str(self.session.path),
73
+ "input": str(input_path),
74
+ "output": str(output_path),
75
+ "input_samples": int(samples.size),
76
+ "output_samples": int(enhanced.size),
77
+ "elapsed_seconds": round(elapsed, 4),
78
+ "audio_seconds": round(audio_seconds, 4),
79
+ "real_time_factor": round(elapsed / audio_seconds, 4),
80
+ }
81
+ return report
82
+
83
+ def save_report(self, report: dict, path: str | Path) -> None:
84
+ path = Path(path)
85
+ path.parent.mkdir(parents=True, exist_ok=True)
86
+ path.write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")
python/gcrn_sdk/runtime.py ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # coding: utf-8
2
+ """axengine / onnxruntime 推理会话封装。"""
3
+ from __future__ import annotations
4
+
5
+ from dataclasses import dataclass
6
+ from pathlib import Path
7
+ from typing import Any
8
+
9
+ import numpy as np
10
+
11
+
12
+ @dataclass(frozen=True)
13
+ class TensorInfo:
14
+ name: str
15
+ shape: tuple[int, ...]
16
+ dtype: np.dtype
17
+
18
+
19
+ def _numpy_dtype(value: Any) -> np.dtype:
20
+ text = str(value).lower()
21
+ mapping = (
22
+ (("tensor(float)", "float32", "fp32", "f32"), np.float32),
23
+ (("tensor(float16)", "float16", "fp16", "f16"), np.float16),
24
+ (("tensor(int64)", "int64", "s64"), np.int64),
25
+ (("tensor(int32)", "int32", "s32"), np.int32),
26
+ (("tensor(uint16)", "uint16", "u16"), np.uint16),
27
+ (("tensor(uint8)", "uint8", "u8"), np.uint8),
28
+ )
29
+ for aliases, dtype in mapping:
30
+ if any(alias in text for alias in aliases):
31
+ return np.dtype(dtype)
32
+ raise ValueError(f"unsupported runtime tensor dtype: {value}")
33
+
34
+
35
+ def _tensor_info(value: Any) -> TensorInfo:
36
+ shape = getattr(value, "shape", None)
37
+ if shape is None:
38
+ shape = getattr(value, "dims", None)
39
+ if shape is None or any(dim is None for dim in shape):
40
+ raise ValueError(f"dynamic or missing tensor shape for {value.name}: {shape}")
41
+ dtype = getattr(value, "dtype", None)
42
+ if dtype is None:
43
+ dtype = getattr(value, "type", None)
44
+ return TensorInfo(
45
+ name=value.name,
46
+ shape=tuple(int(dim) for dim in shape),
47
+ dtype=_numpy_dtype(dtype),
48
+ )
49
+
50
+
51
+ class InferenceSession:
52
+ """.axmodel -> axengine;.onnx -> onnxruntime CPU。"""
53
+
54
+ def __init__(self, model_path: str | Path, backend: str | None = None):
55
+ self.path = Path(model_path)
56
+ if not self.path.is_file():
57
+ raise FileNotFoundError(self.path)
58
+ if backend is None:
59
+ backend = "axengine" if self.path.suffix == ".axmodel" else "onnx"
60
+ self.backend = backend
61
+ if backend == "axengine":
62
+ try:
63
+ import axengine
64
+ except ImportError as error:
65
+ raise RuntimeError(
66
+ "axengine is unavailable; run this backend on an AXERA board"
67
+ ) from error
68
+ self._session = axengine.InferenceSession(str(self.path))
69
+ elif backend == "onnx":
70
+ try:
71
+ import onnxruntime as ort
72
+ except ImportError as error:
73
+ raise RuntimeError("onnxruntime is required for --backend onnx") from error
74
+ options = ort.SessionOptions()
75
+ options.inter_op_num_threads = 1
76
+ options.intra_op_num_threads = 1
77
+ self._session = ort.InferenceSession(
78
+ str(self.path),
79
+ sess_options=options,
80
+ providers=["CPUExecutionProvider"],
81
+ )
82
+ else:
83
+ raise ValueError(f"unsupported backend: {backend}")
84
+
85
+ self.inputs = [_tensor_info(value) for value in self._session.get_inputs()]
86
+ self.outputs = [_tensor_info(value) for value in self._session.get_outputs()]
87
+ self.input_by_name = {value.name: value for value in self.inputs}
88
+
89
+ def run(self, feed: dict[str, np.ndarray]) -> dict[str, np.ndarray]:
90
+ missing = [value.name for value in self.inputs if value.name not in feed]
91
+ if missing:
92
+ raise KeyError(f"missing inputs for {self.path.name}: {missing}")
93
+ prepared = {
94
+ name: np.ascontiguousarray(np.asarray(feed[name], dtype=meta.dtype))
95
+ for name, meta in self.input_by_name.items()
96
+ }
97
+ values = self._session.run(None, prepared)
98
+ if isinstance(values, dict):
99
+ return {name: np.asarray(value) for name, value in values.items()}
100
+ if not isinstance(values, (list, tuple)):
101
+ values = [values]
102
+ if len(values) != len(self.outputs):
103
+ raise RuntimeError(
104
+ f"unexpected output count from {self.path.name}: "
105
+ f"{len(values)} != {len(self.outputs)}"
106
+ )
107
+ return {
108
+ meta.name: np.asarray(value)
109
+ for meta, value in zip(self.outputs, values)
110
+ }
111
+
112
+
113
+ def first_output(session: InferenceSession, outputs: dict[str, np.ndarray]) -> np.ndarray:
114
+ return outputs[session.outputs[0].name]
python/requirements.txt ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ # GCRN AX Python SDK 依赖
2
+ # 运行时(板端 / 带 AX runtime 主机):
3
+ axengine>=0.1.3 # pyaxengine:.axmodel NPU 推理
4
+ numpy>=1.24
5
+ # 仅主机调试(无 NPU 时用 onnx 模型 + onnxruntime):
6
+ onnxruntime>=1.16
reports/compile_report.md ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Compile Report
2
+
3
+ - image: pulsar2:7.0-lite
4
+ - target: AX650 (AX650N_CHIP)
5
+ - npu_mode: NPU3
6
+ - input: stft_input:1x2x401x161 (FP32)
7
+ - model: model.onnx -> model.axmodel
8
+ - quant: default U16, Pow/Sqrt/Div/LSTM FP32, conv_bias FP32, MinMax, 25 calib samples
9
+ - macs: 9,799,365,320 (~9.8 GMACs)
10
+ - max_cycle: 38,038,900
11
+ - end-to-end quant cosine (Pulsar2 precision table): 0.99963 (stft_output)
12
+ - axmodel size: 28 MB (source onnx 38 MB, compression 0.74x)
13
+ - compile time: ~1h51m (LSTM unrolled to ~300k ops; host heavily loaded)
14
+ - highest_mix_precision: false
reports/export_report.md ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ # Export Report
2
+
3
+ - ONNX: model.onnx (simplified from gcrn_fixed_4sec.onnx)
4
+ - Input: stft_input [1, 2, 401, 161] float32
5
+ - Output: stft_output [1, 2, 401, 161] float32
6
+ - Opset: 12
7
+ - onnxsim cosine (vs source, 25 windows): min=1.000000 mean=1.000000
8
+ - Calibration data: stft_input.tar.gz (25 samples)
reports/performance_report.md ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # GCRN AX650 性能报告
2
+
3
+ ## 模型概况
4
+
5
+ | 项目 | 数值 |
6
+ |------|------|
7
+ | 模型 | GCRN 语音增强(4 秒固定输入)|
8
+ | 输入/输出 | `[1, 2, 401, 161]` float32 |
9
+ | 计算量 | 9.80 GMACs |
10
+ | 量化 | U16(LSTM/Pow/Sqrt/Div/conv_bias FP32)|
11
+ | ONNX 大小 | 38 MB |
12
+ | AXMODEL 大小 | 27.6 MB |
13
+
14
+ ## 精度(ONNX vs AXMODEL,8 个样本)
15
+
16
+ | 指标 | mean ± std |
17
+ |------|-----------|
18
+ | cosine_similarity | 0.99922 ± 0.00039(min 0.99850)|
19
+ | MAE | 0.00888 ± 0.00066 |
20
+ | max_abs_diff | 1.37 ± 0.87 |
21
+ | 增强波形 cosine(首样本)| 0.99974 |
22
+
23
+ ## 板端延迟(AX650N,NPU3 三核)
24
+
25
+ 测试音频 9.77 秒(156302 样本,4 个 4 秒分块 + 尾部)。
26
+
27
+ | 语言 | 处理总耗时 | RTF |
28
+ |------|-----------|-----|
29
+ | Python(axengine)| 0.28 s | 0.0287 |
30
+ | C++(axengine)| ~28 s(含直接 DFT 预处理)| 2.9 |
31
+
32
+ 说明:
33
+ - Python RTF 0.029 为完整链路(STFT + NPU + ISTFT + 拼接),NPU 单模型推理约 0.064 s/块;
34
+ - C++ 耗时主要花在 320 点直接 DFT(每块 401×161×320 复数乘加),若接入 FFT 库可降到亚秒级;
35
+ - 板端内存占用:模型 + 双 SDK 运行峰值 < 512 MB(板总内存 3.4 GB)。
36
+
37
+ ## 编译信息
38
+
39
+ - Pulsar2 镜像:`pulsar2:7.0-lite`
40
+ - 目标:AX650 / NPU3;`highest_mix_precision=false`
41
+ - 编译耗时:约 1 小时 51 分(LSTM 展开约 30 万算子)
reports/runonboard_report.md ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # RunOnBoard Report
2
+
3
+ - board: AX650N_CHIP(Engine 2.12.0s, Compiler 7.0 6b1bcdf8)
4
+ - model: model.axmodel (NPU3, triple core)
5
+
6
+ ## Python SDK (gcrn_sdk, axengine backend)
7
+ - input: test_audio/mix.wav (16 kHz mono, 9.77s, 156302 samples)
8
+ - output: enhanced_board.wav
9
+ - elapsed: 0.280s audio: 9.769s RTF: 0.0287
10
+ - board-vs-host ONNX enhanced waveform cosine: 0.999674, MAE 0.001599
11
+ - import gcrn_sdk: OK; model input/output [1,2,401,161] float32
12
+
13
+ ## C++ SDK (gcrn_example, cross-compiled aarch64 + BSP)
14
+ - fixed: AX_SYS_Init before AX_ENGINE_Init; WAV 16-bit read/write; [c,t,f] row-major layout; braced-init vector size
15
+ - output: enhanced_cpp.wav (156302 samples)
16
+ - cpp-vs-python board waveform cosine: 0.999997, MAE 0.000063
17
+ - runtime: ~28s (direct DFT 320-point pre/post in C++, no FFT lib on board)
18
+
19
+ ## 结论
20
+ - Python/C++ 双 SDK 在 AX650 板端跑通,输出一致,端到端音频质量与 ONNX 参考几乎一致。
reports/simulate_report.md ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ # Simulate Report
2
+
3
+ - Method: board axengine(AX650N 板端实测)
4
+ - Samples: 8
5
+ - cosine_similarity: mean=0.999222 std=0.000389 min=0.998504
6
+ - mae: mean=0.008880 std=0.000658 min=0.007749
7
+ - max_abs_diff: mean=1.373579 std=0.870593 min=0.490195
8
+ - audio_level: {"enhanced_waveform_cosine": 0.9997435882461786, "enhanced_waveform_mae": 0.0013857042649760842}
run.sh ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+
4
+ echo "=== 运行 GCRN 语音增强推理 ==="
5
+ if python3 -c "import axengine" 2>/dev/null; then
6
+ python3 python/demo.py --model models/model.axmodel --input test_audio/mix.wav --output enhanced.wav
7
+ else
8
+ echo "未检测到 axengine(本机无 NPU),自动使用 ONNX 模型在 CPU 上演示:"
9
+ python3 python/demo.py --model model_convert/model.onnx --input test_audio/mix.wav --output enhanced.wav
10
+ fi
11
+
12
+ echo "输出音频: enhanced.wav"
setup.sh ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+
4
+ echo "=== 安装 Python 依赖 ==="
5
+ pip install numpy>=1.24 onnxruntime>=1.16
6
+
7
+ echo "=== 尝试安装 axengine(NPU 推理;仅 AX650 板端可用)==="
8
+ if python3 -c "import axengine" 2>/dev/null; then
9
+ echo "axengine 已就绪"
10
+ else
11
+ if pip install axengine 2>/dev/null; then
12
+ echo "axengine 安装成功"
13
+ else
14
+ echo "⚠ 本机无法安装 axengine(通常因为不是 AX 板端环境)。"
15
+ echo " 板端请先安装匹配的 axengine wheel,例如:"
16
+ echo " pip3 install axengine-<版本>-py3-none-any.whl"
17
+ fi
18
+ fi
19
+
20
+ echo "=== C++ SDK(可选)==="
21
+ echo " 需要 AX650 BSP SDK(含 ax_engine 头文件与库),然后:"
22
+ echo " mkdir -p cpp/build && cd cpp/build"
23
+ echo " cmake .. -DAX650_SDK_ROOT=/path/to/bsp/out && make -j$(nproc)"
24
+
25
+ echo "✅ 环境准备完成"
test_audio/mix.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d47e1d03eeb457c2549be79c8ec33a349ccd79f21c3add05f946f8f760c5a99
3
+ size 312648