3v324v23 commited on
Commit
025dd96
·
1 Parent(s): f9a0a8f

Use Audio8 native compile batch policy

Browse files
Files changed (2) hide show
  1. Dockerfile +1 -2
  2. README.md +2 -2
Dockerfile CHANGED
@@ -13,7 +13,7 @@ RUN apt-get update && \
13
  WORKDIR /home/user/app
14
 
15
  ARG SGLANG_OMNI_REVISION=68a572348837f7b004857b4b07993c20ade4c017
16
- ARG AUDIO8_TTS_REVISION=5bdcbaddee56599483d7ecf422a9595dc5e5ab32
17
 
18
  RUN git init /opt/sglang-omni && \
19
  git -C /opt/sglang-omni remote add origin https://github.com/sgl-project/sglang-omni.git && \
@@ -71,7 +71,6 @@ ENV HOME=/home/user \
71
  AUDIO8_TTS_MAX_RUNNING_REQUESTS=8 \
72
  AUDIO8_TTS_DISABLE_CUDA_GRAPH=0 \
73
  AUDIO8_TTS_ENABLE_TORCH_COMPILE=1 \
74
- AUDIO8_TTS_TORCH_COMPILE_MAX_BS=1 \
75
  AUDIO8_TTS_VERIFY_FAST_KV=1 \
76
  FLASHINFER_WORKSPACE_BASE=/tmp/audio8-flashinfer \
77
  TORCHINDUCTOR_CACHE_DIR=/tmp/audio8-torchinductor \
 
13
  WORKDIR /home/user/app
14
 
15
  ARG SGLANG_OMNI_REVISION=68a572348837f7b004857b4b07993c20ade4c017
16
+ ARG AUDIO8_TTS_REVISION=59ca6d27d441302eaa568bcb140b63c50a879859
17
 
18
  RUN git init /opt/sglang-omni && \
19
  git -C /opt/sglang-omni remote add origin https://github.com/sgl-project/sglang-omni.git && \
 
71
  AUDIO8_TTS_MAX_RUNNING_REQUESTS=8 \
72
  AUDIO8_TTS_DISABLE_CUDA_GRAPH=0 \
73
  AUDIO8_TTS_ENABLE_TORCH_COMPILE=1 \
 
74
  AUDIO8_TTS_VERIFY_FAST_KV=1 \
75
  FLASHINFER_WORKSPACE_BASE=/tmp/audio8-flashinfer \
76
  TORCHINDUCTOR_CACHE_DIR=/tmp/audio8-torchinductor \
README.md CHANGED
@@ -17,8 +17,8 @@ The service uses the official Audio8 adapter with SGLang Omni 0.1.0 and
17
  SGLang 0.5.8.
18
 
19
  Reference audio uploaded through the web interface is limited to 30 seconds.
20
- The A10G runtime enables `torch.compile` for batch size 1 and uses CUDA Graph
21
- for larger batches.
22
 
23
  ## API
24
 
 
17
  SGLang 0.5.8.
18
 
19
  Reference audio uploaded through the web interface is limited to 30 seconds.
20
+ The A10G runtime enables `torch.compile` with SGLang's native batch policy and
21
+ uses CUDA Graph without an Audio8-specific compile batch-size limit.
22
 
23
  ## API
24