Skip to content

feat(asr): support additional Bailian speech models#866

Open
LauraGPT wants to merge 2 commits into
Open-Less:betafrom
LauraGPT:codex/funasr-bailian-model-routing
Open

feat(asr): support additional Bailian speech models#866
LauraGPT wants to merge 2 commits into
Open-Less:betafrom
LauraGPT:codex/funasr-bailian-model-routing

Conversation

@LauraGPT

@LauraGPT LauraGPT commented Jul 26, 2026

Copy link
Copy Markdown

User description

Closes #850

Summary

  • route Bailian model families to classic realtime, Qwen realtime, synchronous multimodal, or asynchronous transcription clients
  • support Fun-ASR, Fun-ASR-MTL, Paraformer file transcription, Qwen3-ASR Flash/filetrans, and 8 kHz Fun-ASR realtime models
  • align desktop and mobile routing, provider validation, model hints, localized UI, and documentation
  • implement temporary OSS upload, async task polling, model-specific response parsing, cancellation-aware QA flows, and dynamic long-recording timeouts
  • harden credential and anonymous requests against redirects, untrusted transfer URLs, SSRF, and stalled requests

Verification

  • cargo check --tests
  • cargo test --lib: 773 passed (serial full-suite run)
  • npm run build: 2445 modules transformed
  • npm test: 26 frontend tests passed
  • git diff --check

Notes

DashScope temporary OSS upload is documented as a development and low-concurrency path. The added documentation calls out that limitation for users.


PR Type

Enhancement, Tests


Description

  • Extend Bailian ASR model routing to all supported families

  • Implement async transcription protocol with OSS upload and polling

  • Add 8kHz PCM downsample and chunk size for 8k realtime models

  • Align desktop and mobile routing, model hints, and validation


Diagram Walkthrough

flowchart LR
  model["Model Name"] --> classify{"Model Prefix"}
  classify -->|"fun-asr-realtime* / paraformer-realtime* / sensevoice-realtime* / fun-asr-flash-8k-realtime*"| ws["Classic WebSocket (bailian.rs)"]
  classify -->|"qwen3-asr-flash-realtime*"| qwen["Qwen Realtime WebSocket (qwen_realtime.rs)"]
  classify -->|"fun-asr-flash* / qwen3-asr-flash"| sync["Synchronous Multimodal (dashscope_multimodal.rs)"]
  classify -->|"fun-asr / fun-asr-mtl* / paraformer* / qwen3-asr-flash-filetrans*"| async["Async Transcription (dashscope_multimodal.rs)"]
Loading

File Walkthrough

Relevant files
Enhancement
6 files
bailian.rs
Add 8kHz downsample and chunk size                                             
+131/-7 
dashscope_multimodal.rs
Implement async transcription protocol                                     
+677/-11
providers.rs
Extend model list and validation                                                 
+93/-36 
coordinator.rs
Extend model routing and endpoint protocols                           
+51/-30 
dictation.rs
Use dynamic timeout for multimodal ASR                                     
+1/-1     
ProvidersSection.tsx
Classify model protocol for UI hints                                         
+14/-6   
Bug fix
1 files
qa_session.rs
Add cancel support for DashScope multimodal                           
+34/-21 
Documentation
6 files
en.ts
Update model hint labels for sync/async                                   
+2/-1     
ja.ts
Update model hint labels for sync/async                                   
+2/-1     
ko.ts
Update model hint labels for sync/async                                   
+2/-1     
zh-CN.ts
Update model hint labels for sync/async                                   
+2/-1     
zh-TW.ts
Update model hint labels for sync/async                                   
+2/-1     
bailian-asr-models.md
Document Bailian ASR model support                                             
+32/-0   

@github-actions

github-actions Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

(Review updated until commit a58630b)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

850 - Partially compliant

Compliant requirements:

  • 用户手填 fun-asrqwen3-asr-flashqwen3-asr-flash-filetrans 等官方模型时,不再被 resolve_effective_asr_provider 直接拒绝
  • 每种新协议有对应的 Rust 客户端 + coordinator 分支 + 凭据校验
  • 设置页模型提示能区分:实时 WebSocket / 同步录音文件 / 异步文件转写
  • 现有已支持模型(fun-asr-realtimeqwen3-asr-flash-realtime*fun-asr-flash-2026-06-15)行为不回归
  • 补充单元测试覆盖模型名 → 协议路由表

Non-compliant requirements:

""

Requires further human verification:

""

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ No major issues detected

@LauraGPT

Copy link
Copy Markdown
Author

Follow-up on the PR-Agent 8 kHz tail note: the decimator intentionally keeps source indices 0, 2, 4, ... . Therefore, when a stream ends after three complete 16 kHz samples, the buffered third sample is the next kept 8 kHz sample rather than half of an output pair.

Signed commit a58630b extracts that tail flush into a named helper and adds a regression proving [1, 2, 3] -> [1, 3] at end of stream. Fresh verification passed cargo check --tests and all 773 Rust tests serially. (Two unrelated localhost connection-pool tests raced in one parallel run; both passed in isolation, and the full serial run is 773/773.)

@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit a58630b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[asr] 放宽百炼 ASR 模型白名单,支持官方文档中的更多模型

1 participant