This PR adds support for FireRedASR CTC models to the sherpa-onnx offline speech recognition system. The implementation includes new model configuration classes, integration into the existing CTC recognition pipeline, and comprehensive testing infrastructure.
This pull request significantly enhances the Whisper integration by introducing comprehensive timestamping capabilities. It provides both fine-grained token-level timestamps, derived from cross-attention weights using Dynamic Time Warping, and broader segment-level timestamps, based on Whisper's native timestamp tokens. This allows for more detailed temporal alignment of transcribed text with audio, which is crucial for applications like subtitle generation and improving the handling of long audio inputs. The changes span model export utilities, core C++ logic, and Python API bindings to fully support these new features.
This pull request integrates support for the latest Meta Omnilingual ASR v2 models, enhancing the project's multilingual speech recognition capabilities without requiring modifications to the core C++ codebase. It introduces a dedicated Python example to facilitate usage and demonstrates the model's performance and compatibility.
This pull request significantly enhances the FunASR-nano model integration by introducing advanced control over the transcription process. Users can now leverage hotwords to guide recognition, specify the target language, and manage Inverse Text Normalization (ITN) settings. These capabilities are seamlessly exposed through comprehensive API updates across Python, Java, Kotlin, and Swift. Additionally, the underlying C++ implementation benefits from performance optimizations via CPU IoBinding and pre-allocated buffers, alongside more informative error messages and debugging logs, contributing to a more robust and user-friendly experience.
This pull request significantly enhances the PocketTTS system's ability to process long input texts. By introducing a sophisticated text segmentation and chunking mechanism, the system can now reliably handle extensive content, breaking it down into smaller, more manageable units for synthesis. This refactoring improves the overall stability and user experience when generating speech from lengthy passages, as demonstrated by the updated example script.
This PR updates FunASR-nano inference from the prefill+decode dual-model pipeline to a single unified KV-cache model.
## Summary
Previously, FunASR-nano required two separate ONNX models:
- `llm_prefill.onnx`
- `llm_decode.onnx`
This PR switches to a single model:
- `llm.onnx`
The new pipeline uses a static KV cache + KV-delta incremental update mechanism, and relies on `cache_position` to differentiate prefill vs. decode steps. This significantly simplifies model/session management and reduces deployment complexity.
## Key changes
- **Single LLM session / single model file**: `llm.onnx` replaces `llm_prefill.onnx` + `llm_decode.onnx`.
- **Unified KV-cache implementation**:
- static KV cache layout
- KV-delta update for decode
- `cache_position` distinguishes prefill vs. decode behavior
- **Config changes (breaking)**:
- `funasr_nano.llm_prefill` and `funasr_nano.llm_decode` are deprecated/removed
- use only `funasr_nano.llm`
- **Not backward compatible**:
- users must re-export models in KV-delta/unified-KV format
- **Trade-off**: slightly slower, but lower VRAM duplication
This pull request significantly enhances sherpa-onnx by integrating comprehensive support for Google MedASR CTC models. The changes encompass the foundational C++ model implementation, a streamlined Python API for ease of use, and specialized audio feature processing and text post-processing logic to ensure accurate medical speech recognition. This integration also includes updates to the build system and continuous integration tests, alongside a practical Python example, making MedASR models readily available and verifiable within the framework.
This pull request addresses and resolves building errors by refactoring the OfflineTtsZipVoiceModelConfig across various language bindings. The core change involves renaming the pinyinDict parameter to lexicon to standardize terminology and improve clarity in text-to-speech configurations. Additionally, the Python example script for offline zero-shot TTS has been updated to align with these changes, including new model paths and example content, ensuring the examples remain functional and up-to-date.
This PR adds C++ and Python API support for Omnilingual ASR models with CTC decoding, enabling speech recognition across 1600+ languages using models from Facebook Research's omnilingual-asr project.
Key changes:
- Implements OfflineOmnilingualAsrCtcModel class with feature normalization and model inference
- Adds Python bindings through from_omnilingual_asr_ctc() method on OfflineRecognizer
- Integrates omnilingual models into existing CTC recognition pipeline with special handling for input shapes and frame shifts
This PR removes the cppjieba dependency from the sherpa-onnx project by replacing its usage with character-based text processing. The main purpose is to simplify the codebase by eliminating the need for external jieba dictionary files and the cppjieba library.
- Replaces jieba-based word segmentation with UTF-8 character-level tokenization
- Removes all references to dict_dir and dictDir parameters across APIs
- Adds a new CharacterLexicon class to replace JiebaLexicon
This PR adds support for T-one streaming Russian ASR models in both C++ and Python APIs. The T-one model is a CTC-based Russian speech recognition model with specific characteristics including float16 state handling, 300ms frame lengths, and 8kHz sampling rate.
- Added new OnlineToneCtcModel implementation with specialized processing for T-one models
- Integrated T-one support into the existing CTC model pipeline and Python bindings
- Added Python example and test scripts for the new functionality
A new method and property were introduced in the VoiceActivityDetector C++ and Python APIs to
provide access to the current speech segment as soon as speech is detected, rather than only
after the segment completes.
This PR adds support for the TEN VAD model alongside the existing Silero VAD in both C++ and Python interfaces.
- Introduces TenVadModelConfig with Python bindings and integrates it into VadModelConfig.
- Implements TenVadModel in C++ and extends the factory (VadModel::Create) and detector logic to choose between Silero and TEN VAD.
- Updates build files (CMake), fixes a spelling typo, and extends the Python example script to demonstrate --ten-vad-model.
This PR integrates LODR (Level-Ordered Deterministic Rescoring) support from Icefall into both online and offline recognizers, enabling LODR for LM shallow fusion and LM rescore.
- Extended OnlineLMConfig and OfflineLMConfig to include lodr_fst, lodr_scale, and lodr_backoff_id.
- Implemented LodrFst and LodrStateCost classes and wired them into RNN LM scoring in both online and offline code paths.
- Updated Python bindings, CLI entry points, examples, and CI test scripts to accept and exercise the new LODR options.
This PR adds support for non-streaming Zipformer CTC ASR models across
multiple language bindings, WebAssembly, examples, and CI workflows.
- Introduces a new OfflineZipformerCtcModelConfig in C/C++, Python, Swift, Java, Kotlin, Go, Dart, Pascal, and C# APIs
- Updates initialization, freeing, and recognition logic to include Zipformer CTC in WASM and Node.js
- Adds example scripts and CI steps for downloading, building, and running Zipformer CTC models
Model doc is available at
https://k2-fsa.github.io/sherpa/onnx/pretrained_models/offline-ctc/icefall/zipformer.html
Fangjun Kuang
Joe Cheng
Edison dos Santos
Wasser1462
Fangjun Kuang
Wei Kangandyaozengwei
Brad Murray
AlexWhisperandwangmh
Askars Salimbajevs
luffy
JV_X
goddamnVincent