Alibaba upgrades real-time speech large model Fun-ASR-Realtime: first-word latency in hundreds of milliseconds, Wenzhou dialect recognition accuracy exceeds 82%

News on July 6, Alibaba Tongyi Lab has reduced the first-character latency of its streaming speech recognition large model Fun-ASR-Realtime to the millisecond level, achieving real-time character output as soon as speech ends, with accuracy approaching that of offline models.

In the recently concluded 100-hour desert island live broadcast by Film Storm, the model provided real-time subtitle support throughout the entire event under harsh conditions including outdoor rainstorms and frequent speaker switching, recognizing over 60k entries totaling 1.32 million characters.

To address the pain point of real-time speech recognition being prone to taking things out of context, the new model enhances contextual awareness, combining historical dialogue with real-time hot words for dynamic error correction (for example, automatically correcting "Yelu" to "Night Heron" based on subsequent context). The model currently supports 30 languages and 16 dialects, with an average character accuracy of 88.62% in dialect tests, including 92.41% for Shanghainese and 82.74% for Wenzhounese, which is known as the most difficult dialect.

The offline model Fun-ASR-Flash has achieved first place on the word error rate leaderboard of the global AI evaluation platform Artificial Analysis. The API services for both models are now available on Alibaba Cloud Bailian, and the underlying open-source toolkit and models can be obtained on ModelScope and GitHub.