Tencent Hunyuan Hy ASR 3.0 Preview: Context-Aware Speech Recognition for Mandarin, English, and 20 Dialects

Introduction

Tencent Hunyuan has released Hy ASR 3.0 Preview, a new real-time speech-recognition model built around the language-understanding capabilities of Hy3.

The model is designed to move automatic speech recognition beyond isolated acoustic decoding.

Traditional ASR systems mainly answer this question:

Which sequence of words most closely matches this audio?

Hy ASR 3.0 adds a second question:

Which transcription makes the most sense in this context?

That difference matters when the audio contains:

• Homophones.
• Regional accents.
• Dialects.
• Mixed Chinese and English.
• Product names and personal names.
• Background noise.
• Whispered or low-volume speech.
• Long or semantically dependent utterances.

Tencent says the model combines a high-accuracy acoustic system with the contextual modeling and semantic reasoning of Hy3. Its goal is not to rewrite what the user said creatively, but to use linguistic context to choose a more plausible transcription when the audio alone is ambiguous.

The company reports word error rates of 3.34% for Mandarin Chinese, 2.62% for English, and 3.12% for Cantonese on aggregated public evaluation sets.

Hy ASR 3.0 Preview is now documented as a Tencent Cloud real-time WebSocket engine and has also been integrated into Tencent’s Yuanbao assistant. WorkBuddy and other Tencent products are being connected gradually.

The public Preview is useful, but it is not yet the finished product described in the broader launch announcement. Its present API limits are important for anyone planning a production integration.

Reported Public-Benchmark Results

Tencent evaluated Hy ASR 3.0 Preview on several public speech datasets and compared it with other ASR systems.

The company reports the following aggregated word error rates:

Lower WER is better.

The chart’s notes indicate that the aggregate evaluation draws from datasets such as:

• WenetSpeechMeeting.
• WenetSpeechNet.
• FLEURS Chinese.
• LibriSpeech clean.
• LibriSpeech other.
• FLEURS English.
• MLS English.
• FLEURS Cantonese.
• Common Voice Cantonese.

An aggregate number is useful for a broad comparison, but it can conceal important variation.

A model may perform differently on:

• Read speech versus spontaneous conversation.
• Near-field versus far-field microphones.
• Clean studio audio versus street noise.
• Short commands versus continuous discussion.
• Native speakers versus accented speech.
• Formal Mandarin versus code-switching.

A production team should therefore test its own audio rather than selecting an ASR model from one headline WER number.

What Word Error Rate Measures

Word Error Rate is commonly defined as:

WER = (Substitutions + Deletions + Insertions) / Reference words

The three error types are:

• Substitution: The model outputs the wrong word.
• Deletion: A spoken word is missing from the transcript.
• Insertion: The transcript contains a word that was not spoken.

A lower WER usually indicates a more accurate transcript.

However, WER does not capture every practical failure.

Consider two one-word errors:

“Ship the order tomorrow”
→ “Ship the order today”

and:

“The color is navy blue”
→ “The colour is navy blue”

Both may produce a similar error count, but the first can change a business action while the second may have no operational effect.

For customer service, healthcare, finance, manufacturing, and command interfaces, teams should evaluate semantic impact as well as WER.

Tencent’s Internal Scenario Evaluation

Tencent also published results from internal evaluation sets covering four practical categories:

1. General speech recognition.
2. Dialect recognition.
3. Context understanding.
4. Complex acoustic conditions such as high noise and whispering.

The reported results are:

Tencent reports that Hy ASR 3.0 Preview achieved the lowest WER among the compared systems in all four internal categories.

These results are useful as company-reported product evidence, but the internal test sets are not a neutral public benchmark.

Before adoption, an organization should ask:

• Which accents and dialect regions were included?
• How noisy was the audio?
• How was punctuation normalized?
• Were numbers and English terms standardized before scoring?
• Were competing systems configured with hotwords or context?
• How many utterances were used?
• Did the test include the organization’s actual microphones and channels?

Four User-Facing Improvements

Tencent groups the practical improvements into four areas.

1. More Accurate General Recognition

The model is intended to improve recognition across:

• Standard Mandarin.
• English.
• Cantonese.
• Regional dialects.
• Mixed Chinese and English.
• Different speakers and accents.

Tencent also says the model reduces accumulated errors in longer utterances.

That claim describes the model’s underlying capability. The current Tencent Cloud Preview still limits one public API input to 60 seconds, so applications working with meetings or recordings must currently segment the audio and manage cross-segment context themselves.

Poor segmentation can reintroduce the same problem the model is designed to reduce.

For example, cutting audio at an arbitrary 60-second boundary can split:

• A person’s full name.
• A product code.
• A sentence with delayed meaning.
• A Chinese–English mixed phrase.
• A correction made by the speaker.

Chunking logic should therefore preserve sentence and voice-activity boundaries where possible.

2. Better Context and Intent Recognition

Speech contains many ambiguous sounds.

Mandarin has numerous homophones. English has words and names that sound similar. Dialect pronunciation can make several candidate transcripts acoustically plausible.

A conventional decoder may choose the locally most likely word.

A context-aware system can evaluate the full utterance.

For example, a user may say a phrase that could be transcribed as two different homophones. Nearby words about finance, gaming, medicine, or travel can indicate which meaning is more likely.

The intended processing can be simplified as:

Audio features
→ candidate words
→ sentence context
→ semantic consistency check
→ final transcript

This does not mean the model literally understands speech in the same way a person does.

It means the language component uses a broader context window and semantic probabilities to improve the transcription decision.

A contextual ASR system also creates a new risk: semantic overcorrection.

The model may occasionally replace an unusual but correctly spoken phrase with a more common phrase that appears to make more sense.

Production evaluations should therefore include:

• Rare names.
• Deliberately unusual statements.
• New product terms.
• Domain abbreviations.
• Contradictory or surprising sentences.

The objective is to use context without inventing speech that was never present.

3. Easier Adaptation to Professional Vocabulary

The launch material highlights hotword enhancement for:

• Brand names.
• Product names.
• Personal names.
• Industry terms.
• Abbreviations.
• Internal vocabulary.

Hotwords can be valuable when a general model has not seen a rare term frequently enough.

Examples include:

A proprietary medicine name
A factory machine model
A customer account code
A newly launched brand
A technical abbreviation

Tencent Cloud’s general ASR product already has hotword-list APIs and a hotword_id parameter.

However, the current Hy ASR 3.0 Preview documentation explicitly says hotword enhancement is not yet open for this Preview engine.

The public product message and the currently accessible API should therefore be separated:

Businesses should not design a launch plan that depends on external context injection or hotword lists until Tencent Cloud confirms support in the live API documentation.

4. More Stable Recognition in Difficult Acoustic Conditions

Tencent says the model has been optimized for:

• High background noise.
• Whispering.
• Quiet speech.
• Different recording environments.
• Multiple accents.
• Long-tail acoustic conditions.

Noise robustness matters because real speech rarely arrives as a clean laboratory recording.

A customer-service microphone may capture keyboard sound and nearby agents.

A mobile assistant may hear traffic, wind, music, or another person speaking.

A meeting application may receive compressed audio from a distant laptop microphone.

A model can improve robustness, but it cannot recover information that was never captured.

Teams should still use:

• Appropriate microphones.
• Echo cancellation.
• Gain control.
• Voice-activity detection where supported.
• Sensible audio compression.
• Channel monitoring.
• Retry or clarification flows.

ASR quality is a system property, not only a model property.

Architecture: Hy3 Plus a Speech Encoder

Tencent says Hy ASR 3.0 Preview combines three broad components:

1. An MoE language-model foundation based on Hy3.
2. A self-developed unsupervised speech encoder.
3. Joint pretraining and multi-stage post-training.

Hy3 as the Language Foundation

Hy3 provides the language-understanding component.

Its role includes:

• Modeling sentence context.
• Resolving ambiguous candidates.
• Understanding mixed-language structure.
• Using semantic relationships.
• Improving output coherence.

Tencent describes the architecture as a Mixture of Experts design that balances capability and efficiency.

The public ASR documentation does not disclose the full parameter count, active-parameter count, latency profile, or complete inference architecture of Hy ASR 3.0 Preview.

The Unsupervised Speech Encoder

The speech encoder transforms raw audio into acoustic representations that the language model can process.

Tencent says the encoder was trained on tens of millions of hours of unlabeled speech.

Unsupervised or self-supervised audio training is useful because manually transcribing that quantity of speech would be prohibitively expensive.

The encoder can learn recurring structure from raw audio, such as:

• Phonetic patterns.
• Speaker variation.
• Accent variation.
• Background conditions.
• Timing and prosody.

The quality of this representation affects everything that follows.

If two sounds are confused at the encoder stage, the language model must rely more heavily on context to recover the correct word.

Joint Speech-and-Language Pretraining

Tencent says the speech encoder and language model were jointly trained using large, multi-source speech datasets covering:

• Dialects.
• Accents.
• Acoustic environments.
• Different speaker groups.
• Contextual language patterns.

Joint training is intended to reduce the gap between acoustic recognition and language understanding.

Instead of treating speech recognition as:

Audio model finishes
→ language model cleans the transcript afterward

Hy ASR 3.0 is presented as a more integrated process:

Speech representation and language modeling
→ trained to cooperate
→ one contextual transcription result

The exact internal boundary between encoder, decoder, language model, and reinforcement-learning stages has not been fully published.

SFT and Multi-Stage Reinforcement Learning

Tencent describes a supervised fine-tuning recipe that covers:

• General transcription.
• Any-context tasks.
• Professional terms.
• Different acoustic environments.
• Diverse speaker groups.
• Ten broad dialect regions.
• More than 20 smaller dialect areas.

The company also reports using multi-stage reinforcement learning for:

• General transcription accuracy.
• Contextual behavior.
• Complex long-tail cases.
• Reducing substitutions and deletions.

No public technical paper currently provides the complete reward design, data mixture, training compute, or ablation results.

The architecture claims should therefore be treated as a product-level technical description rather than a reproducible research specification.

Languages and Dialects in the Current Tencent Cloud Engine

Tencent Cloud documents the current Hy-ASR-3.0-preview engine as supporting:

• Mandarin Chinese.
• English.
• 20 Chinese dialects or regional varieties.

The documented dialect list is:

Dialect labels in commercial ASR documentation are broad product categories.

Real speech varies within each category by city, age, social background, code-switching, vocabulary, and speaker.

A claim of dialect support should not be interpreted as identical accuracy for every speaker in the region.

Current Preview Availability

Tencent Cloud added the Hy ASR 3.0 Preview engine to the real-time WebSocket product on August 4, 2026.

The public service is currently described as an internal-test or Preview release.

The general WebSocket API reference contains parameters used by other engines, including VAD and hotword IDs.

The engine-specific Preview note takes precedence for Hy ASR 3.0.

A parameter appearing in the shared API schema does not guarantee that the Preview engine currently implements it.

Basic Tencent Cloud Integration Flow

The official setup has three main stages.

Step 1: Activate Tencent Cloud Speech Recognition

Open the Tencent Cloud speech-recognition service and complete the account activation process.

The official quick-start documentation is available at:

https://cloud.tencent.com/document/product/1093/54362

Review billing before enabling postpaid usage.

Tencent Cloud states that postpaid billing is disabled by default for newer accounts and must be turned on manually.

Step 2: Create API Credentials

Create or obtain:

• AppID
• SecretID
• SecretKey

These credentials are used to sign the WebSocket connection request.

Store them in a secrets manager or protected environment variables.

Do not place long-lived credentials in:

• Front-end JavaScript.
• Mobile application source code.
• Public repositories.
• Shared documents.
• Client-visible URLs.

For browser or mobile products, create the signed connection information on a trusted backend.

Step 3: Connect to the Real-Time WebSocket Endpoint

Tencent Cloud documents the endpoint format as:

wss://asr.cloud.tencent.com/asr/v2/?{request_parameters}

Replace `` with the Tencent Cloud AppID.

The request includes signed parameters such as:

• secretid
• timestamp
• expired
• nonce
• voice_id
• engine_model_type

For Hy ASR 3.0 Preview, set:

engine_model_type=Hy-ASR-3.0-preview

Each WebSocket connection needs a unique voice_id.

If the connection ends or fails, the old voice_id becomes invalid and a new one must be generated.

Step 4: Prepare Compatible Audio

The current Preview requires:

Sample rate: 16 kHz
Channels: mono
Format: PCM
Maximum input duration: 60 seconds

Test the complete audio path, not only the original file.

Microphone SDKs, browser media APIs, telephony systems, and conferencing platforms may resample or compress audio before it reaches the server.

Step 5: Stream Audio and Read Incremental Results

The service supports real-time transcription, returning text while audio is being transmitted.

Applications should handle:

• Partial results.
• Final results.
• Connection errors.
• Authentication failures.
• Timeouts.
• Reconnection.
• Audio-duration limits.
• Duplicate or revised text.

Do not assume that every partial token is final.

A real-time ASR interface may revise earlier words when more context becomes available.

The user interface should distinguish temporary text from committed text.

Step 6: Test Before Production

Build a representative evaluation set containing:

• Real customer audio.
• Common accents.
• Dialects.
• Mixed Chinese and English.
• Names and product terms.
• Noise.
• Whispering.
• Poor microphones.
• Short commands.
• Complete sentences.

Measure both recognition quality and system behavior.

Pricing for the Preview Engine

Tencent Cloud classifies Hy ASR 3.0 Preview as a Large Model 2.0 real-time speech-recognition engine.

The current billing page lists:

Tencent’s billing table currently lists no free audio quota for Large Model 2.0 recognition.

The 20 included concurrent routes are a concurrency allowance, not free recognition time.

Pricing can change. Check the live billing page before publishing a commercial quote or estimating a long-term budget.

Cost Example

At the currently listed RMB 1.00 per hour postpaid rate:

100 hours of successful recognition
× RMB 1.00/hour
= RMB 100

A production budget should also include:

• Audio preprocessing.
• Network transfer.
• Backend servers.
• Storage.
• Monitoring.
• Human correction.
• Retry traffic.
• Downstream language-model processing.

The ASR fee is only one part of the complete transcription system.

Yuanbao and Product Integrations

Tencent says Yuanbao participated in the model’s development and was the first Tencent product to integrate it.

Users can access the experience through voice input in Yuanbao, where the model is intended to improve:

• Dialect recognition.
• Contextual correction.
• Recognition under difficult acoustic conditions.

Tencent says WorkBuddy and other products are being connected gradually.

A consumer-product integration may expose capabilities differently from the public Tencent Cloud Preview API.

For example, Yuanbao may use internal services, product-specific context, or deployment configurations that are not yet exposed to external developers.

An experience in Yuanbao should not be assumed to map one-to-one to public API parameters.

Suitable Use Cases

Hy ASR 3.0 Preview is positioned for short, low-latency voice interactions.

Intelligent Customer Service

Potential uses include:

• Live agent transcripts.
• Voice-bot command recognition.
• Call-note generation.
• Intent extraction.
• Quality-review support.

The current lack of speaker separation in the Preview may limit multi-speaker call transcription unless another component separates channels or speakers.

Real-Time Captions

The engine can support short real-time captions for:

• Live video.
• Audio rooms.
• Internal meetings.
• Voice messaging.
• Accessibility interfaces.

Applications should test partial-result stability and punctuation behavior.

Voice Search

Context-aware recognition can improve searches involving:

• Long natural-language questions.
• Product names.
• Mixed Chinese and English.
• Regional pronunciation.

Until hotword injection opens for the Preview, rare catalog terms may still need post-processing or a separate correction layer.

Voice Commands and Assistants

The engine can convert spoken instructions into text for:

• AI assistants.
• Enterprise agents.
• Smart-device controls.
• Workflow commands.

A command system should never execute high-impact actions solely because one transcript appears confident.

For destructive or financial operations, confirm the parsed intent and require explicit user approval.

Content Understanding

Short audio segments can be transcribed before being sent into:

• Summarization.
• Classification.
• Search indexing.
• Moderation.
• Knowledge extraction.

The quality of every downstream AI step depends on the transcript.

Store the original audio or confidence evidence where policy permits so uncertain outputs can be reviewed.

Current Limitations

Preview Status

The product is still labeled Preview or internal test.

Interfaces, pricing, limits, and supported functions may change.

Sixty-Second Input Limit

The current public API is not a direct replacement for batch transcription of long recordings.

Long audio requires segmentation and possibly an application-level context layer.

PCM-Only Input

The Preview currently requires 16 kHz mono PCM.

Many production sources use MP3, AAC, Opus, or telephony codecs and will need conversion.

No Speaker Separation

The current engine-specific documentation says speaker separation is not supported.

Multi-speaker transcripts may require separate audio channels or another diarization service.

Context and Hotwords Are Not Yet Public

The announced capabilities are not yet exposed as usable Preview API features according to the official documentation.

Company-Reported Benchmarks

The benchmark charts come from Tencent.

Independent evaluation on matched settings would strengthen the comparison.

No Complete Technical Paper

Tencent has described the architecture and training process at a high level but has not published full reproducible details for Hy ASR 3.0 Preview.

Context Can Introduce Overcorrection

A language-aware decoder may prefer a common sentence over an unusual but correctly spoken one.

Rare and surprising phrases must be included in testing.

A Practical Evaluation Checklist

1. Build a Domain Test Set

Include at least several hundred representative utterances rather than a handful of clean demos.

2. Preserve Raw References

Create human-verified reference transcripts using a clear normalization policy.

Decide how to treat:

• Punctuation.
• Numbers.
• English capitalization.
• Filler words.
• Repetitions.
• Traditional and simplified Chinese.

3. Measure Several Metrics

Use:

• WER or character error rate.
• Name accuracy.
• Number accuracy.
• Semantic error rate.
• Latency.
• Partial-result revision rate.
• Failure rate.

4. Test Dialects Separately

Do not combine all dialect speakers into one average.

Report results by region and recording condition.

5. Test Contextual Ambiguity

Create pairs where acoustic content is similar but sentence context changes the correct transcription.

6. Test Rare Terms

Evaluate product names and technical terms now, then repeat the test after Tencent opens hotword support.

7. Test Noise and Whispering

Use real environmental recordings rather than only digitally added noise.

8. Test Segment Boundaries

Confirm that the 60-second implementation does not cut important phrases or create duplicated text.

9. Measure End-to-End Latency

Include:

Capture
+ upload
+ recognition
+ finalization
+ downstream processing

10. Review Privacy and Compliance

Voice recordings may contain personal or sensitive information.

Define retention, access, encryption, consent, and deletion policies before deployment.

Hy ASR 3.0 Preview Versus a Traditional ASR Pipeline

The newer approach does not remove the need for traditional ASR engineering.

It adds a stronger language layer to the same end-to-end system.

常见问题

What is Hy ASR 3.0 Preview?

Hy ASR 3.0 Preview is Tencent Hunyuan’s real-time speech-recognition model based on the Hy3 language foundation and a self-developed speech encoder. It is designed to combine acoustic recognition with contextual language understanding.

What languages and dialects does Hy ASR 3.0 support?

Tencent Cloud documents support for Mandarin Chinese, English, and 20 Chinese dialects or regional varieties, including Cantonese, Northeastern Mandarin, Henan, Shaanxi, Chengdu, Chongqing, Wuhan, and others. Accuracy can vary across speakers and regions.

What are the reported WER scores?

Tencent reports aggregated public-benchmark WER values of 3.34% for Mandarin, 2.62% for English, and 3.12% for Cantonese. These are company-reported results across several datasets rather than one independently reproduced universal test.

Can Hy ASR 3.0 transcribe long recordings?

The current public Preview accepts no more than 60 seconds of audio per input. Longer recordings require segmentation, and the application must preserve useful context across segments.

Does the current API support hotwords and custom context?

Not yet, according to Tencent Cloud’s Preview documentation dated August 4, 2026. The launch material describes these capabilities, but the official API page says context input and hotword enhancement will open later.

Which audio formats are supported?

The current Hy ASR 3.0 Preview documentation specifies 16 kHz mono PCM input. Applications using MP3, AAC, Opus, or another format must convert the audio before sending it to this engine.

How do developers call Hy ASR 3.0?

Developers use Tencent Cloud’s real-time speech-recognition WebSocket API and set engine_model_type to Hy-ASR-3.0-preview. The connection must be signed with Tencent Cloud credentials.

Is Hy ASR 3.0 free?

Tencent Cloud’s current billing page lists no free audio quota for Large Model 2.0 recognition. It lists prepaid packages starting at RMB 1 per hour for 60 hours and postpaid usage at RMB 1 per hour, while 20 concurrent routes are included.

相关工具

• Tencent Cloud Hunyuan ASR Preview: Official feature, limitation, dialect, and quick-access documentation for Hy ASR 3.0 Preview.
• Tencent Cloud Real-Time ASR WebSocket API: Official endpoint, authentication parameters, engine selection, and response documentation.
• Tencent Cloud API Explorer: Tencent’s official interface for inspecting APIs and generating SDK request examples.
• Tencent Cloud Python SDK: Official Python SDK for Tencent Cloud APIs and credential management.
• FFmpeg: An open-source audio and video toolkit useful for converting input audio to compatible sample rates and channel layouts.
• Websocat: A command-line WebSocket client useful for testing streaming endpoints during development.

Related Links

• Hunyuan ASR Preview Documentation: Current official status, supported dialects, limitations, and basic setup.
• Real-Time Speech Recognition API: WebSocket endpoint format and request-parameter reference.
• One-Minute Server API Quick Start: Tencent Cloud’s official service-activation and credential guide.
• Tencent Cloud ASR Billing: Current prepaid, postpaid, concurrency, and free-quota information.
• Tencent Cloud ASR Product Updates: Official release log showing the Hy ASR 3.0 Preview engine added on August 4, 2026.
• Tencent Cloud Hotword API: Official general ASR hotword-list API; Hy ASR 3.0 Preview support is still listed as coming soon.
• Tencent Hunyuan: Official Hunyuan model and product portal.

Summary

Tencent Hunyuan’s Hy ASR 3.0 Preview combines a speech encoder with Hy3’s language-model capabilities to improve Mandarin, English, dialect, mixed-language, noisy, and context-dependent transcription.

Tencent reports WER values of 3.34% for Mandarin, 2.62% for English, and 3.12% for Cantonese on aggregated public datasets, along with leading results on its internal scenario tests. These figures are promising but should be validated on each organization’s own audio.

The current Tencent Cloud Preview is narrower than the complete launch vision. It supports real-time WebSocket recognition, 16 kHz mono PCM, and audio inputs of up to 60 seconds. External context injection, hotword enhancement, speaker separation, and several other functions are not yet available for this engine.

The key change is not that ASR has stopped listening to sound—it is that the language model now helps decide what the sound most plausibly means.