> ## Documentation Index > Fetch the complete documentation index at: https://docs.voxnexus.ai/llms.txt > Use this file to discover all available pages before exploring further. # Translate Realtime > WebSocket Real-time Speech Translation Channel Through this channel, clients can: 1. Send initialization messages to configure translation parameters 2. Send audio data streams for real-time speech translation 3. Send command messages to control the translation flow 4. Receive source transcript results and translated results (partial and final) **Connection**: - Authentication via header: `X-Api-Key: YOUR_API_KEY` - Authentication via query: `?token=YOUR_API_KEY` **Message Format**: All messages are in JSON format, containing a `type` field to identify the message type ## AsyncAPI ````yaml api-reference/asyncapi.yaml translateRealtime id: translateRealtime title: Translate Realtime description: > WebSocket Real-time Speech Translation Channel Through this channel, clients can: 1. Send initialization messages to configure translation parameters 2. Send audio data streams for real-time speech translation 3. Send command messages to control the translation flow 4. Receive source transcript results and translated results (partial and final) **Connection**: - Authentication via header: `X-Api-Key: YOUR_API_KEY` - Authentication via query: `?token=YOUR_API_KEY` **Message Format**: All messages are in JSON format, containing a `type` field to identify the message type servers: - id: production protocol: wss host: api.voxnexus.ai bindings: [] variables: [] address: /v1/translate/realtime parameters: [] bindings: [] operations: - &ref_1 id: receiveTranslateClientMessages title: Receive translate client messages type: receive messages: - &ref_3 id: TranslateInitMessage payload: - name: Translate Initialization Message type: object properties: - name: type type: string description: Message type enumValues: - init required: true - name: model_id type: string description: Model ID (required) required: true - name: format type: string description: >- Audio format (required). Real-time translation accepts raw PCM only. enumValues: - pcm required: true - name: sample_rate type: integer description: >- Sample rate (required). Real-time translation accepts 16000 only. enumValues: - 16000 required: true - name: language_hints type: array description: > Optional list of expected source language codes (ISO 639-1 / BCP-47), used as hints to improve automatic source-language detection. When omitted, detection considers all supported languages. These are hints, not a restriction: a language outside the list is still detected and transcribed. They only help the detector when the spoken language is ambiguous (e.g. similar-sounding languages), so listing fewer, more likely languages works better than listing many. In `mode=two_way` both languages of the pair are spoken by definition, so `target_language_a` and `target_language_b` are APPENDED to whatever is sent here (deduplicated, hints first). Sending nothing therefore yields exactly the pair. The effective value is echoed back in the `ready` message. required: false properties: - name: item type: string required: false - name: enable_speaker_diarization type: boolean description: > Whether to enable speaker diarization — identifying and separating different speakers in the audio. Optional, defaults to false. required: false - name: mode type: string description: > Translation mode (optional, defaults to `one_way`). - `one_way`: the source language is auto-detected and translated into `target_language`, which is REQUIRED in this mode. Any detected source language is translated. - `two_way`: translate between `target_language_a` and `target_language_b` in either direction. Both `target_language_a` and `target_language_b` are REQUIRED in this mode. The two modes differ in how they treat the source language, which matters when the speaker uses an unexpected one. `one_way` translates whatever it hears. `two_way` translates only speech detected as `target_language_a` or `target_language_b` — speech in any other language is still transcribed, but produces NO `translation` message. Speech cannot be constrained to the pair, so treat this as a normal condition rather than an error: compare `transcript.language` against the pair to tell an untranslated segment from a missing one. enumValues: - one_way - two_way required: false - name: target_language type: string description: >- Target language code. Required when mode=one_way; ignored otherwise. required: false - name: target_language_a type: string description: >- First language code. Required when mode=two_way; ignored otherwise. required: false - name: target_language_b type: string description: >- Second language code. Required when mode=two_way; ignored otherwise. required: false headers: [] jsonPayloadSchema: type: object description: >- Real-time translation WebSocket initialization message (sent by client) properties: type: type: string enum: - init description: Message type x-parser-schema-id: model_id: type: string description: Model ID (required) example: vn-translate-rt-basic x-parser-schema-id: format: type: string description: >- Audio format (required). Real-time translation accepts raw PCM only. enum: - pcm example: pcm x-parser-schema-id: sample_rate: type: integer description: >- Sample rate (required). Real-time translation accepts 16000 only. enum: - 16000 default: 16000 example: 16000 x-parser-schema-id: language_hints: type: array items: type: string x-parser-schema-id: description: > Optional list of expected source language codes (ISO 639-1 / BCP-47), used as hints to improve automatic source-language detection. When omitted, detection considers all supported languages. These are hints, not a restriction: a language outside the list is still detected and transcribed. They only help the detector when the spoken language is ambiguous (e.g. similar-sounding languages), so listing fewer, more likely languages works better than listing many. In `mode=two_way` both languages of the pair are spoken by definition, so `target_language_a` and `target_language_b` are APPENDED to whatever is sent here (deduplicated, hints first). Sending nothing therefore yields exactly the pair. The effective value is echoed back in the `ready` message. example: - en - fr x-parser-schema-id: enable_speaker_diarization: type: boolean description: > Whether to enable speaker diarization — identifying and separating different speakers in the audio. Optional, defaults to false. default: false example: false x-parser-schema-id: mode: type: string description: > Translation mode (optional, defaults to `one_way`). - `one_way`: the source language is auto-detected and translated into `target_language`, which is REQUIRED in this mode. Any detected source language is translated. - `two_way`: translate between `target_language_a` and `target_language_b` in either direction. Both `target_language_a` and `target_language_b` are REQUIRED in this mode. The two modes differ in how they treat the source language, which matters when the speaker uses an unexpected one. `one_way` translates whatever it hears. `two_way` translates only speech detected as `target_language_a` or `target_language_b` — speech in any other language is still transcribed, but produces NO `translation` message. Speech cannot be constrained to the pair, so treat this as a normal condition rather than an error: compare `transcript.language` against the pair to tell an untranslated segment from a missing one. enum: - one_way - two_way default: one_way x-parser-schema-id: target_language: type: string description: >- Target language code. Required when mode=one_way; ignored otherwise. example: en x-parser-schema-id: target_language_a: type: string description: >- First language code. Required when mode=two_way; ignored otherwise. example: en x-parser-schema-id: target_language_b: type: string description: >- Second language code. Required when mode=two_way; ignored otherwise. example: fr x-parser-schema-id: required: - type - model_id - format - sample_rate allOf: - if: properties: mode: const: two_way required: - mode x-parser-schema-id: then: required: - target_language_a - target_language_b x-parser-schema-id: else: required: - target_language x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: title: Translate Initialization Message example: |- { "type": "", "model_id": "", "format": "", "sample_rate": 123, "language_hints": { "item": "" }, "enable_speaker_diarization": true, "mode": "", "target_language": "", "target_language_a": "", "target_language_b": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: TranslateInitMessage - id: x-parser-message-name value: TranslateInitMessage - &ref_4 id: TranslateAudioMessage payload: - name: Translate Audio Message type: object properties: - name: type type: string description: Message type enumValues: - audio required: true - name: data type: string description: Base64-encoded PCM audio data required: true headers: [] jsonPayloadSchema: type: object description: Real-time translation WebSocket audio data message (sent by client) properties: type: type: string enum: - audio description: Message type x-parser-schema-id: data: type: string format: byte description: Base64-encoded PCM audio data x-parser-schema-id: required: - type - data x-parser-schema-id: title: Translate Audio Message example: |- { "type": "", "data": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: TranslateAudioMessage - id: x-parser-message-name value: TranslateAudioMessage - &ref_5 id: TranslateCommandMessage payload: - name: Translate Command Message type: object properties: - name: type type: string description: Message type enumValues: - command required: true - name: command type: string description: >- Command type: 'flush' tells the server that no more audio will be sent. The server will respond with a 'flush_done' message after translation is complete. Subsequent audio will start a new translation session. enumValues: - flush required: true headers: [] jsonPayloadSchema: type: object description: Real-time translation WebSocket command message (sent by client) properties: type: type: string enum: - command description: Message type x-parser-schema-id: command: type: string enum: - flush description: >- Command type: 'flush' tells the server that no more audio will be sent. The server will respond with a 'flush_done' message after translation is complete. Subsequent audio will start a new translation session. x-parser-schema-id: required: - type - command x-parser-schema-id: title: Translate Command Message example: |- { "type": "", "command": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: TranslateCommandMessage - id: x-parser-message-name value: TranslateCommandMessage bindings: [] extensions: &ref_0 - id: x-parser-unique-object-id value: translateRealtime - &ref_2 id: sendTranslateServerMessages title: Send translate server messages type: send messages: - &ref_6 id: TranslateReadyMessage payload: - name: Translate Ready Message type: object properties: - name: type type: string description: Message type enumValues: - ready required: true - name: request_id type: string description: Request ID required: true - name: format type: string description: Audio format required: true - name: sample_rate type: integer description: Sample rate required: true - name: mode type: string description: Translation mode enumValues: - one_way - two_way required: false - name: target_language type: string description: Target language code (mode=one_way) required: false - name: target_language_a type: string description: First language code (mode=two_way) required: false - name: target_language_b type: string description: Second language code (mode=two_way) required: false - name: language_hints type: array description: > Source language hints in effect for this session. In `mode=two_way` the language pair is appended to what init sent, so this may differ from it; compare the two to confirm how the request was interpreted. required: false properties: - name: item type: string required: false - name: enable_speaker_diarization type: boolean description: >- Whether speaker diarization is enabled for this session (echoed from init) required: false headers: [] jsonPayloadSchema: type: object description: Real-time translation WebSocket ready message (sent by server) properties: type: type: string enum: - ready description: Message type x-parser-schema-id: request_id: type: string description: Request ID example: req_1234567890 x-parser-schema-id: format: type: string description: Audio format example: pcm x-parser-schema-id: sample_rate: type: integer description: Sample rate example: 16000 x-parser-schema-id: mode: type: string description: Translation mode enum: - one_way - two_way example: one_way x-parser-schema-id: target_language: type: string description: Target language code (mode=one_way) example: en x-parser-schema-id: target_language_a: type: string description: First language code (mode=two_way) example: en x-parser-schema-id: target_language_b: type: string description: Second language code (mode=two_way) example: zh x-parser-schema-id: language_hints: type: array items: type: string x-parser-schema-id: description: > Source language hints in effect for this session. In `mode=two_way` the language pair is appended to what init sent, so this may differ from it; compare the two to confirm how the request was interpreted. example: - en - zh x-parser-schema-id: enable_speaker_diarization: type: boolean description: >- Whether speaker diarization is enabled for this session (echoed from init) example: false x-parser-schema-id: required: - type - request_id - format - sample_rate x-parser-schema-id: title: Translate Ready Message example: |- { "type": "", "request_id": "", "format": "", "sample_rate": 123, "mode": "", "target_language": "", "target_language_a": "", "target_language_b": "", "language_hints": { "item": "" }, "enable_speaker_diarization": true } bindings: [] extensions: - id: x-parser-unique-object-id value: TranslateReadyMessage - id: x-parser-message-name value: TranslateReadyMessage - &ref_7 id: TranslateTranscriptMessage payload: - name: Translate Transcript Message type: object properties: - name: type type: string description: Message type enumValues: - transcript required: true - name: request_id type: string description: Request ID required: true - name: text type: string description: >- Recognized source text segment. Append when is_final=true; replace the pending text when is_final=false. required: true - name: is_final type: boolean description: >- Whether the text is confirmed (false=partial, may still be revised; true=confirmed, will not change) required: true - name: language type: string description: Detected language code required: false - name: speaker_id type: string description: > Speaker label for this text increment. Present only when speaker diarization is enabled; absent otherwise. An opaque string label (e.g. `1`, `2`) that is stable within a session but not correlated across sessions. required: false - name: confidence type: number description: Confidence score 0.0-1.0 (only valid when is_final=true) required: false - name: offset type: integer description: Text offset in audio stream (milliseconds) required: false - name: duration type: integer description: >- Duration of corresponding audio (milliseconds, only valid when is_final=true) required: false - name: words type: array description: Word-level information array (only valid when is_final=true) required: false properties: - name: word type: string description: Word required: true - name: offset type: integer description: Word offset in audio stream (milliseconds) required: false - name: duration type: integer description: Word duration (milliseconds) required: false - name: confidence type: number description: Confidence score 0.0-1.0 required: false headers: [] jsonPayloadSchema: type: object description: > Real-time translation source transcript message (sent by server). Carries a SEGMENT of recognized source-language text — never a single word — and `is_final` decides how to apply it: - `is_final=true` — a confirmed segment. APPEND it to the confirmed text; it will never be revised. - `is_final=false` — the current pending (unconfirmed) text in full. REPLACE the previous pending text with it; do not append. It is superseded by the next message, and the words in it disappear from pending once they arrive in a confirmed segment. So the full text at any moment is: (all `is_final=true` texts appended in arrival order) followed by (the latest `is_final=false` text). When speaker diarization is enabled (`enable_speaker_diarization=true` in init), each segment is attributed to a speaker via `speaker_id`; a segment never spans two speakers, and consecutive segments with the same `speaker_id` belong to the same speaker turn. properties: type: type: string enum: - transcript description: Message type x-parser-schema-id: request_id: type: string description: Request ID example: req_1234567890 x-parser-schema-id: text: type: string description: >- Recognized source text segment. Append when is_final=true; replace the pending text when is_final=false. example: Hello, this is a test message. x-parser-schema-id: is_final: type: boolean description: >- Whether the text is confirmed (false=partial, may still be revised; true=confirmed, will not change) example: true x-parser-schema-id: language: type: string description: Detected language code example: en x-parser-schema-id: speaker_id: type: string description: > Speaker label for this text increment. Present only when speaker diarization is enabled; absent otherwise. An opaque string label (e.g. `1`, `2`) that is stable within a session but not correlated across sessions. example: '1' x-parser-schema-id: confidence: type: number format: float description: Confidence score 0.0-1.0 (only valid when is_final=true) minimum: 0 maximum: 1 example: 0.95 x-parser-schema-id: offset: type: integer description: Text offset in audio stream (milliseconds) example: 0 x-parser-schema-id: duration: type: integer description: >- Duration of corresponding audio (milliseconds, only valid when is_final=true) example: 2500 x-parser-schema-id: words: type: array description: Word-level information array (only valid when is_final=true) items: title: STT Word Information type: object properties: word: type: string description: Word example: hello x-parser-schema-id: offset: type: integer description: Word offset in audio stream (milliseconds) example: 0 x-parser-schema-id: duration: type: integer description: Word duration (milliseconds) example: 500 x-parser-schema-id: confidence: type: number format: float description: Confidence score 0.0-1.0 minimum: 0 maximum: 1 example: 0.98 x-parser-schema-id: required: - word x-parser-schema-id: STTWordInfo x-parser-schema-id: required: - type - request_id - text - is_final x-parser-schema-id: title: Translate Transcript Message example: |- { "type": "", "request_id": "", "text": "", "is_final": true, "language": "", "speaker_id": "", "confidence": 123, "offset": 123, "duration": 123, "words": { "word": "", "offset": 123, "duration": 123, "confidence": 123 } } bindings: [] extensions: - id: x-parser-unique-object-id value: TranslateTranscriptMessage - id: x-parser-message-name value: TranslateTranscriptMessage - &ref_8 id: TranslateTranslationMessage payload: - name: Translate Translation Message type: object properties: - name: type type: string description: Message type enumValues: - translation required: true - name: request_id type: string description: Request ID required: true - name: text type: string description: >- Translated text segment. Append when is_final=true; replace the pending text when is_final=false. required: true - name: is_final type: boolean description: >- Whether the text is confirmed (false=partial, may still be revised; true=confirmed, will not change) required: true - name: source_language type: string description: Source language code required: false - name: target_language type: string description: Target language code required: false - name: speaker_id type: string description: > Speaker label for this translated increment. Present only when speaker diarization is enabled; absent otherwise. Matches the `speaker_id` of the corresponding source speaker turn — an opaque string label (e.g. `1`, `2`) stable within a session. required: false headers: [] jsonPayloadSchema: type: object description: > Real-time translation result message (sent by server). Carries a SEGMENT of translated text — never a single word — and `is_final` decides how to apply it, exactly as for `transcript`: - `is_final=true` — a confirmed segment. APPEND it to the confirmed translation. - `is_final=false` — the current pending translation in full. REPLACE the previous pending translation with it; do not append. Translation segments are NOT positionally correlated with individual `transcript` messages: maintain the two texts independently. properties: type: type: string enum: - translation description: Message type x-parser-schema-id: request_id: type: string description: Request ID example: req_1234567890 x-parser-schema-id: text: type: string description: >- Translated text segment. Append when is_final=true; replace the pending text when is_final=false. example: 你好，这是一条测试消息。 x-parser-schema-id: is_final: type: boolean description: >- Whether the text is confirmed (false=partial, may still be revised; true=confirmed, will not change) example: true x-parser-schema-id: source_language: type: string description: Source language code example: en x-parser-schema-id: target_language: type: string description: Target language code example: zh x-parser-schema-id: speaker_id: type: string description: > Speaker label for this translated increment. Present only when speaker diarization is enabled; absent otherwise. Matches the `speaker_id` of the corresponding source speaker turn — an opaque string label (e.g. `1`, `2`) stable within a session. example: '1' x-parser-schema-id: required: - type - request_id - text - is_final x-parser-schema-id: title: Translate Translation Message example: |- { "type": "", "request_id": "", "text": "", "is_final": true, "source_language": "", "target_language": "", "speaker_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: TranslateTranslationMessage - id: x-parser-message-name value: TranslateTranslationMessage - &ref_9 id: TranslateFlushDoneMessage payload: - name: Translate Flush Done Message type: object properties: - name: type type: string description: Message type enumValues: - flush_done required: true - name: request_id type: string description: Request ID required: true headers: [] jsonPayloadSchema: type: object description: > Real-time translation session-complete message (sent by server). Emitted whenever the translation stream ends and all final results have been delivered — after a `flush` command, and also when the stream ends on its own. The next `audio` message starts a new session. properties: type: type: string enum: - flush_done description: Message type x-parser-schema-id: request_id: type: string description: Request ID example: req_1234567890 x-parser-schema-id: required: - type - request_id x-parser-schema-id: title: Translate Flush Done Message example: |- { "type": "", "request_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: TranslateFlushDoneMessage - id: x-parser-message-name value: TranslateFlushDoneMessage - &ref_10 id: TranslateErrorMessage payload: - name: Translate Error Message type: object properties: - name: type type: string description: Message type enumValues: - error required: true - name: error type: string description: Error description required: true - name: code type: string description: > Error code. Common values: `INVALID_MESSAGE` (malformed init/audio or a rejected field such as a non-pcm format or a sample_rate other than 16000), `INVALID_AUDIO_DATA` (audio `data` is not valid base64), `NOT_INITIALIZED`, `ALREADY_INITIALIZED`, `UNKNOWN_MESSAGE_TYPE`, `UNKNOWN_COMMAND`, `NO_ACTIVE_SESSION` (a `flush` arrived with no session in progress), `QUEUE_FULL` (too many messages are pending while the previous session finalizes; send audio at real-time pace and wait for `flush_done` before pipelining further), `CREDIT_EXCEEDED`, `TRANSLATION_ERROR`, `SERVICE_UNAVAILABLE`, `INTERNAL_ERROR`. required: true - name: request_id type: string description: Request ID required: false headers: [] jsonPayloadSchema: type: object description: Real-time translation WebSocket error message (sent by server) properties: type: type: string enum: - error description: Message type x-parser-schema-id: error: type: string description: Error description example: sample_rate must be 16000 x-parser-schema-id: code: type: string description: > Error code. Common values: `INVALID_MESSAGE` (malformed init/audio or a rejected field such as a non-pcm format or a sample_rate other than 16000), `INVALID_AUDIO_DATA` (audio `data` is not valid base64), `NOT_INITIALIZED`, `ALREADY_INITIALIZED`, `UNKNOWN_MESSAGE_TYPE`, `UNKNOWN_COMMAND`, `NO_ACTIVE_SESSION` (a `flush` arrived with no session in progress), `QUEUE_FULL` (too many messages are pending while the previous session finalizes; send audio at real-time pace and wait for `flush_done` before pipelining further), `CREDIT_EXCEEDED`, `TRANSLATION_ERROR`, `SERVICE_UNAVAILABLE`, `INTERNAL_ERROR`. example: INVALID_MESSAGE x-parser-schema-id: request_id: type: string description: Request ID example: req_1234567890 x-parser-schema-id: required: - type - error - code x-parser-schema-id: title: Translate Error Message example: |- { "type": "", "error": "", "code": "", "request_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: TranslateErrorMessage - id: x-parser-message-name value: TranslateErrorMessage bindings: [] extensions: *ref_0 sendOperations: - *ref_1 receiveOperations: - *ref_2 sendMessages: - *ref_3 - *ref_4 - *ref_5 receiveMessages: - *ref_6 - *ref_7 - *ref_8 - *ref_9 - *ref_10 extensions: - id: x-parser-unique-object-id value: translateRealtime securitySchemes: - id: apiKeyAuth name: token type: httpApiKey description: API Key passed as query parameter, e.g. ?token=YOUR_API_KEY in: query extensions: [] ````