Skip to main content

Publishing and receiving audio

You can publish audio, receive translated audio, and control the translation through either transport:

  • WebRTC — best for client applications (browsers, mobile apps). Handled by LiveKit.
  • WebSockets — convenient for server-side integrations. Authenticate with your API Key directly — see Translation management API for the connection details.

For WebRTC (or if you manage sessions yourself), start a streaming session first — Palabra returns two URLs and a JWT access token:

PurposeFieldTypical value
WebRTC (audio/control)webrtc_urlhttps://<STREAMING_SERVER>.palabra.ai/livekit/
WebSocket (audio/control)ws_urlwss://<STREAMING_SERVER>.palabra.ai/streaming-api/v1/speech-to-speech/stream
AuthenticationpublishereyJhbGciOiJIUzI1NiIsInR5cCI6…
Important
  • Regardless of the transport, you control the translation by sending JSON text messages — through the WebRTC data channel or the WebSocket connection respectively. See the Translation management API.
  • If you choose WebSockets as the audio transport, the audio chunks you push must match the format / sample_rate / channels declared in your set_task command.

Using the WebRTC transport​

Use any LiveKit client library to publish your audio track, then create a translation task using the Translation management API. Palabra will publish a translated audio track for each target language.

The Palabra translator joins the LiveKit room a moment after you do, and a control message published before it is in the room is silently lost — the room accepts the data message, but there is no one subscribed to receive it, so the translation never starts. Wait for the translator to join before sending your first set_task; see the waitForTranslator helper in the WebRTC Quick Start (Step 6). This applies to WebRTC only — over WebSockets you can send set_task as soon as the connection is open.

Code examples​

See the Quick Start Guide for code examples of publishing original audio (Step 4) and receiving translated audio (Step 5).

Using the WebSocket transport​

Connect to the WebSocket endpoint using your API Key (or to ws_url with a session publisher token — see Translation management API), create a translation task, then start sending and receiving audio chunks as described below.

Publishing​

Send base64-encoded audio chunks over the WebSocket. The chunks must match the format, sample_rate, and channels declared in your set_task command. The optimal chunk length is 320 ms.

Message format example:

{
  "message_type": "input_audio_data",
  "data": {
    "data": "base64 encoded data"
  }
}

Receiving​

Palabra sends TTS audio chunks as output_audio_data messages over the same WebSocket connection. The chunks are base64-encoded; the default format is 24 kHz 16-bit mono PCM (can be changed with the set_task command).

Message format example:

{
  "message_type": "output_audio_data",
  "data": {
    "transcription_id": "190983855fe3404e",
    "language": "es",     // TTS language
    "last_chunk": false,  // true on the last generated chunk for this `transcription_id`
    "data": "base64 string"
  }
}