Multi-Language Options

You can publish only one audio track at a time for translation, but it can be translated into multiple languages — either simultaneously or conditionally.

Simultaneous Multi-Language Translation

The speech-to-speech translation API can publish multiple translated tracks using either WebRTC or WebSockets. To enable this, add a separate translation and speech generation configuration for each target language in the translations section of the set_task command.

Below is an example of translating from French to both English and Spanish:

{
  // ......
  "transcription": {
    "source_language": "fr",
    // ......
  },
  "translations": [
    {
      "target_language": "en-us",
      // ......
      "speech_generation": {
        // ......
      }
    },
    {
      "target_language": "es",
      // ......
      "speech_generation": {
        // ......
      }
    }
  ],
  // ......
}

Conditional Multi-Language Translation

Alternatively, you can perform conditional translation into different languages within a single output track. This works by defining language pairs and enabling automatic source language detection: when you speak language M, it is translated into language X; when you speak language N, it is translated into language Y. You can define more than two language pairs if needed.

Note the requirements for automatic language detection — it is an experimental feature with a limited set of supported languages.

To set it up:

1. Set "source_language": "auto" to enable automatic detection.
2. Optionally limit detection to specific languages with the detectable_languages field — for example, "detectable_languages": ["fr", "es"].
3. Define a separate translation config for each target language and restrict each one to specific source languages using the allowed_source_languages field.

Below is an example that translates:

• French (fr) to American English (en-us)
• Spanish (es) to German (de)

{
  // ......
  "transcription": {
    "source_language": "auto",
    "detectable_languages": ["fr", "es"],
    // ......
  },
  "translations": [
    {
      // fr → en-us pair
      "target_language": "en-us",
      "allowed_source_languages": ["fr"],
      // ......
      "speech_generation": {
        // ......
      }
    },
    {
      // es → de pair
      "target_language": "de",
      "allowed_source_languages": ["es"],
      // ......
      "speech_generation": {
        // ......
      }
    }
  ],
  // ......
}