Multi-Language Options

Currently, you can only publish one audio track at a time to the API for translation, but it can be translated into multiple languages simultaneously or conditionally.

Simultaneous Multi-Language Translation

The speech-to-speech translation API can publish multiple translated tracks using either WebRTC or WebSockets. To enable this, you need to add separate translation and speech generation configurations for each target language in the translations section of the set_task command.

Below is an example of translating from French to both English and Spanish:

---

{
  // ......
  "transcription": {
    "source_language": "fr",
    // ......
  },
  "translations": [
    {
      "target_language": "en-us",
      // ......
      "speech_generation": {
        // ......
      }
    },
    {
      "target_language": "es",
      // ......
      "speech_generation": {
        // ......
      }
    }
  ],
  // ......
}

Conditional Multi-Language Translation

Alternatively, you can perform conditional translation into different languages within a single output track. This works by defining translation language pairs and enabling automatic source language detection. For example, when you speak language M, it will be translated into language X; and when you speak language N, it will be translated into language Y. You can define more than two language pairs if needed.

To achieve this:

1. Set "source_language": "auto" to enable automatic detection.
2. Optionally limit the detection to specific languages using the "detectable_languages" field — for example, "detectable_languages": ["fr", "es"].
3. Define a separate translation config for each target language and restrict each to specific source languages using the allowed_source_languages field.

Below is an example that translates:

• French (fr) to American English (en-us)
• Spanish (es) to German (de)

{
  // ......
  "transcription": {
    "source_language": "auto",
    "detectable_languages": ["fr", "es"],
    // ......
  },
  "translations": [
    {
      // fr → en-us pair
      "target_language": "en-us",
      "allowed_source_languages": ["fr"],
      // ......
      "speech_generation": {
        // ......
      }
    },
    {
      // es → de pair
      "target_language": "de",
      "allowed_source_languages": ["es"],
      // ......
      "speech_generation": {
        // ......
      }
    }
  ],
  // ......
}