> For the complete documentation index, see [llms.txt](https://docs-conversational-ai.syntphony.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs-conversational-ai.syntphony.com/voice-gateway/evg-connector/elevenlabs.md). # ElevenLabs ## Overview ElevenLabs is available in Syntphony Conversational AI (SCAI) as a voice provider alongside the platform's other supported providers. It supports two independent capabilities: * **Text-to-Speech (TTS)** converts conversation text into speech for the caller. * **Speech-to-Text (STT)** converts caller speech into text for the conversation. You can use ElevenLabs for TTS, STT, or both in the same Voice Gateway channel. Each capability is configured independently in Cockpit. Voice Gateway manages the voice channel and conversation behavior. ElevenLabs provides speech synthesis and speech recognition. Configure ElevenLabs speech settings separately from Voice Gateway turn-taking settings. ### How it works ``` Caller │ │ speech ▼ ElevenLabs STT │ │ transcription ▼ Voice Gateway │ │ response text ▼ ElevenLabs TTS │ │ synthesized speech ▼ Caller ``` ### Availability ElevenLabs is available in these Voice Gateway environments: | Environment | | ---------------------- | | Europe – Google Cloud | | Europe – Azure | | America – Google Cloud | ### Before you begin To configure ElevenLabs, you need an ElevenLabs API key. For Text-to-Speech, you also need: * A supported ElevenLabs TTS model * An ElevenLabs Voice ID. You can use a free ElevenLabs account to explore voices, listen to samples, and obtain Voice IDs. For production channels, use the ElevenLabs account that provides access to the voice required by your project. ## Configure Text-to-Speech Text-to-Speech converts response text into audio that callers hear. The resulting speech depends on the combination of: * TTS model * Voice * Synthesis settings * Input text. Choose the model and voice first. Then configure and tune the channel. ### Choose a TTS model Choose the model according to the language, latency, and expressiveness required for the experience: | If you need | Model | What to know | | -------------------------------------------------------------- | ------------------------ | ---------------------------------------------------------------- | | Realtime Spanish or multilingual speech | `eleven_flash_v2_5` | Low latency; 32 languages | | Realtime English speech | `eleven_flash_v2` | Low latency; English only | | Multilingual speech where realtime latency is not the priority | `eleven_multilingual_v2` | 29 languages; text normalization enabled by default | | Highly expressive speech | `eleven_v3` | Audio Tags; higher latency; **not recommended for realtime IVR** | #### Flash v2.5 Use `eleven_flash_v2_5` for realtime Spanish and other supported multilingual voice experiences. It provides approximately 75 ms latency, supports 32 languages including Spanish from Spain and Mexico, supports ``, and accepts up to 40,000 characters per request. Text normalization is disabled by default. #### Flash v2 Use `eleven_flash_v2` for realtime English voice experiences. It provides approximately 75 ms latency, supports English, supports ``, and accepts up to 40,000 characters per request. #### Multilingual v2 Use `eleven_multilingual_v2` for multilingual synthesis where realtime latency is not the primary requirement. It supports 29 languages, enables number normalization by default, supports ``, and accepts up to 10,000 characters per request. #### Eleven v3 Use `eleven_v3` when expressive delivery is more important than realtime performance. It supports Audio Tags that can influence emotion, delivery, rhythm, accent, and sound effects. Eleven v3: * Does not support SSML * Accepts up to 5,000 characters per request * Has high latency * Is not recommended for realtime IVR or streaming scenarios Professional Voice Clone support is limited. #### Eleven v3 Conversational `eleven_v3_conversational` is a conversational variant of Eleven v3. It retains Audio Tag support and expressive speech behavior while providing latency improvements for conversational interactions. It does not support SSML and accepts up to 5,000 characters per request. ### Choose and test a voice Choose a voice that matches the language, accent, and type of conversation you are designing. Test the voice with the same TTS model and representative content you plan to use in production. #### Choose a voice In ElevenLabs, open **Voices** and select a voice available to your account. For IVR and other conversational experiences, prioritize voices intended for **Conversational** or **Customer service** use. For Spanish voice experiences, you can narrow the search using: | Filter | Examples | | -------- | ---------------------------------------------------- | | Language | Spanish (Spain), Spanish (Mexico) | | Accent | Spain, Mexico, or another available regional variant | | Use case | Conversational, Customer service | | Gender | Male, Female | | Age | Young, Middle-aged, Old | {% hint style="info" %} Voice availability depends on the ElevenLabs account. Project-specific cloned voices require access through the account where those voices are available. {% endhint %} #### Test the voice 1. Open the voice details in ElevenLabs 2. Enter text representative of your production conversations 3. Select the TTS model you plan to use 4. Generate the sample 5. Evaluate the result. Check: * Pronunciation * Clarity * Pacing * Accent and language suitability * Tone for the intended conversation. Include realistic content such as names, numbers, dates, abbreviations, and domain-specific terminology. A prerecorded voice sample can differ from speech generated with your selected model and configuration. Use generated samples with representative content when evaluating a voice for production. ### Get the Voice ID After selecting a voice, get its Voice ID. You will use this value as `voiceId` when configuring TTS in Cockpit: | Method | How to get the Voice ID | | ----------------- | --------------------------------------------------- | | My Voices | Open the voice options and select **Copy Voice ID** | | Voice details URL | Copy the identifier from the end of the URL | | API | Use `GET /v1/voices` with the ElevenLabs API key | Example: ``` UOIqAnmS11Reiei1Ytkc ``` {% hint style="info" %} Some voices use two or three times the normal credits per character. **Check the applicable cost before selecting a production voice.** {% endhint %} ### Configure TTS in Cockpit ElevenLabs can be selected directly as the TTS provider in Cockpit. Selecting the provider does not require additional development. After choosing a model and voice: 1. Open Cockpit 2. Open the Voice Gateway channel you want to configure 3. Select **Edit channel** 4. Go to **TTS / Audio Configuration** 5. Enable **Barge-in** if callers should be able to interrupt synthesized speech 6. Enable **Voice Barge-in** if caller speech should trigger the interruption 7. Set **TTS Default Mask** to `$TEXT` 8. Set **TTS Voice Provider** to **Eleven Labs** 9. Enter the provider configuration in **TTS Voice Provider Configuration** 10. Save the channel. #### TTS settings | Cockpit field | Value | What it does | | -------------------------------- | ----------- | ------------------------------------------------------------- | | Barge-in | On | Allows callers to interrupt TTS playback | | Voice Barge-in | On | Allows caller speech to trigger an interruption | | TTS Default Mask | `$TEXT` | Sends plain text to ElevenLabs | | TTS Voice Provider | Eleven Labs | Uses ElevenLabs for speech synthesis | | TTS Voice Provider Configuration | JSON | Defines the ElevenLabs connection and synthesis configuration | Set **TTS Default Mask** to `$TEXT` only. Do not add an SSML wrapper or Audio Tags to the mask. Unsupported markup can be synthesized as spoken text. ### Configure the TTS provider A TTS configuration requires three identifiers: | Parameter | Value | What it does | | --------- | ----------------------------- | ----------------------------- | | `apiKey` | String | Authenticates with ElevenLabs | | `voiceId` | String | Identifies the voice | | `modelId` | Supported ElevenLabs Model ID | Selects the TTS model | You can obtain the API key from the ElevenLabs dashboard under **Developer > API Key**. Use the Voice ID from the voice you selected earlier. #### Supported synthesis parameters The provider configuration supports: | Parameter | Accepted values | What it affects | | -------------------------- | ----------------- | ------------------------------------------ | | `stability` | `0.0`–`1.0` | Consistency of the generated voice | | `similarityBoost` | `0.0`–`1.0` | Similarity to the source voice | | `style` | `0.0`–`1.0` | Stylistic variation | | `useSpeakerBoost` | `true` / `false` | Speaker enhancement | | `optimizeStreamingLatency` | `0`–`4` or `null` | Balance between latency and speech quality | #### Realtime Spanish example ``` { "apiKey": "sk_xxxxxxx", "modelId": "eleven_flash_v2_5", "voiceId": "YOUR_VOICE_ID", "stability": 0.5, "similarityBoost": 0.7, "style": 0.4, "useSpeakerBoost": true, "optimizeStreamingLatency": null } ``` #### Realtime English example ``` { "apiKey": "sk_xxxxxxx", "modelId": "eleven_flash_v2", "voiceId": "YOUR_VOICE_ID", "stability": 0.5, "similarityBoost": 0.7, "style": 0.4, "useSpeakerBoost": true, "optimizeStreamingLatency": null } ``` #### Cached multilingual example ``` { "apiKey": "sk_xxxxxxx", "modelId": "eleven_multilingual_v2", "voiceId": "YOUR_VOICE_ID", "stability": 0.6, "similarityBoost": 0.75, "style": 0.35, "useSpeakerBoost": true, "optimizeStreamingLatency": null } ``` ### Tune the generated speech Start with a working configuration before adjusting individual synthesis settings. Change one setting at a time and test the result with representative production content. #### Voice quality | Parameter | Accepted values | Typical values | What it affects | | ----------------- | ---------------- | --------------------------------------------------------- | ---------------------------------- | | `stability` | `0.0`–`1.0` | Realtime IVR: `0.45`–`0.55`; cached speech: `0.55`–`0.65` | Consistency of the generated voice | | `similarityBoost` | `0.0`–`1.0` | `0.70`–`0.75` | Similarity to the source voice | | `style` | `0.0`–`1.0` | Realtime IVR: `0.35`–`0.45`; cached speech: `0.3`–`0.4` | Stylistic variation | | `useSpeakerBoost` | `true` / `false` | `true` | Speaker enhancement | Higher `stability` values prioritize consistency. #### Latency and speech rate | Parameter | Accepted values | Typical value | What it affects | | -------------------------- | ----------------- | ---------------------------------------- | ------------------------------------------ | | `optimizeStreamingLatency` | `0`–`4` or `null` | Realtime IVR: `2`; cached speech: `null` | Balance between latency and speech quality | | `speed` | `0.7`–`1.2` | Default: `1.0`; IVR: `0.9` | Speech rate | For `optimizeStreamingLatency`: * `null` prioritizes speech quality * `2` balances latency and speech quality for realtime IVR * `4` provides the lowest latency with reduced quality. ### Control pauses and expressiveness ElevenLabs **does not support the same markup across all TTS models:** | Capability | Eleven v3 | Flash v2.5 / Flash v2 | | --------------------- | ------------- | -------------------------- | | `` | Not supported | Not supported | | `` | Not supported | Supported, up to 3 seconds | | `` | Not supported | Not supported | | `` | Not supported | Not supported | | `` | Not supported | Not supported | | Audio Tags | Supported | Not supported | #### Flash v2.5 and Flash v2 Flash models **do not support** Audio Tags. Use text and synthesis settings to influence delivery: * Use commas and periods to introduce pauses * Use ellipses to introduce hesitation * Use exclamation marks to increase energy * Use capitalization selectively for emphasis * Use `speed` to control speech rate * Normalize numbers and dates when predictable pronunciation is required. Use no more than one capitalized word per sentence for emphasis. Flash models support `` for pauses of up to three seconds. Example: ``` Your order has been confirmed. It will arrive tomorrow. ``` #### Eleven v3 Eleven v3 uses Audio Tags instead of SSML to influence expressive delivery. Place a tag before the text it affects. ``` [tired][nervously] I have checked the order again. ``` Examples include: | Purpose | Examples | | -------------------- | ---------------------------------------------------------------- | | Emotion | `[excited]`, `[nervous]`, `[frustrated]`, `[tired]`, `[curious]` | | Delivery | `[whispers]`, `[quietly]`, `[hesitant]`, `[dramatically]` | | Rhythm | `[pause]`, `[rushed]`, `[drawn out]`, `[slowly]` | | Non-verbal reactions | `[sighs]`, `[laughs]`, `[light chuckle]` | | Accent or character | `[French accent]`, `[American accent]` | | Environmental sounds | `[clapping]`, `[explosion]` | Multiple tags can be combined. Eleven v3 can produce non-deterministic results and is not recommended for cached Voice Gateway scenarios. ### Control text normalization Text normalization determines how values such as numbers, dates, and currencies are prepared for speech. Configure `applyTextNormalization` inside `customParameters`. | Value | Behavior | | ------ | ------------------------------------------------------------ | | `auto` | Lets the provider determine whether normalization is applied | | `on` | Enables text normalization | | `off` | Disables text normalization | For Flash v2.5, text normalization is disabled by default to preserve low latency. Enabling normalization with `on` is available only for Enterprise customers. `auto` does not enable normalization on non-Enterprise plans. #### Specify the TTS language Use `language_code` when a supported model needs an explicit synthesis language. Example values: * `es` — Spanish * `en` — English `language_code` uses ISO 639-1 language codes. If the selected model does not support the configured code, the value is ignored. `language_code` is not supported by `eleven_multilingual_v2`. {% hint style="info" %} The TTS `language_code` is separate from the STT `language_code`. **Changing one does not configure the other.** {% endhint %} ### Control pronunciation Use pronunciation dictionaries when names, brands, acronyms, or domain-specific terminology require controlled pronunciation. A dictionary entry requires: ``` pronunciationDictionaryId ``` You can optionally specify: ``` versionId ``` If `versionId` is omitted, the latest published dictionary version is used. You can apply up to **three dictionaries to one request**. They are processed in the order in which they appear, and entries are case-sensitive. For production channels, specify `versionId` when pronunciation must remain tied to a specific published dictionary version. Phoneme tags using IPA or CMU Arpabet are supported by `eleven_flash_v2` and `eleven_v3`. Flash v2.5 ignores these phoneme tags. Use aliases with models that do not support them. ### Supported custom parameters The supported `customParameters` are: | Parameter | Purpose | | --------------------------- | ------------------------------------------------------- | | `applyTextNormalization` | Controls text normalization | | `speed` | Controls speech rate | | `pronunciationDictionaries` | Defines pronunciation dictionaries applied to synthesis | Each item in `pronunciationDictionaries` can include: | Parameter | Purpose | | --------------------------- | ----------------------------------------- | | `pronunciationDictionaryId` | Identifies the pronunciation dictionary | | `versionId` | Identifies a published dictionary version | {% hint style="info" %} Do not add additional custom parameters unless they are supported by the SCAI connector. {% endhint %} #### Advanced TTS example ``` { "apiKey": "sk_xxxxxxx", "modelId": "eleven_flash_v2_5", "voiceId": "YOUR_VOICE_ID", "stability": 0.5, "similarityBoost": 0.7, "style": 0.4, "useSpeakerBoost": true, "optimizeStreamingLatency": null, "customParameters": { "applyTextNormalization": "auto", "speed": 1.1, "pronunciationDictionaries": [ { "pronunciationDictionaryId": "AwNGRHGmbR9Xl49IXJ3c", "versionId": "TldyRzXBT3pWJDWk3ZpF" } ] } } ``` ## Configure Speech-to-Text Speech-to-Text converts caller speech into text during the conversation. ElevenLabs STT uses the Scribe model family: | Model | API ID | Use | | ------------------ | -------------------- | ------------------------------------------ | | Scribe v2 | `scribe_v2` | Batch transcription | | Scribe v2 Realtime | `scribe_v2_realtime` | Realtime voice and telephony transcription | For Voice Gateway telephony, use `scribe_v2_realtime`. Scribe v2 Realtime provides approximately 150 ms latency, supports more than 90 languages, and accepts up to 50 keyterms. ### Choose the transcription language You do not need a different Scribe model for each language. Choose between: * A known language * Automatic language detection. #### Use a known language When the channel language is known, configure `language_code`. | Language | ISO 639-1 | ISO 639-3 | | -------- | --------- | --------- | | Spanish | `es` | `spa` | | Catalan | `ca` | `cat` | | English | `en` | `eng` | Valencian is transcribed as Catalan. Specifying `language_code` is particularly useful when callers provide short inputs such as names or identification numbers. #### Use automatic language detection If you omit `language_code`, Scribe predicts the language from the incoming audio. Automatic detection supports multilingual interactions, but it can be unreliable for very short utterances. The configuration does not provide a language whitelist. Use either: * One explicit `language_code`, or * Automatic detection across the supported languages. ### Configure STT in Cockpit 1. Open the Voice Gateway channel in Cockpit 2. Select **Edit channel** 3. Go to **Voice Menu** 4. Set **ASR Voice Provider** to **Eleven Labs** 5. Enter the provider configuration in **ASR Voice Provider Configuration** 6. Save the channel. **ASR Voice Provider Configuration** supports up to **4,096 characters**. ### Realtime endpoint Use: ``` wss://api.elevenlabs.io/v1/speech-to-text/realtime ``` The model and STT options are passed as query-string parameters in the endpoint URL. The API key is configured separately in the provider JSON. #### Spanish example ``` { "customParameters": { "apiKey": "sk_xxxxxxx", "url": "wss://api.elevenlabs.io/v1/speech-to-text/realtime?model_id=scribe_v2_realtime&language_code=es" } } ``` #### Catalan example ``` { "customParameters": { "apiKey": "sk_xxxxxxx", "url": "wss://api.elevenlabs.io/v1/speech-to-text/realtime?model_id=scribe_v2_realtime&language_code=ca" } } ``` The `&` character used between query-string parameters can be entered directly in the JSON string. It does not require escaping. ### Configure transcription behavior Add supported STT options to the realtime endpoint as query-string parameters:
ParameterAccepted valueWhat it does
model_idscribe_v2_realtimeSelects the realtime transcription model
language_codeISO 639-1 or ISO 639-3Sets the transcription language
keytermsMaximum 50 termsBiases recognition toward selected vocabulary
no_verbatimtrue / false; default falseRemoves fillers, false starts, and disfluencies
include_timestampstrue / false; default falseReturns word-level timestamps
include_language_detectiontrue / false; default falseAdds detected-language information
enable_loggingtrue / false; default trueControls ElevenLabs logging behavior
{% hint style="info" %} `audio_format`, `commit_strategy`, and VAD-related parameters are managed internally by the SCAI connector. Do not add them to the channel URL. {% endhint %} ### Improve recognition of important vocabulary Use `keyterms` when callers are likely to use important words that need additional recognition bias. Typical candidates include: * Product names * Brand names * Domain-specific terminology * Other vocabulary important to the conversation. Scribe v2 Realtime accepts up to 50 keyterms. Each term can contain up to 20 characters. {% hint style="info" %} Using `keyterms` applies a **20% surcharge over the base transcription cost**. {% endhint %} ## Tune conversation turn-taking Speech recognition determines **what the caller said**. Voice Gateway turn-taking determines **when the caller's turn starts and ends**. These behaviors are configured separately. | Cockpit setting | Value | What it controls | | ------------------ | -------- | --------------------------------------------------- | | Timeout | 2800 ms | Maximum wait for caller input before reprompting | | Sensitivity | 0.65 | SCAI voice-detection sensitivity | | Confidence | 0.7 | Recognition confidence threshold | | Max Speech Timeout | 60000 ms | Maximum duration of one speech turn | | Incomplete Timeout | 1500 ms | Silence after partial speech before the turn closes | **Timeout**, **Sensitivity**, and **Incomplete Timeout** control turn-taking in SCAI. They do not configure ElevenLabs VAD behavior. Pay particular attention to **Incomplete Timeout**. If it is too short, Voice Gateway can close the turn while a caller is: * Spelling a surname * Reading a number * Pausing between pieces of information. Increasing the value gives callers more time to continue speaking, but also delays turn completion. ## Test the voice experience Test the complete channel through a real call using representative production scenarios. ### Test Text-to-Speech Confirm that: * **TTS Voice Provider** is **Eleven Labs** * **TTS Default Mask** contains only `$TEXT` * The intended `apiKey`, `modelId`, and `voiceId` are configured * Pronunciation is correct * Speech is clear and appropriate for the conversation * Latency is acceptable * Callers can interrupt synthesized speech as intended. Test the representative content used during voice selection. ### Test Speech-to-Text Confirm that: * **ASR Voice Provider** is **Eleven Labs** * The endpoint uses `scribe_v2_realtime` * `language_code` is configured when the channel uses a known language * The connector session information reports the expected language. Test realistic caller behavior, including: * Short answers * Names * Numbers * Spelling * Pauses * Longer speech turns. When testing turn-taking, verify that callers have enough time to complete their input without introducing unnecessary delays. ### Calibrate with recorded calls For recognition calibration using recorded calls, you can use `scribe_v2` batch transcription. Batch transcription can help evaluate recognition performance against recorded audio. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs-conversational-ai.syntphony.com/voice-gateway/evg-connector/elevenlabs.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.