Transcripts and commit strategies

Overview

When transcribing audio, you will receive partial and committed transcripts.

• Partial transcripts - the interim results of the transcription
• Committed transcripts - the final results of the transcription segment that are sent when a “commit” message is received. A session can have multiple committed transcripts.

The commit transcript can optionally contain word-level timestamps. This is only received when the “include timestamps” option is set to true.

# Initialize the connection
connection = await elevenlabs.speech_to_text.realtime.connect(RealtimeUrlOptions(
  model_id="scribe_v2_realtime",
  include_timestamps=True, # Include this to receive the RealtimeEvents.COMMITTED_TRANSCRIPT_WITH_TIMESTAMPS event with word-level timestamps
))

Commit strategies

When sending audio chunks via the WebSocket, transcript segments can be committed in two ways: Manual Commit or Voice Activity Detection (VAD).

Manual commit

With the manual commit strategy, you control when to commit transcript segments. This is the strategy that is used by default. Committing a segment will clear the processed accumulated transcript and start a new segment without losing context. Committing every 20-30 seconds is good practice to improve latency. Even if you do not commit manually, the model automatically commits after approximately 36 seconds of accumulated audio.

For best results, commit during silence periods or another logical point like a turn model.

await connection.send({
  "audio_base_64": audio_base_64,
  "sample_rate": 16000,
})
# When ready to finalize the segment
await connection.commit()

Sending previous text context

When sending audio for transcription, you can send previous text context alongside the first audio chunk to help the model understand the context of the speech. This is useful in a few scenarios:

• Agent text for conversational AI use cases - Allows the model to more easily understand the context of the conversation and produce better transcriptions.
• Reconnection after a network error - This allows the model to continue transcribing, using the previous text as guidance.
• General contextual information - A short description of what the transcription will be about helps the model understand the context.

await connection.send({
  "audio_base_64": audio_base_64,
  "previous_text": "The previous text context",
})

Voice Activity Detection (VAD)

With the VAD strategy, the transcription engine automatically detects speech and silence segments. When a silence threshold is reached, the transcription engine will commit the transcript segment automatically.

When transcribing audio from the microphone in the client-side integration, it is recommended to use the VAD strategy.

import { Scribe, AudioFormat, CommitStrategy } from "@elevenlabs/client";
const connection = Scribe.connect({
  token: "sutkn_1234567890",
  modelId: "scribe_v2_realtime",
  audioFormat: AudioFormat.PCM_16000,
  commitStrategy: CommitStrategy.VAD,
  vadSilenceThresholdSecs: 1.5,
  vadThreshold: 0.4,
  minSpeechDurationMs: 100,
  minSilenceDurationMs: 100,
});

Keeping the connection alive during silence

The official SDKs don’t drop the connection when no messages arrive, so most integrations don’t need this. If your own WebSocket client, or a proxy or load balancer in between, closes the connection when no frames arrive for a while, pass the optional keepalive_interval_ms query parameter when connecting. This matters during long stretches of silence, for example a phone call with 10-15 second pauses. About once per interval, the server sends a keepalive partial_transcript: empty (text: "") if the current segment has no uncommitted text, or a repeat of the latest partial text if it does.

• Accepts an integer between 500 and 10000 (milliseconds). It is disabled by default; omit the parameter to keep the existing behavior.
• Out-of-range or non-integer values cause the server to send an invalid_request error and close the connection.
• Keepalives are driven by the model actually processing the silent audio you send, not a free-running timer, so they also confirm the transcription path is alive.
• Audio is processed in roughly 1-second chunks, so keepalives arrive about once per interval rounded to that cadence (for example, 1000 fires about once a second, 3000 about once every 3 seconds). The first keepalive of a session arrives about 2 seconds after it starts, since the server buffers the first ~2 seconds of audio before transcribing. Set your interval to at most about a third of your own read timeout to leave margin.
• A keepalive’s text is only empty when the current segment has no uncommitted text yet. If a pause happens after speech but before a commit — most visible with filter_background_audio=true or in manual commit mode without committing — the keepalive repeats the latest partial text instead, so it never blanks interim text. After a commit, keepalives go back to empty until new speech arrives.
• Works with both commit_strategy=manual and commit_strategy=vad, and with filter_background_audio=true. Has no billing impact beyond the audio you’re already streaming.
• The session_started message’s config echoes back keepalive_interval_ms (null when disabled).

Add the parameter to the WebSocket URL:

wss://api.elevenlabs.io/v1/speech-to-text/realtime?model_id=scribe_v2_realtime&commit_strategy=vad&keepalive_interval_ms=1000

Supported audio formats

Best practices

Audio quality

• For best results, use a 16kHz sample rate for an optimum balance of quality and bandwidth.
• Ensure clean audio input with minimal background noise.
• Use an appropriate microphone gain to avoid clipping.
• Only mono audio is supported at this time.

Chunk size

• Send audio chunks of 0.1 - 1 second in length for smooth streaming.
• Smaller chunks result in lower latency but more overhead.
• Larger chunks are more efficient but can introduce latency.

Next steps