STT for Humans

This page describes the WebSocket protocol for Krisp streaming speech-to-text (STT) with the krisp-realtime-stt-humans-v1 model, built for transcribing human conversations. It supports speaker diarization and PII redaction.

Endpoint

• WebSocket URL: wss://streaming.krisp.ai/stt
• Auth: temporary session token. See Authentication.

Use the session token as the API key when opening the WebSocket. Do not send the master token on the WebSocket.

Authorization: api-key <session-token>

If your client cannot set headers, pass the same value as the authorization query parameter. See the API overview.

Connection and handshake flow

1) Client initiates connection + sends handshake message

After opening the WebSocket connection with the session token, the client must send an initial JSON handshake that defines the session.

Handshake message (client → server, JSON):

{
  "model": "krisp-realtime-stt-humans-v1",
  "vocabulary": ["word1", "word2"],
  "sample_rate": 16000,
  "speaker_diarization": false,
  "pii_redaction": false,
  "channels": 1,
  "audio_format": "pcm_s16le",
  "language_hints": ["en"],
  "metadata": {}
}

2) Server acknowledges readiness (or returns an error)

Once the server validates the handshake and is ready to receive audio, it responds with a ready message.

Ready response (server → client, JSON):

{
  "type": "ready",
  "request_id": "9f2c...",
  "metadata": {}
}

• request_id: unique identifier for the streaming session. Include it if you contact Krisp support.
• metadata: echo of the metadata object from the handshake.

Do not send audio before this message. If no handshake is received within 15 seconds, the server closes the session.

Error response

If the server cannot accept the session, it responds with an error message and closes the connection. Besides the errors below, the common errors apply.

Error response (server → client, JSON):

{
  "type": "error",
  "request_id": "9f2c...",
  "error": {
    "code": "invalid_api_key",
    "status_code": 401,
    "message": "Invalid Authorization header.",
    "details": {
      "hint": "Check authorization header."
    }
  }
}

Audio streaming

3) Client streams audio frames (raw bytes)

After receiving type: "ready", the client sends raw audio bytes as binary WebSocket messages.

• Message type: binary (raw bytes)
• Encoding: according to audio_format in the handshake: pcm_s16le (PCM signed 16-bit little-endian) or opus
• Sample rate: 16,000 Hz
• Channels: 1 (mono)
• Recommended chunk duration: 80 ms, or multiples of 80 ms
• Maximum message size: 65,536 bytes

Stream audio in realtime. Do not wrap audio in JSON.

Transcription messages

4) Server sends transcription results (JSON)

As the server processes audio, it returns transcription messages as JSON. Messages may be partial or final.

Example transcript response (server → client, JSON):

{
  "type": "transcript",
  "metadata": {},
  "transcript": {
    "words": [
      {
        "text": " weather",
        "start": 12480,
        "end": 12880,
        "language": "en",
        "language_confidence": 0.9921
      }
    ],
    "isFinal": true
  }
}

• text keeps its leading space, so joining text in arrival order gives the transcript.
• start and end bound the word, in milliseconds from the start of the stream. language and language_confidence are present when the server runs language ID (labels: see Supported languages).
• metadata is always empty.
• A final never changes and is never repeated.

5) Diarization

Add "speaker_diarization": true to the handshake:

{
  "model": "krisp-realtime-stt-humans-v1",
  "speaker_diarization": true
}

Diarization frames arrive between transcript frames:

{
  "type": "diarization",
  "committed": [{ "start": 463400, "end": 469400, "speaker": 5 }],
  "provisional": [
    { "start": 471500, "end": 498500, "speaker": 5 },
    { "start": 499100, "end": 502300, "speaker": 2 }
  ],
  "n_speakers": 6
}

A frame is sent only when something changed.

The current speaker timeline is committed followed by the latest provisional. The two never overlap.

A word belongs to the segment where start <= word.start < end:

• Segment edges are snapped to word starts, so every word falls in exactly one segment.
• Punctuation and word pieces stay with their word.
• A word under a committed segment keeps its speaker for good. A word under a provisional segment can still change speaker.

Timing

Show provisional labels straight away and repaint them as frames arrive. Don't wait for committed labels in a live UI: commits exist to give storage and analytics a guarantee that labels won't change.

Session end

After end_of_audio, frames arrive in this order:

1. the remaining transcript frames;
2. the final diarization frame with "last": true, after which every segment is committed;
3. end.

The session is over at end, not at last. Closing the socket without end_of_audio loses the final diarization state.

6) Stream end

To end the stream, the client sends one text frame after its last audio message:

{"type": "end_of_audio"}

The server then finishes the session and closes the socket itself. The client shouldn't close it; it should read frames until end arrives.

After end_of_audio, the last frame the server sends is end, and it closes the socket right after:

{
  "type": "end",
  "language": {
    "overall_language": "en",
    "overall_confidence": 0.982,
    "current_language": "en",
    "current_confidence": 0.991,
    "per_language": {
      "de": 0.003,
      "en": 0.982,
      "es": 0.004,
      "fr": 0.002,
      "it": 0.001,
      "nl": 0.001,
      "pt": 0.001,
      "zh-CN": 0.001,
      "ru": 0.003,
      "hy-AM": 0.002
    }
  }
}

The last frame of a session. language is present when the server runs language ID:

• overall_* averages the whole session.
• current_* covers the last 0.5 s.
• per_language holds the mean probability of every language ID label over the session (all ten, see Supported languages).

Supported languages

The model transcribes the ten languages below. Output is limited to Latin, Cyrillic, Armenian and Chinese characters, so languages written in other scripts are not transcribed. The language ID labels are the values of words[].language and the keys of end.language.per_language; language_hints takes the bare code.

Several hints allow the union of their scripts. Hints covering Cyrillic, Armenian and Chinese together, an unknown code or an empty list allow every script. Language ID only knows these ten labels: speech in another language is labelled as the closest of them.

Example (Python)

Streams a 16 kHz, mono, 16-bit PCM WAV file at realtime pace with diarization on, and prints every message from the server. Requires pip install "websockets>=14".

import asyncio
import json
import sys
import wave

import websockets

SESSION_TOKEN = "YOUR_SESSION_TOKEN"  # see the Authentication page
URL = "wss://streaming.krisp.ai/stt"
CHUNK_MS = 480  # a multiple of 80 ms


async def main(path: str) -> None:
    headers = {"Authorization": f"api-key {SESSION_TOKEN}"}
    async with websockets.connect(URL, additional_headers=headers) as ws:
        await ws.send(json.dumps({
            "model": "krisp-realtime-stt-humans-v1",
            "sample_rate": 16000,
            "channels": 1,
            "audio_format": "pcm_s16le",
            "speaker_diarization": True,
        }))
        hello = json.loads(await ws.recv())
        if hello["type"] != "ready":
            raise RuntimeError(hello)

        async def send_audio() -> None:
            with wave.open(path, "rb") as wav:
                frames = 16000 * CHUNK_MS // 1000
                while chunk := wav.readframes(frames):
                    await ws.send(chunk)
                    await asyncio.sleep(CHUNK_MS / 1000)
            await ws.send(json.dumps({"type": "end_of_audio"}))

        sender = asyncio.create_task(send_audio())
        async for message in ws:
            msg = json.loads(message)
            print(msg)
            if msg.get("type") == "end":
                break
        await sender


asyncio.run(main(sys.argv[1]))

Changelog

• 2026-10-01: Page created.