Ambient Audio Streaming - Overview

Stream Ambient or Form filling visit audio on GET /ws/stream after session creation: PCM chunking, JSON control frames, REST completion, and result retrieval

Quick summary

Streaming sends live visit audio to Suki over WebSocket after you create a session and seed context. The WebSocket carries JSON text frames for audio and control messages. Final notes, form fields, transcripts, and structured data come from the matching product REST APIs after you end the session.

Last updated:August 2026

Ambient streaming lets your product send a visit conversation to Suki while the clinician is still in the room. First, create the ambient session and seed session context. Then open the GET /ws/stream WebSocket and send the visit audio in small chunks. When you stop streaming and end the session, Suki uses that audio to generate the clinical note.

The Ambient REST APIs handle session creation, session completion, status, and note retrieval. They do not carry live visit audio. The WebSocket handles the live audio stream. After you end the session, use the Ambient REST APIs to retrieve the note and other supported results.

What you can do with ambient streaming

Use the Ambient and Streaming APIs together to:

• Capture audio from your own backend, mobile app, or custom client.
• Stream visit audio to Suki in small chunks while the encounter is in progress.
• Pause and resume the stream, or keep a paused stream alive, by sending EVENT messages.
• Complete the stream and retrieve the resulting data through the Ambient REST APIs.

WebSocket and REST responsibilities

Interface	Purpose
GET /ws/stream	Sends live visit audio and stream control messages to Suki
Ambient REST APIs	Create and end the session, check processing status, and retrieve the note and other supported results

The WebSocket is the live audio path. The REST APIs are the session and results path.

This guide focuses on GET /ws/stream for Ambient and Form filling. For real-time transcript text in your application, see Stream Dictation audio. For recorder Start, Pause, and Stop, see Stream ambient audio in your product.

These patterns apply when you build your own streaming client with the Ambient APIs. The headed Web SDK already captures and streams visit audio. The Headless Web SDK uses React hooks instead of this Partner WebSocket wire format.

Enable streaming

Sessions shorter than 1 minute may not contain enough audio for note generation and can be marked as skipped.

1. Create the Ambient Session

   Call Create ambient session before you open the socket. Store ambient_session_id from create. The WebSocket handshake needs that ID, and the session must still be CREATED.

   Opening /ws/stream before the session exists, or after the session leaves CREATED, returns FailedPrecondition.

2. Seed Session Context

   Call Seed session context so note generation has encounter details. Context is not what authenticates the socket, but you should send it before you stream.

3. Open the Socket and Stream Audio

   After create and context succeed, open wss://sdp.suki.ai/ws/stream. Send one START_TIME message to mark the start of this recording segment, then one AUDIO message per PCM chunk. When the clinician taps Stop, send RU9G as the last AUDIO message and close the socket.

Audio Streaming API

Create Ambient Session API

The following code samples show how to loop through chunks:

TypeScript

js

const ws = new WebSocket("wss://sdp.suki.ai/ws/stream", [
  `SukiAmbientAuth,${sdpSukiToken},${ambientSessionId}`,
]);

ws.onopen = () => {
  const startTime = new Date().toISOString().replace(/\.\d{3}Z$/, "Z");
  ws.send(
    JSON.stringify({
      type: "START_TIME",
      data: btoa(startTime),
    })
  );

  // Send ~100 ms LINEAR16 PCM chunks as standard Base64.
  for (const pcmChunkBase64 of pcmChunks) {
    ws.send(JSON.stringify({ type: "AUDIO", data: pcmChunkBase64 }));
  }

  ws.send(JSON.stringify({ type: "AUDIO", data: "RU9G" }));
  ws.close();
};

Python

python

import base64
import json
from datetime import datetime, timezone

import websocket

ws = websocket.create_connection(
    "wss://sdp.suki.ai/ws/stream",
    header=[
        f"sdp_suki_token: {sdp_suki_token}",
        f"ambient_session_id: {ambient_session_id}",
    ],
)

rfc3339 = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
ws.send(
    json.dumps(
        {
            "type": "START_TIME",
            "data": base64.b64encode(rfc3339.encode("utf-8")).decode("ascii"),
        }
    )
)

# Send ~100 ms LINEAR16 PCM chunks as standard Base64.
for pcm_chunk in pcm_chunks:
    ws.send(
        json.dumps(
            {
                "type": "AUDIO",
                "data": base64.b64encode(pcm_chunk).decode("ascii"),
            }
        )
    )

ws.send(json.dumps({"type": "AUDIO", "data": "RU9G"}))
ws.close()

The TypeScript sample authenticates a browser client. The Python sample authenticates a non-browser client.

Client	How you authenticate
Browser	Pass Sec-WebSocket-Protocol as SukiAmbientAuth,<sdp_suki_token>,<ambient_session_id>. The token comes before the session ID.
Non-browser	Send sdp_suki_token and ambient_session_id as HTTP headers on the WebSocket upgrade. Do not use Sec-WebSocket-Protocol.

For the browser handshake recipe, see Authenticate browser WebSocket handshake.

If you do not have credentials yet, complete Partner onboarding and Partner authentication to get an sdp_suki_token. Staging examples use wss://sdp.suki-stage.com/ws/stream.

Form filling uses the same Partner WebSocket and the same ambient message protocol. Use only the ID from Create Form filling session on Form filling REST, including End Form filling session, and on /ws/stream. That ID is not an Ambient clinical-note ambient_session_id. Ambient End rejects Form filling jobs.

Send JSON text frames

/ws/stream does not accept raw PCM as a binary WebSocket frame. Each message is a UTF-8 JSON text frame that contains exactly one JSON object. Base64 encode the audio bytes, then put that string in the JSON data field.

A stream segment is one recording pass on the socket: start, audio, optional controls, then end-of-audio. Send messages in this order:

{ "type": "START_TIME", "data": "<base64(timestamp)>" }

{ "type": "AUDIO", "data": "<base64(pcm chunk 1)>" }
{ "type": "AUDIO", "data": "<base64(pcm chunk 2)>" }

{ "type": "EVENT", "event": "PAUSE" }

{ "type": "AUDIO", "data": "RU9G" }

What each message does

Message	Purpose
START_TIME	Marks the start of this recording segment. data is Base64 of an RFC 3339 timestamp, such as 2026-04-25T12:34:56Z.
AUDIO	Carries visit audio. data is standard Base64 of raw LINEAR16 PCM. Do not use hex, URL-safe Base64, or WAV headers.
EVENT	Sends control, not audio. Use {"type":"EVENT","event":"<VALUE>"}. The stream handler acts on PAUSE, RESUME, and CANCEL. Send KEEP_ALIVE while paused so the connection does not idle out. ABORT is deprecated and is not handled. Do not put those values in data.
RU9G	Ends this recording segment. RU9G is Base64 for ASCII EOF. Send it as the final AUDIO message.

For exact field shapes and required order, see Ambient streaming wire format.

Do not send binary WebSocket frames, multiple JSON objects in one frame, or raw audio over HTTP. If the server receives non-JSON payloads, it returns parsing errors, such as invalid character or null byte errors.

Complete the session after streaming

Ending the stream and ending the ambient session are two different steps.

RU9G is the Ambient end-of-audio marker. It is Base64 for ASCII EOF. Send it as the last AUDIO message so Suki knows this recording segment is finished and should not wait for more chunks. Closing the WebSocket then drops the live audio connection. That only stops the stream. It does not close the ambient session, and it does not start clinical note generation.

To finish the visit recording, call End ambient session with the same ambient_session_id you got from create. End is the REST step that closes the ambient session and tells Suki to generate the note.

Action	What it does
Send RU9G	Finishes this recording segment
Close the WebSocket	Stops the live audio connection
Call End	Ends the ambient session and starts note generation

After the last PCM chunk:

• Send { "type": "AUDIO", "data": "RU9G" }.
• Close the socket.
• Call End.
• Show Generating and poll status.
• When status is completed, retrieve the transcript, note content, and structured data with REST.

Final transcripts and notes are not guaranteed to arrive over WebSocket. Treat REST APIs as the source of truth.

For the full shutdown sequence, see Complete the session after streaming, Stop Ambient, and End Ambient after streaming.

If status is skipped, the note was not generated because the transcript was empty or the session was too short. Treat that as no note this time, not as a successful empty chart. Plan for about one minute of audio when you can.

Pause, keep-alive, and reconnect

/ws/stream carries live audio and control messages only. Create, End, and result retrieval stay on REST.

If the clinician pauses, send PAUSE and keep the connection alive with KEEP_ALIVE. If they resume, send RESUME on the same ambient session. If the network drops while the session is still CREATED, you can reopen /ws/stream with the same ambient_session_id. After End, that ID is no longer available for a new stream.

Situation	What to send
Audio is flowing	Send audio at least once every 25 seconds. If you send nothing, Suki disconnects the stream.
Stream is paused	Send KEEP_ALIVE at least every 5 seconds. A paused session can stay open for up to 30 minutes when keep-alives are maintained.
You need to reconnect	Reconnect with the same ambient_session_id only while status is CREATED. Otherwise the handshake returns FailedPrecondition.

For capture and idle-timeout detail, see Audio capture best practices and Streaming architecture.

Audio format

Suki expects raw speech audio, not a WAV file. Capture a single mono channel as LINEAR16 (PCM signed 16-bit little-endian) at 16 kHz. Split it into about 100 ms chunks, Base64 encode each chunk, and put that string in AUDIO data.

At 16 kHz mono 16-bit, 100 ms is about 3200 bytes of raw PCM. If your source is WAV, strip the header or decode to raw PCM before you encode. Pace chunks to match their duration so you stream at or near real time, rather than dumping a buffered recording as fast as the network allows.

Sample Rate of 16 kHz

Suki streams audio at 16 kHz, which captures the full range of clinical speech.

Mono Channel

Send a single mono channel of audio, not stereo or multi-channel.

LINEAR16 Encoding

Encode as LINEAR16 (PCM signed 16-bit little-endian). Remove WAV headers or decode to raw PCM before you send.

Audio Chunk Size of 100 ms

Suki supports 100 ms chunks to balance recognition quality, latency, and efficiency. At 16 kHz mono 16-bit, that is about 3200 bytes of raw PCM per chunk.

Stream at Real-Time Speed

Pace audio chunks to match their actual duration and stream at or near real time, rather than sending buffered audio as fast as possible.

Decide who owns the streaming client

Your application then owns the microphone or media pipeline, PCM chunking, Base64 encoding, the WebSocket client, and the session lifecycle. If you do not want to own that stack, use Web SDK or Headless Web SDK instead of this wire format.

This guide is for Ambient APIs. Use it when your product owns capture and the /ws/stream client.

If You Stream from Your Own Backend or Custom Client

Use this path when audio comes from your server, a mobile app you built, or a custom desktop client. You implement GET /ws/stream yourself: create the session, seed context, send JSON frames, then End and retrieve the note with REST.

Use the same base host for REST and WebSocket in a given environment. Your partnership team confirms which host and credentials apply. See Streaming architecture.

If Clinicians Capture and Review in the Browser

Use the headed Web SDK. Suki provides browser capture and the note review UI. Your application supplies encounter context and handles note handoff after submit.

You do not implement this Partner WebSocket wire format. See Web SDK quickstart.

If You Build Custom React Recording Controls

Use the Headless Web SDK. Your React application owns Start, Pause, Stop, status, and review UI through SDK hooks. The Headless Web SDK owns upload and session lifecycle.

Do not send /ws/stream JSON frames from that React app. See Headless Web SDK quickstart.

If You Need Live Transcript Text in the App

Use Stream Dictation audio on GET /ws/transcribe. Dictation returns partial and final transcript text during the stream.

Ambient streaming is for visit audio into note generation or Form filling. Those results come mainly from REST after End, not as live transcript frames on /ws/stream.

Common streaming mistakes to avoid

Use this table to troubleshoot common ambient streaming mistakes before you ship. For the full Ambient troubleshooting table, see Complete the session after streaming.

Problem	Why it happens	How to fix it
Server returns JSON parse errors (for example, invalid character or null byte)	The WebSocket received a binary frame, a non-JSON payload, or multiple JSON objects in a single frame.	Send one UTF-8 JSON object per WebSocket text frame. Do not send binary frames or concatenate multiple JSON objects.
Audio is ignored or transcription is incorrect	Audio was not encoded as expected. Common causes include hexadecimal encoding, URL-safe Base64, or sending WAV headers instead of raw PCM audio.	Encode raw LINEAR16 PCM audio using standard Base64 before sending it in the data field.
The session never completes	The server never received the end-of-stream marker.	End the stream by sending a final AUDIO message with "data": "RU9G".
Control messages are ignored	A control action was sent in the data field instead of as an event message.	Send control actions as {"type":"EVENT","event":"<VALUE>"}. Do not place control values in data.
No final note or transcript is available	Closing the WebSocket does not end the ambient session or trigger note generation.	After streaming completes, call the ambient REST API to end the session, then retrieve the generated note and transcript.
The WebSocket disconnects during a long pause	No messages were sent before the keep-alive timeout expired.	While audio is paused, send a KEEP_ALIVE event at least every 5 seconds.
Unable to reconnect after a disconnect	The session has already moved beyond the CREATED state.	Reconnect to /ws/stream using the same ambient_session_id only while the session status is CREATED.

Available cookbooks

AmbientAPI

End Ambient After Streaming

Send RU9G, then end session.

5 min

AmbientAPI

Authenticate Browser WebSocket Handshake

Auth browser WebSocket with protocols.

5 min

Available tutorials

Ambient

Build an Ambient Streaming Client

Authenticate, create a session, stream PCM audio over WebSocket, and retrieve clinical note results.

20 minIntermediate

Next steps

Ambient streaming wire format - JSON frames, START_TIME, and RU9G.

Complete the session after streaming - End, poll status, and retrieve results.

Build an ambient streaming client - End-to-end login, stream, and retrieve.

Stream ambient audio in your product - Recorder Start, Pause, and Stop.