> ## Documentation Index
> Fetch the complete documentation index at: https://developer.suki.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Ambient Audio Streaming - Overview

> Stream Ambient or Form filling visit audio on GET /ws/stream after session creation: PCM chunking, JSON control frames, REST completion, and result retrieval

<div className="quick-summary-wrapper">
  <div className="quick-summary-header">
    <span className="quick-summary-icon" aria-hidden="true" />

    <span className="quick-summary-title">Quick summary</span>
  </div>

  <div className="quick-summary-content">
    Streaming sends live visit audio to Suki over WebSocket after you create a session and seed context. The WebSocket carries JSON text frames for audio and control messages. Final notes, form fields, transcripts, and structured data come from the matching product REST APIs after you end the session.
  </div>

  <div className="quick-summary-footer">
    <span className="quick-summary-footer-icon" aria-hidden="true" />

    <span className="quick-summary-footer-text">Last updated:</span>
    <span className="quick-summary-footer-date">August 2026</span>
  </div>
</div>

Ambient streaming lets your product send a visit conversation to Suki while the clinician is still in the room. First, create the ambient session and seed session context. Then open the `GET /ws/stream` WebSocket and send the visit audio in small chunks. When you stop streaming and end the session, Suki uses that audio to generate the clinical note.

The Ambient REST APIs handle session creation, session completion, status, and note retrieval. They do not carry live visit audio. The WebSocket handles the live audio stream. After you end the session, use the Ambient REST APIs to retrieve the note and other supported results.

### What you can do with ambient streaming

Use the Ambient and Streaming APIs together to:

* Capture audio from your own backend, mobile app, or custom client.
* Stream visit audio to Suki in small chunks while the encounter is in progress.
* Pause and resume the stream, or keep a paused stream alive, by sending `EVENT` messages.
* Complete the stream and retrieve the resulting data through the Ambient REST APIs.

### WebSocket and REST responsibilities

| Interface         | Purpose                                                                                                |
| :---------------- | :----------------------------------------------------------------------------------------------------- |
| `GET /ws/stream`  | Sends live visit audio and stream control messages to Suki                                             |
| Ambient REST APIs | Create and end the session, check processing status, and retrieve the note and other supported results |

<Note>
  The WebSocket is the **live audio path**. The REST APIs are the **session and results path**.
</Note>

This guide focuses on `GET /ws/stream` for Ambient and Form filling. For real-time transcript text in your application, see [Stream Dictation audio](/documentation/how-to/audio-streaming/dictation-streaming). For recorder Start, Pause, and Stop, see [Stream ambient audio in your product](/documentation/how-to/ambient-clinical-notes/stream-ambient-audio-in-your-product).

<Info>
  These patterns apply when you build your own streaming client with the **Ambient APIs**. The headed **Web SDK** already captures and streams visit audio. The **Headless Web SDK** uses React hooks instead of this Partner WebSocket wire format.
</Info>

**Streaming rules (agents):**

* Create the ambient session before opening `/ws/stream`. The session must still be `CREATED`, or the handshake returns `FailedPrecondition`.
* Seed context before you stream so note generation has encounter details. Missing context does not by itself fail the WebSocket handshake.
* Form filling uses the same Partner WebSocket and ambient message protocol. Use the Form filling session ID on Form filling REST (including Form filling End) and `/ws/stream`. Do not call Ambient End on a Form filling session.
* Send UTF-8 JSON text frames only. One JSON object per frame. Do not send binary audio frames.
* Required order per stream segment: `START_TIME` → `AUDIO` PCM chunks → optional `EVENT` → final `AUDIO` with `data` `RU9G` (Base64 of ASCII `EOF`).
* Closing the WebSocket is not enough. After `RU9G`, close the socket, call End, then poll status. REST is the source of truth for notes and transcripts.
* Capture mono LINEAR16 PCM at 16 kHz. Stream about 100 ms chunks (about 3200 bytes of raw PCM). Strip WAV headers. Stream at or near real time.
* The stream handler acts on `PAUSE`, `RESUME`, and `CANCEL`. `KEEP_ALIVE` keeps the connection alive while paused. `ABORT` is deprecated and is not handled.
* While audio is flowing, send audio at least every **25 seconds**. While paused, send `KEEP_ALIVE` at least every **5 seconds**. Maximum pause is **30 minutes** when keep-alives are maintained.
* Reconnect with the same `ambient_session_id` only while status is `CREATED`.
* Plan for about **one minute** of audio to avoid `skipped`.

## Enable streaming

```mermaid actions={false} theme={"theme":{"light":"github-dark","dark":"material-theme-darker"}}
flowchart LR
    A[Create ambient session<br/>and seed context] --> B[Open GET /ws/stream]
    B --> C[START_TIME then<br/>AUDIO chunks]
    C --> D[Optional EVENT<br/>controls]
    D --> E[RU9G end marker<br/>and close socket]
    E --> F[End session and<br/>retrieve results with REST]

    style A fill:#FFF394,stroke:#333,color:#000
    style B fill:#FFF394,stroke:#333,color:#000
    style C fill:#FFF394,stroke:#333,color:#000
    style D fill:#FFF394,stroke:#333,color:#000
    style E fill:#FFF394,stroke:#333,color:#000
    style F fill:#FFF394,stroke:#333,color:#000
```

<Note>
  Sessions shorter than **1 minute** may not contain enough audio for note generation and can be marked as **`skipped`**.
</Note>

<Steps>
  <Step title="Create the Ambient Session">
    Call [Create ambient session](/api-reference/ambient-sessions/create) before you open the socket. Store `ambient_session_id` from create. The WebSocket handshake needs that ID, and the session must still be **`CREATED`**.

    Opening `/ws/stream` before the session exists, or after the session leaves **`CREATED`**, returns `FailedPrecondition`.
  </Step>

  <Step title="Seed Session Context">
    Call [Seed session context](/api-reference/ambient-sessions/context) so note generation has encounter details. Context is not what authenticates the socket, but you should send it before you stream.
  </Step>

  <Step title="Open the Socket and Stream Audio">
    After create and context succeed, open `wss://sdp.suki.ai/ws/stream`. Send one `START_TIME` message to mark the start of this recording segment, then one `AUDIO` message per PCM chunk. When the clinician taps Stop, send `RU9G` as the last `AUDIO` message and close the socket.
  </Step>
</Steps>

<div className="doc-guide-btn-row">
  <a href="/api-reference/ambient-sessions/audio-stream" className="doc-guide-btn">
    Audio Streaming API
  </a>

  <a href="/api-reference/ambient-sessions/create" className="doc-guide-btn">
    Create Ambient Session API
  </a>
</div>

The following code samples show how to loop through chunks:

**Language tabs (agents):** Equivalent code samples are available in: TypeScript, Python. Humans see one language at a time. Use the variant that matches the user's stack; behavior is the same across tabs. TypeScript uses browser `Sec-WebSocket-Protocol` auth. Python uses HTTP headers on the WebSocket upgrade.

<Tabs>
  <Tab title="TypeScript">
    ```typescript js theme={"theme":{"light":"github-dark","dark":"material-theme-darker"}}
    const ws = new WebSocket("wss://sdp.suki.ai/ws/stream", [
      `SukiAmbientAuth,${sdpSukiToken},${ambientSessionId}`,
    ]);

    ws.onopen = () => {
      const startTime = new Date().toISOString().replace(/\.\d{3}Z$/, "Z");
      ws.send(
        JSON.stringify({
          type: "START_TIME",
          data: btoa(startTime),
        })
      );

      // Send ~100 ms LINEAR16 PCM chunks as standard Base64.
      for (const pcmChunkBase64 of pcmChunks) {
        ws.send(JSON.stringify({ type: "AUDIO", data: pcmChunkBase64 }));
      }

      ws.send(JSON.stringify({ type: "AUDIO", data: "RU9G" }));
      ws.close();
    };
    ```
  </Tab>

  <Tab title="Python">
    ```python python theme={"theme":{"light":"github-dark","dark":"material-theme-darker"}}
    import base64
    import json
    from datetime import datetime, timezone

    import websocket

    ws = websocket.create_connection(
        "wss://sdp.suki.ai/ws/stream",
        header=[
            f"sdp_suki_token: {sdp_suki_token}",
            f"ambient_session_id: {ambient_session_id}",
        ],
    )

    rfc3339 = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
    ws.send(
        json.dumps(
            {
                "type": "START_TIME",
                "data": base64.b64encode(rfc3339.encode("utf-8")).decode("ascii"),
            }
        )
    )

    # Send ~100 ms LINEAR16 PCM chunks as standard Base64.
    for pcm_chunk in pcm_chunks:
        ws.send(
            json.dumps(
                {
                    "type": "AUDIO",
                    "data": base64.b64encode(pcm_chunk).decode("ascii"),
                }
            )
        )

    ws.send(json.dumps({"type": "AUDIO", "data": "RU9G"}))
    ws.close()
    ```
  </Tab>
</Tabs>

The TypeScript sample authenticates a browser client. The Python sample authenticates a non-browser client.

| Client      | How you authenticate                                                                                                             |
| :---------- | :------------------------------------------------------------------------------------------------------------------------------- |
| Browser     | Pass `Sec-WebSocket-Protocol` as `SukiAmbientAuth,<sdp_suki_token>,<ambient_session_id>`. The token comes before the session ID. |
| Non-browser | Send `sdp_suki_token` and `ambient_session_id` as HTTP headers on the WebSocket upgrade. Do not use `Sec-WebSocket-Protocol`.    |

For the browser handshake recipe, see [Authenticate browser WebSocket handshake](/documentation/cookbooks/browser-websocket-auth).

<Tip>
  If you do not have credentials yet, complete [Partner onboarding](/documentation/get-started/partner-onboarding) and [Partner authentication](/documentation/how-to/partner-authentication) to get an `sdp_suki_token`. Staging examples use `wss://sdp.suki-stage.com/ws/stream`.
</Tip>

<Note>
  Form filling uses the same Partner WebSocket and the same ambient message protocol. Use only the ID from [Create Form filling session](/form-filling-api-reference/form-filling-sessions/create) on Form filling REST, including [End Form filling session](/form-filling-api-reference/form-filling-sessions/end), and on `/ws/stream`. That ID is not an Ambient clinical-note `ambient_session_id`. Ambient End rejects Form filling jobs.
</Note>

## Send JSON text frames

`/ws/stream` does not accept raw PCM as a binary WebSocket frame. Each message is a UTF-8 JSON text frame that contains exactly one JSON object. Base64 encode the audio bytes, then put that string in the JSON `data` field.

A stream segment is one recording pass on the socket: start, audio, optional controls, then end-of-audio. Send messages in this order:

```json theme={"theme":{"light":"github-dark","dark":"material-theme-darker"}}
{ "type": "START_TIME", "data": "<base64(timestamp)>" }

{ "type": "AUDIO", "data": "<base64(pcm chunk 1)>" }
{ "type": "AUDIO", "data": "<base64(pcm chunk 2)>" }

{ "type": "EVENT", "event": "PAUSE" }

{ "type": "AUDIO", "data": "RU9G" }
```

### What each message does

| Message      | Purpose                                                                                                                                                                                                                                                                          |
| :----------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `START_TIME` | Marks the start of this recording segment. `data` is Base64 of an RFC 3339 timestamp, such as `2026-04-25T12:34:56Z`.                                                                                                                                                            |
| `AUDIO`      | Carries visit audio. `data` is standard Base64 of raw LINEAR16 PCM. Do not use hex, URL-safe Base64, or WAV headers.                                                                                                                                                             |
| `EVENT`      | Sends control, not audio. Use `{"type":"EVENT","event":"<VALUE>"}`. The stream handler acts on `PAUSE`, `RESUME`, and `CANCEL`. Send `KEEP_ALIVE` while paused so the connection does not idle out. `ABORT` is deprecated and is not handled. Do not put those values in `data`. |
| `RU9G`       | Ends this recording segment. `RU9G` is Base64 for ASCII `EOF`. Send it as the final `AUDIO` message.                                                                                                                                                                             |

For exact field shapes and required order, see [Ambient streaming wire format](/documentation/how-to/audio-streaming/websocket-streaming-wire-format-ambient).

<Warning>
  Do not send binary WebSocket frames, multiple JSON objects in one frame, or raw audio over HTTP. If the server receives non-JSON payloads, it returns parsing errors, such as invalid character or null byte errors.
</Warning>

## Complete the session after streaming

Ending the stream and ending the ambient session are two different steps.

`RU9G` is the Ambient end-of-audio marker. It is Base64 for ASCII `EOF`. Send it as the last `AUDIO` message so Suki knows this recording segment is finished and should not wait for more chunks.
Closing the WebSocket then drops the live audio connection. That only stops the stream. It does not close the ambient session, and it does not start clinical note generation.

To finish the visit recording, call [End ambient session](/api-reference/ambient-sessions/end) with the same `ambient_session_id` you got from create. End is the REST step that closes the ambient session and tells Suki to generate the note.

| Action              | What it does                                        |
| :------------------ | :-------------------------------------------------- |
| Send `RU9G`         | Finishes this recording segment                     |
| Close the WebSocket | Stops the live audio connection                     |
| Call End            | Ends the ambient session and starts note generation |

After the last PCM chunk:

* Send `{ "type": "AUDIO", "data": "RU9G" }`.
* Close the socket.
* Call End.
* Show Generating and poll status.
* When status is **`completed`**, retrieve the transcript, note content, and structured data with REST.

<Note>
  Final transcripts and notes are not guaranteed to arrive over WebSocket. Treat REST APIs as the source of truth.
</Note>

For the full shutdown sequence, see [Complete the session after streaming](/documentation/how-to/audio-streaming/websocket-streaming-complete-session), [Stop Ambient](/documentation/how-to/ambient-clinical-notes/end-ambient-session), and [End Ambient after streaming](/documentation/cookbooks/end-ambient-after-streaming).

<Warning>
  If status is **`skipped`**, the note was not generated because the transcript was empty or the session was too short. Treat that as no note this time, not as a successful empty chart. Plan for about **one minute** of audio when you can.
</Warning>

## Pause, keep-alive, and reconnect

`/ws/stream` carries live audio and control messages only. Create, End, and result retrieval stay on REST.

If the clinician pauses, send `PAUSE` and keep the connection alive with `KEEP_ALIVE`. If they resume, send `RESUME` on the same ambient session. If the network drops while the session is still **`CREATED`**, you can reopen `/ws/stream` with the same `ambient_session_id`. After End, that ID is no longer available for a new stream.

| Situation             | What to send                                                                                                                             |
| :-------------------- | :--------------------------------------------------------------------------------------------------------------------------------------- |
| Audio is flowing      | Send audio at least once every **25 seconds**. If you send nothing, Suki disconnects the stream.                                         |
| Stream is paused      | Send `KEEP_ALIVE` at least every **5 seconds**. A paused session can stay open for up to **30 minutes** when keep-alives are maintained. |
| You need to reconnect | Reconnect with the same `ambient_session_id` only while status is **`CREATED`**. Otherwise the handshake returns `FailedPrecondition`.   |

For capture and idle-timeout detail, see [Audio capture best practices](/documentation/how-to/audio-streaming/audio-capture-best-practices) and [Streaming architecture](/documentation/how-to/audio-streaming/streaming-architecture#network-and-connection-management).

## Audio format

Suki expects raw speech audio, not a WAV file. Capture a single mono channel as **LINEAR16** (PCM signed 16-bit little-endian) at **16 kHz**. Split it into about **100 ms** chunks, Base64 encode each chunk, and put that string in `AUDIO` `data`.

At 16 kHz mono 16-bit, 100 ms is about **3200 bytes** of raw PCM. If your source is WAV, strip the header or decode to raw PCM before you encode. Pace chunks to match their duration so you stream at or near real time, rather than dumping a buffered recording as fast as the network allows.

<CardGroup cols={2}>
  <Card title="Sample Rate of 16 kHz" icon="waveform">
    Suki streams audio at **16 kHz**, which captures the full range of clinical speech.
  </Card>

  <Card title="Mono Channel" icon="microphone">
    Send a **single mono channel** of audio, not stereo or multi-channel.
  </Card>

  <Card title="LINEAR16 Encoding" icon="file-audio">
    Encode as **LINEAR16** (PCM signed 16-bit little-endian). Remove WAV headers or decode to raw PCM before you send.
  </Card>

  <Card title="Audio Chunk Size of 100 ms" icon="clock">
    Suki supports **100 ms** chunks to balance recognition quality, latency, and efficiency. At 16 kHz mono 16-bit, that is about **3200 bytes** of raw PCM per chunk.
  </Card>
</CardGroup>

<Card title="Stream at Real-Time Speed" icon="gauge">
  Pace audio chunks to match their actual duration and stream at or near real time, rather than sending buffered audio as fast as possible.
</Card>

## Decide who owns the streaming client

Your application then owns the microphone or media pipeline, PCM chunking, Base64 encoding, the WebSocket client, and the session lifecycle. If you do not want to own that stack, use Web SDK or Headless Web SDK instead of this wire format.

<Info>
  This guide is for **Ambient APIs**. Use it when your product owns capture and the `/ws/stream` client.
</Info>

<AccordionGroup>
  <Accordion title="If You Stream from Your Own Backend or Custom Client" icon="code">
    Use this path when audio comes from your server, a mobile app you built, or a custom desktop client. You implement `GET /ws/stream` yourself: create the session, seed context, send JSON frames, then End and retrieve the note with REST.

    Use the same base host for REST and WebSocket in a given environment. Your partnership team confirms which host and credentials apply. See [Streaming architecture](/documentation/how-to/audio-streaming/streaming-architecture).
  </Accordion>

  <Accordion title="If Clinicians Capture and Review in the Browser" icon="window">
    Use the headed **Web SDK**. Suki provides browser capture and the note review UI. Your application supplies encounter context and handles note handoff after submit.

    You do not implement this Partner WebSocket wire format. See [Web SDK quickstart](/web-sdk/quickstart).
  </Accordion>

  <Accordion title="If You Build Custom React Recording Controls" icon="react">
    Use the **Headless Web SDK**. Your React application owns Start, Pause, Stop, status, and review UI through SDK hooks. The Headless Web SDK owns upload and session lifecycle.

    Do not send `/ws/stream` JSON frames from that React app. See [Headless Web SDK quickstart](/headless-web-sdk/quickstart).
  </Accordion>

  <Accordion title="If You Need Live Transcript Text in the App" icon="microphone">
    Use [Stream Dictation audio](/documentation/how-to/audio-streaming/dictation-streaming) on `GET /ws/transcribe`. Dictation returns partial and final transcript text during the stream.

    Ambient streaming is for visit audio into note generation or Form filling. Those results come mainly from REST after End, not as live transcript frames on `/ws/stream`.
  </Accordion>
</AccordionGroup>

## Common streaming mistakes to avoid

Use this table to troubleshoot common ambient streaming mistakes before you ship. For the full Ambient troubleshooting table, see [Complete the session after streaming](/documentation/how-to/audio-streaming/websocket-streaming-complete-session).

| Problem                                                                        | Why it happens                                                                                                                                   | How to fix it                                                                                                             |
| :----------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------ |
| Server returns JSON parse errors (for example, invalid character or null byte) | The WebSocket received a binary frame, a non-JSON payload, or multiple JSON objects in a single frame.                                           | Send one UTF-8 JSON object per WebSocket text frame. Do not send binary frames or concatenate multiple JSON objects.      |
| Audio is ignored or transcription is incorrect                                 | Audio was not encoded as expected. Common causes include hexadecimal encoding, URL-safe Base64, or sending WAV headers instead of raw PCM audio. | Encode raw LINEAR16 PCM audio using standard Base64 before sending it in the `data` field.                                |
| The session never completes                                                    | The server never received the end-of-stream marker.                                                                                              | End the stream by sending a final `AUDIO` message with `"data": "RU9G"`.                                                  |
| Control messages are ignored                                                   | A control action was sent in the `data` field instead of as an event message.                                                                    | Send control actions as `{"type":"EVENT","event":"<VALUE>"}`. Do not place control values in `data`.                      |
| No final note or transcript is available                                       | Closing the WebSocket does not end the ambient session or trigger note generation.                                                               | After streaming completes, call the ambient REST API to end the session, then retrieve the generated note and transcript. |
| The WebSocket disconnects during a long pause                                  | No messages were sent before the keep-alive timeout expired.                                                                                     | While audio is paused, send a `KEEP_ALIVE` event at least every **5 seconds**.                                            |
| Unable to reconnect after a disconnect                                         | The session has already moved beyond the **`CREATED`** state.                                                                                    | Reconnect to `/ws/stream` using the same `ambient_session_id` only while the session status is **`CREATED`**.             |

## Available cookbooks

<div className="hp-io-method-grid tut-hub-card-grid" data-cookbook-related-grid>
  <a className="hp-io-method-card tut-hub-method-card" href="/documentation/cookbooks/end-ambient-after-streaming">
    <div className="tut-hub-card-media" aria-hidden="true" />

    <div className="hp-io-method-card-body">
      <div className="tut-hub-card-badges">
        <span className="hp-wn-badge hp-wn-badge-new">Ambient</span>
        <span className="hp-wn-badge cookbook-hub-badge-surface cookbook-hub-badge-surface--api">API</span>
      </div>

      <h3 className="hp-io-method-card-title">End Ambient After Streaming</h3>

      <p className="hp-io-method-card-desc cookbook-hub-card-desc">
        Send RU9G, then end session.
      </p>

      <div className="hp-io-method-card-meta tut-hub-card-foot" aria-label="5 min">
        <div className="tut-hub-card-foot-meta">
          <span className="hp-io-method-card-meta-time">5 min</span>
        </div>
      </div>
    </div>
  </a>

  <a className="hp-io-method-card tut-hub-method-card" href="/documentation/cookbooks/browser-websocket-auth">
    <div className="tut-hub-card-media tut-hub-card-media--blue" aria-hidden="true" />

    <div className="hp-io-method-card-body">
      <div className="tut-hub-card-badges">
        <span className="hp-wn-badge hp-wn-badge-new">Ambient</span>
        <span className="hp-wn-badge cookbook-hub-badge-surface cookbook-hub-badge-surface--api">API</span>
      </div>

      <h3 className="hp-io-method-card-title">Authenticate Browser WebSocket Handshake</h3>

      <p className="hp-io-method-card-desc cookbook-hub-card-desc">
        Auth browser WebSocket with protocols.
      </p>

      <div className="hp-io-method-card-meta tut-hub-card-foot" aria-label="5 min">
        <div className="tut-hub-card-foot-meta">
          <span className="hp-io-method-card-meta-time">5 min</span>
        </div>
      </div>
    </div>
  </a>
</div>

## Available tutorials

<div className="hp-io-method-grid tut-hub-card-grid">
  <a className="hp-io-method-card tut-hub-method-card" href="/documentation/tutorials/ambient-websocket-code-example">
    <div className="tut-hub-card-media" aria-hidden="true" />

    <div className="hp-io-method-card-body">
      <span className="hp-wn-badge hp-wn-badge-new">Ambient</span>
      <h3 className="hp-io-method-card-title">Build an Ambient Streaming Client</h3>

      <p className="hp-io-method-card-desc">
        Authenticate, create a session, stream PCM audio over WebSocket, and retrieve clinical note results.
      </p>

      <div className="hp-io-method-card-meta tut-hub-card-foot" aria-label="20 min, Intermediate">
        <div className="tut-hub-card-foot-meta">
          <span className="hp-io-method-card-meta-time">20 min</span>
          <span className="tut-hub-level">Intermediate</span>
        </div>
      </div>
    </div>
  </a>
</div>

## Next steps

<Icon icon="file-lines" iconType="solid" /> **[Ambient streaming wire format](/documentation/how-to/audio-streaming/websocket-streaming-wire-format-ambient)** - JSON frames, `START_TIME`, and `RU9G`.

<Icon icon="file-lines" iconType="solid" /> **[Complete the session after streaming](/documentation/how-to/audio-streaming/websocket-streaming-complete-session)** - End, poll status, and retrieve results.

<Icon icon="file-lines" iconType="solid" /> **[Build an ambient streaming client](/documentation/tutorials/ambient-websocket-code-example)** - End-to-end login, stream, and retrieve.

<Icon icon="file-lines" iconType="solid" /> **[Stream ambient audio in your product](/documentation/how-to/ambient-clinical-notes/stream-ambient-audio-in-your-product)** - Recorder Start, Pause, and Stop.