Audio Capture & Streaming

1. Precise audio streaming format: Mentioned on above?

LINEAR16 (16KHz sampling rate) over the mono channel. Audio should be chunked into 100ms packets for optimal performance. We only support 100ms audio chunks, this is to achieve right balance between quality, latency and efficiency.

2. Bandwidth & latency strategies?

Network connection and speed and consistency is important for Suki to perform well.

Suki requires:
1Mbps Upload speed
Bitrate 768kbps
150ms Ping time
Unloaded latency <50ms
Loaded latency <150ms

3. What is the streaming protocol?

Client should set up a WebSocket Secure (wss://) request with suki endpoint stream

4. What's Audio Streaming format?

Send LINEAR16 (16KHz sampling rate) audio over the mono channel. Audio should be chunked into 100ms packets for optimal performance.

It supports two types of messages:
a. AUDIO carrying raw audio data in supported format.
b. EVENT signifying event on the stream.

Supported events are:
i. PAUSE: pause the stream
ii. RESUME: resume the stream
iii. CANCEL: user cancels the stream
iv. ABORT: stream is aborted (interruption)
v. KEEP_ALIVE: pings to sustain the stream during inactivity such as during paused state, it should be every 15 seconds
vi. EOF: indicates closure of stream

{
    "type": "AUDIO",
    "data": "Base64EncodedAudioBytes"
}
{
    "type": "AUDIO",
    "data": "EOF"  // This is not string but bytes
}
{
    "type": "EVENT",
    "data": "PAUSE"
}
{
    "type": "EVENT",
    "data": "RESUME"
}
{
    "type": "EVENT",
    "data": "CANCEL"
}
{
    "type": "EVENT",
    "data": "KEEP_ALIVE"
}

Assistant

Responses are generated using AI and may contain mistakes.