Create a realtime conversation

Connection and authentication

Upgrade an HTTP GET request to WebSocket. Connect to wss://api.mosi.cn/v1/realtime with the model query parameter specifying the model; place the API Key only in the Authorization request header.

FieldTypeRequiredDescription

model: query stringRequired

Model ID. Always moss-live-1.0, passed in the model query parameter of the WebSocket URL.

user: query stringOptional

Your own stable end-user ID, URL-encoded. Required when memory_scope=user; use the same user when reconnecting. Example: wss://api.mosi.cn/v1/realtime?model=moss-live-1.0&user=customer_001.

Authorization: HTTP headerRequired

Bearer <API Key>; put it in the Authorization request header of the WebSocket handshake.

Session flow

Stage	Client	Server events and checks
Connect	Complete the WebSocket handshake	Join the session after the connection opens
Configure	Send attach_session, then session.update	Wait for session_attached and then session.updated, and confirm data.error is an empty string
Send audio and video	Enable the microphone; for frames, enable media.mode and video.state	The same connection receives audio and JPEG frames
Answer	Display text, play voice, and send playback acknowledgements	Receive input.transcript.done, output.transcript.done, and response.ended
Continue or end	Keep sending input, or close the WebSocket	The connection can continue after a turn completes; closing the connection ends the session

Client events

FieldTypeDescription

attach_session: event

Joins the session; billing by online duration starts once session_attached is received.

session.update: event

Sets the persona and business rules; after receiving session.updated, check data.error. A non-empty value means the configuration did not take effect.

mic_state: event

Turns microphone input on or off.

media.mode: event

Enables tagged-v1 before video or screen sharing.

Media	Upstream binary format
Audio without tagged-v1	Raw PCM S16LE; 640 bytes per 20 ms, no WAV header.
Audio with tagged-v1	0x00 + raw PCM bytes; 641 bytes per 20 ms. The prefix stays after frames are turned off.
Camera	0x01 + raw JPEG bytes; one image per message.
Screen sharing	0x02 + raw JPEG bytes; one image per message.

video.state: event

Controls camera or screen frames; turning frames off does not end the session.

playback.chunk.completed: event

Sent after each audio chunk has actually finished playing; put the fields at the top level of the message, not inside data. Acknowledge only audio that has actually finished playing, and do not wait for the answer text to finish before acknowledging.

playback.completed: event

Sent after receiving response.audio.done and finishing playback of all audio in the turn.

video.stream_focus: event

Sets the primary frame source to focus on.

playback.stopped: event

Sent after stopping the corresponding playback on response.audio.stop.

Server events

FieldTypeDescription

session_attached: event

The session has been joined; configure the persona and business rules after receiving it.

session.updated: event

Configuration result; enable media and the microphone when data.error is an empty string.

input.transcript.updated: event

User transcript update; data.text is the current full text, so replace the displayed text instead of appending each update.

input.transcript.done: event

User transcript completed.

output.transcript.updated: event

Model answer text update; data.text is the current full text, so replace the displayed text instead of appending each update.

output.transcript.done: event

Model answer text completed.

response.interrupted: event

The answer was interrupted; mark the matching answer as interrupted by run_id and turn_id.

response.audio.context: event

Save the audio context and pair it in order with the following binary audio.

response.audio.done: event

All audio for this turn has been sent; send playback.completed after playback has actually finished.

response.ended: event

Status notification for this turn; it does not mean the session has ended.

error: event

Error event; record the cause and release the connection and media resources.

response.audio.stop: event

Stop the corresponding voice playback and send playback.stopped.

Protocol limits

Item	Current protocol value and description
Audio	16 kHz upstream and 48 kHz downstream, mono PCM S16LE; no WAV header. Keep sending silence after the user stops speaking.
Media framing	tagged-v1: audio 0x00, camera JPEG 0x01, screen JPEG 0x02; audio keeps the prefix after frames are turned off.
Frames	One binary message per JPEG; no Base64, and complete MP4 files or H.264 streams are not accepted.
Transport	Audio and frames share the same WebSocket.
Frame processing	Under load, the latest frame may be kept and pending older frames dropped; per-frame analysis is not guaranteed. 1 FPS, JPEG quality 0.8, a 640 px long edge for the camera, and 1024 px for the screen are client defaults, not hard limits.

Error codes

For authentication, permissions, rate limiting, and general HTTP errors during connection, see Error Codes. The following are protocol errors specific to the MOSS-LIVE endpoint.

Error codes	Trigger	Action
memory_scope_requires_user	The user memory scope lacks an end-user identity	Pass a stable user, or use the session scope.
memory_scope_immutable	Creation options changed on reconnect	Reconnect with the options used at creation; create a new session to change them.
invalid_memory_scope	Invalid parameter value	Check the enum values.

Next steps

Models

View MOSS-LIVE model details

See the model ID, modality support, and realtime interaction capabilities.

Model overview

View the model list

Compare model capabilities and choose the right model for your scenario.

Request example

import asyncio, json, os
import websockets

URL = "wss://api.mosi.cn/v1/realtime?model=moss-live-1.0"
HEADERS = {
    "Authorization": "Bearer " + os.environ["MOSS_API_KEY"],
}

async def main():
    async with websockets.connect(URL, additional_headers=HEADERS) as ws:
        await ws.send(json.dumps({"action": "attach_session"}))
        async for packet in ws:
            if isinstance(packet, bytes):
                # Pair with response.audio.context, play as 48 kHz PCM S16LE, and acknowledge.
                continue
            msg = json.loads(packet)
            data = msg.get("data") or {}
            action = msg.get("action")
            if action == "session_attached":
                await ws.send(json.dumps({"action": "session.update", "session": {
                    "persona": {"name": "Ann", "prompt": "You are Ann, a friendly assistant."},
                    "instructions": "Reply with one short sentence in Chinese."}}))
            elif action == "session.updated":
                if data.get("error"):
                    raise RuntimeError(str(msg))
                await ws.send(json.dumps({"action": "media.mode",
                    "binary_protocol": "tagged-v1"}))
                await ws.send(json.dumps({"action": "video.state",
                    "source": "camera", "enabled": True}))
                await ws.send(json.dumps({"action": "mic_state", "enabled": True}))
                # Upload from your realtime capture task:
                # await ws.send(b"\x00" + pcm_bytes)   # 16 kHz PCM
                # await ws.send(b"\x01" + jpeg_bytes)  # camera
                # await ws.send(b"\x02" + jpeg_bytes)  # screen
            elif action == "error":
                raise RuntimeError(str(msg))
            else:
                print(action, data)
# Connection and control example; add realtime capture, a playback queue, and acknowledgements.
asyncio.run(main())

import WebSocket from 'ws';
const ws = new WebSocket('wss://api.mosi.cn/v1/realtime?model=moss-live-1.0', {
  headers: { Authorization: `Bearer ${process.env.MOSS_API_KEY}` }
});
const send = (value: object) => ws.send(JSON.stringify(value));
ws.on('open', () => send({ action: 'attach_session' }));
ws.on('message', (packet, isBinary) => {
  if (isBinary) {
    // Pair with response.audio.context, play 48 kHz PCM, and send acknowledgements.
    return;
  }
  const msg = JSON.parse(packet.toString());
  const data = msg.data || {};
  if (msg.action === 'session_attached') {
    send({ action: 'session.update', session: {
      persona: { name: 'Ann', prompt: 'You are Ann, a friendly assistant.' },
      instructions: 'Reply with one short sentence in Chinese.' } });
  } else if (msg.action === 'session.updated') {
    if (data.error) { console.error(msg); ws.close(); return; }
    send({ action: 'media.mode', binary_protocol: 'tagged-v1' });
    send({ action: 'video.state', source: 'camera', enabled: true });
    send({ action: 'mic_state', enabled: true });
  } else if (msg.action === 'error') { console.error(msg); ws.close(); }
});
// Once the session is ready, send from your realtime capture task; pcmBytes / jpegBytes are captured data:
function sendAudio(pcmBytes: Buffer) {
  ws.send(Buffer.concat([Buffer.from([0x00]), pcmBytes]));
}
function sendCamera(jpegBytes: Buffer) {
  ws.send(Buffer.concat([Buffer.from([0x01]), jpegBytes]));
}
function sendScreen(jpegBytes: Buffer) {
  ws.send(Buffer.concat([Buffer.from([0x02]), jpegBytes]));
}
// Keep sending all-zero PCM after the user stops speaking. Start testing frames at 1 FPS.
// Add capture, playback, and acknowledgements; call ws.close() when testing is done.

Response example

// The following are WebSocket events received independently.
{"action":"session_attached","data":{"session_id":"<session_id>","memory_scope":"session","delegation":"none"}}
{"action":"session.updated","data":{"error":""}}
{"action":"input.transcript.done","data":{"text":"What is in the frame?"}}
{"action":"output.transcript.done","data":{"text":"There is a table in the frame.","run_id":"<run_id>","turn_id":"<turn_id>"}}
{"action":"response.audio.context","data":{"chunk_id":"<chunk_id>","run_id":"<run_id>","turn_id":"<turn_id>"}}
// <48 kHz mono PCM S16LE binary; pair it with the context, play it, and acknowledge>
{"action":"response.audio.done","data":{"run_id":"<run_id>","turn_id":"<turn_id>"}}
{"action":"response.ended","data":{"status":"completed","run_id":"<run_id>","turn_id":"<turn_id>"}}