Real-Time Audio and Video

This page explains how a device sends audio and video by stream_id, and how the client binds the same stream IDs to audio and video outputs after a connection is established.

If you have not started the device or connected the client yet, read Device Integration, Client Integration, and Connection first.

Agree on stream_id

stream_id identifies one media stream inside a connection.

• The public range is 0 to 15.
• Audio and video must not use the same stream_id in the same connection.
• The media type tells TiRTC whether a frame is audio or video; stream_id only identifies the stream.

If you start the local device example with the CLI tool, it uses these defaults:

Purpose	stream_id
Audio	10
Video	11

The examples below use the same values. You can choose your own values as long as device sending and client binding stay aligned.

Client Playback

On the client, create the connection and the audio/video outputs, bind each output to the agreed stream_id, then connect.

final TiRtcConn conn = TiRtcConn();
final TiRtcAudioOutput audioOutput = TiRtcAudioOutput();
final TiRtcVideoOutput videoOutput = TiRtcVideoOutput();

Widget buildVideoView() => videoOutput.view();

void startPlayback({
  required String remoteId,
  required String token,
}) {
  conn.onStateChanged = (TiRtcConnState state, int errorCode) {
    debugPrint('conn state=$state error=$errorCode');
  };
  audioOutput.onStateChanged = (TiRtcAudioOutputState state) {
    debugPrint('audio state=$state');
  };
  videoOutput.onStateChanged = (TiRtcVideoOutputState state) {
    debugPrint('video state=$state');
  };
  videoOutput.onRenderSizeChanged = (Size size) {
    debugPrint('video size=${size.width}x${size.height}');
  };

  conn.connect(remoteId: remoteId, token: token);
  audioOutput.attach(connection: conn, streamId: 10);
  videoOutput.attach(connection: conn, streamId: 11);
}

void stopPlayback() {
  videoOutput.detach();
  audioOutput.detach();
  conn.disconnect();
}

void disposePlayback() {
  videoOutput.dispose();
  audioOutput.dispose();
  conn.dispose();
}

For HarmonyOS native apps, mount TiRtcVideoOutputView with the TiRtcVideoOutput object:

import {
  TiRtcAudioOutput,
  TiRtcAudioOutputState,
  TiRtcConn,
  TiRtcConnState,
  TiRtcVideoFit,
  TiRtcVideoOutput,
  TiRtcVideoOutputState,
  TiRtcVideoOutputView,
} from 'tirtc-av/Index';

const conn = new TiRtcConn();
const audioOutput = new TiRtcAudioOutput();
const videoOutput = new TiRtcVideoOutput();

@Builder
function RemoteVideoView() {
  TiRtcVideoOutputView({
    output: videoOutput,
    fit: TiRtcVideoFit.contain,
    width: '100%',
    height: '100%',
  });
}

function startPlayback(remoteId: string, token: string): void {
  conn.onStateChanged = (state: TiRtcConnState, errorCode: number): void => {
    console.info(`conn state=${state} error=${errorCode}`);
  };
  audioOutput.onStateChanged = (state: TiRtcAudioOutputState): void => {
    console.info(`audio state=${state}`);
  };
  videoOutput.onStateChanged = (state: TiRtcVideoOutputState): void => {
    console.info(`video state=${state}`);
  };
  videoOutput.onRenderSizeChanged = (size): void => {
    console.info(`video size=${size.width}x${size.height}`);
  };

  conn.connect({ remoteId, token });
  const audioCode = audioOutput.attach({ connection: conn, streamId: 10 });
  const videoCode = videoOutput.attach({ connection: conn, streamId: 11 });

  console.info(`audio=${audioCode} video=${videoCode}`);
}

function stopPlayback(): void {
  videoOutput.detach();
  audioOutput.detach();
  conn.disconnect();
}

function disposePlayback(): void {
  videoOutput.dispose();
  audioOutput.dispose();
  conn.dispose();
}

Success Signals

• The connection reaches connected.
• The audio output reaches playing.
• The video output reaches rendering, or the video size callback fires.

If the connection is established but there is no video, check:

• whether the client stream IDs match the device stream IDs;
• whether the first video frame from the device is a key frame;
• whether the video output widget is mounted in a visible view tree.