Custom tracks

When the built-in camera and screen sharing capture don’t meet your needs, you can use LocalCustomVideoTrack to publish video generated by your app to the channel. The division of labor is fixed:

You produce the frames: continuously provide unencoded raw YUV (I420) data; no encoding on your side.
The SDK handles encoding and transport: it encodes and publishes according to the preset, and remote users subscribe and play it like any other video track.

Typical scenarios:

Publishing whiteboards / canvases / custom-drawn content
Re-publishing video decoded by a local player
Output processed by third-party SDKs (beauty filters, AR, AI generation)
Frame data from non-standard capture devices (external capture cards, USB devices)

1. Overall flow

Join succeeded (onJoinSucceed)
   ↓
getLocalCustomVideoTrack(preOpt)     Pick a preset, get the track
   ↓
publishLocalVideo(track, ...)        Publish the track, wait for onSuccess
   ↓
inputData(...)  ← feed frames in a loop    Your app keeps producing frames
   ↓
unPublishLocalVideo(track, ...)      Stop publishing

Three ordering constraints you must follow:

Publish only after joining (before joining, publishLocalVideo calls back onFail(102202): channel not started).
Feed frames only after publishing succeeds. While the track isn’t published, inputData is silently dropped without any notice—this is the most common reason “remote users can’t see the video”.
Audience users can’t publish (onFail(102207)). Check with rtcEngine.isAudience() first.

2. Choose a preset

The preset determines the encoding resolution, frame rate, bitrate, and track description (desc). For all fields, see Custom video preset.

// Default preset: 1080p / 10 fps / 1 Mbps, track description "custom"
val preOpt = PreOptionCustomVideo.def

// Publish external video with the screen sharing track description (remote users treat it as "sharing"), track description "screen"
val sharePreOpt = PreOptionCustomVideo.screen

To customize parameters, construct it directly. Keep the resolution the same on the capture and publish sides to avoid extra scaling in the SDK:

val preOpt = PreOptionCustomVideo(
    capture = CustomVideoCaptureOptions(
        width = 1280,
        height = 720,
        maxFps = 15,
        maxBitrate = 1200 * 1024
    ),
    publish = VideoPublishOptions(
        desc = TrackDesc.TRACK_CUSTOM.value,
        codec = CodecType.H264,
        maxBitrate = 1200 * 1024,
        width = 1280,
        height = 720,
        maxFps = 15,
        props = null,
        simulcasts = null      // Custom video doesn't use simulcast
    )
)

Tips for choosing parameters:

Mostly static content (whiteboards, documents, slides): 5–10 fps is enough; prioritize resolution (text clarity depends on resolution, not frame rate).
Dynamic content (re-published video, game footage): use 15–25 fps and raise the bitrate accordingly, or motion will look noticeably blurry.
Width and height must be even (I420 chroma planes are width/2 × height/2).

3. Get the track and publish

private var customTrack: LocalCustomVideoTrack? = null

private fun startCustomPush() {
    if (rtcEngine.isAudience()) {
        // Audience users can't publish
        return
    }

    val track = rtcEngine.getLocalCustomVideoTrack(preOpt)
    customTrack = track

    rtcEngine.publishLocalVideo(track, null, object : RTCResultListener {
        override fun onSuccess() {
            // Start feeding frames only after publishing succeeds
            startFrameLoop(track)
        }

        override fun onFail(code: Int) {
            // 102202: channel not started (not joined yet)
            // 102207: audience users can't publish
            // For other error codes, see the error codes page
        }
    })
}

Two things to note about the track instance:

The SDK caches it as a singleton. Calling getLocalCustomVideoTrack(preOpt) again returns the same object and overwrites the old value with the new preOpt. So don’t alternate frames between two desc values to simulate “two custom tracks”; only one preset should be in effect at a time.
If you override desc with PublishCustomOptions(desc = ...) when publishing, the SDK writes it back to preOpt.publish.desc, and inputData automatically locates the track by the latest desc. You don’t need to do anything extra.

// Overriding the track description with PublishCustomOptions
rtcEngine.publishLocalVideo(
    track,
    PublishCustomOptions(TrackDesc.TRACK_SHARE.value, null, null),
    listener
)

4. Prepare frame data (key)

inputData accepts only tightly packed I420. If you don’t follow the conventions in this section, remote users see corrupted video, misalignment, or green edges.

4.1 Memory layout

byteArray length = width * height * 3 / 2

┌───────────────────────────────┐  offset 0
│ Y plane  width * height       │
├───────────────────────────────┤  offset = width*height
│ U plane  (width/2)*(height/2) │
├───────────────────────────────┤  offset = width*height*5/4
│ V plane  (width/2)*(height/2) │
└───────────────────────────────┘

The three segments must be contiguous with no gaps, and pass the stride parameters as tightly packed values:

strideY = width
strideU = width / 2
strideV = width / 2

⚠️ The SDK currently computes plane offsets assuming a tightly packed layout; row strides with padding are not used for addressing. If upstream data has padding on each row (for example, Camera2’s rowStride > width), first copy the valid pixels row by row into a tightly packed array, then feed it in.

If the data length is insufficient, the SDK throws IllegalArgumentException("Invalid I420 size: ...") before encoding, which helps you quickly locate layout problems.

4.2 Generate I420 from a Bitmap / Canvas

Whiteboards and custom-drawn content are usually drawn to a Bitmap (ARGB_8888) first and then converted to I420:

/**
 * ARGB_8888 → tightly packed I420 (BT.601 limited range)
 * argb: pixels from Bitmap.getPixels; width / height must be even
 * out: reused output array, length >= width * height * 3 / 2
 */
fun argbToI420(argb: IntArray, width: Int, height: Int, out: ByteArray) {
    val frameSize = width * height
    var yIndex = 0
    var uIndex = frameSize
    var vIndex = frameSize + frameSize / 4

    for (y in 0 until height) {
        for (x in 0 until width) {
            val color = argb[yIndex]
            val r = (color shr 16) and 0xFF
            val g = (color shr 8) and 0xFF
            val b = color and 0xFF

            val yy = ((66 * r + 129 * g + 25 * b + 128) shr 8) + 16
            out[yIndex] = yy.coerceIn(16, 235).toByte()

            // Chroma is sampled 2x2; write only on even rows and even columns
            if (y and 1 == 0 && x and 1 == 0) {
                val u = ((-38 * r - 74 * g + 112 * b + 128) shr 8) + 128
                val v = ((112 * r - 94 * g - 18 * b + 128) shr 8) + 128
                out[uIndex++] = u.coerceIn(16, 240).toByte()
                out[vIndex++] = v.coerceIn(16, 240).toByte()
            }
            yIndex++
        }
    }
}

Usage:

private val pixels = IntArray(width * height)
private val i420 = ByteArray(width * height * 3 / 2)

bitmap.getPixels(pixels, 0, width, 0, 0, width, height)
argbToI420(pixels, width, height, i420)

The above is a pure Kotlin implementation that’s easy to understand and self-test, but a per-pixel loop at 1080p is noticeably expensive on low- and mid-range devices. In production, use libyuv (ARGBToI420) or a GPU approach instead; you can also downscale to 720p before publishing. Although the SDK bundles libyuv internally, it doesn’t expose the conversion APIs, so you need to add it to your app yourself.

4.3 Generate I420 from YUV_420_888

If the data comes from ImageReader / third-party capture (YUV_420_888), handle three things before feeding it in:

rowStride may be larger than width → copy row by row with width, removing the padding at the end of each row.
pixelStride may be 2 (semi-planar NV12/NV21 form) → sample by stride and split into separate U and V planes.
The U/V order must be I420 (U first, then V); NV21’s VU order needs to be swapped.

Here too we recommend using libyuv’s Android420ToI420 directly; hand-written loops easily go wrong with stride/pixelStride combinations.

5. Feed frames continuously

customTrack?.inputData(
    yuv = i420,
    width = width,
    height = height,
    strideY = width,
    strideU = width / 2,
    strideV = width / 2,
    rotation = 0,
    stamp = System.nanoTime()
)

Key parameters:

Parameter	Requirement
`rotation`	Only `0` / `90` / `180` / `270`. It’s frame metadata and doesn’t change the pixel layout in `yuv`; rotation is applied on the encoding and remote rendering side. Pass `0` if the image is already upright.
`stamp`	In nanoseconds, the same basis as Camera2’s `timestampNs`; just use `System.nanoTime()`. It must be monotonically increasing—timestamps going backward or jumping cause stuttering on the remote side and abnormal bitrate control.

Threading and pacing:

Don’t feed frames on the main thread. inputData is a synchronous call that copies, aligns/scales as needed, and submits for encoding; per-frame cost at 1080p is not negligible. Use a dedicated HandlerThread.
Throttle to the preset frame rate. Feeding faster than maxFps doesn’t improve quality; it only wastes CPU and memory bandwidth.
Keep feeding frames even for static content. If you stop feeding, remote users see the last frame frozen. If the image doesn’t change for a long time, keep sending the same frame at a minimal frame rate (for example, 1–2 fps) to keep the stream alive.
Reuse the output array. Once inputData returns, the data has been copied internally, so you can reuse the same ByteArray immediately and avoid GC jitter from allocating a new object per frame.

6. Stop and clean up

private fun stopCustomPush() {
    stopFrameLoop()                              // Stop feeding frames first
    customTrack?.let {
        rtcEngine.unPublishLocalVideo(it, null)  // Then unpublish
    }
    customTrack = null
}

Stop feeding frames first, then unpublish. If you reverse the order, a few frames are dropped (harmless, but the logs show invalid calls).
As with the camera, when you publish / unPublish in rapid succession, intermediate calls that get merged may not call back; rely on the callback of the last call or the final state.
leave() releases the media streaming engine. The track instance still exists but its publishing state is no longer valid—after rejoining, you must call publishLocalVideo again.
releaseSDK() also clears the local track cache; afterward you need to call getLocalCustomVideoTrack again.

Local preview: LocalCustomVideoTrack inherits rendering methods such as addPlayView, but the SDK doesn’t echo frames fed through inputData to these views (local echo works only for the camera track). For a local preview of a custom track, display your own data source directly (for example, the whiteboard View itself); you don’t need to add render views to this track.

7. Complete example

A minimal wrapper that publishes a Bitmap at a fixed frame rate:

class CustomVideoPusher(
    private val rtcEngine: RTCEngine,
    private val width: Int = 1280,
    private val height: Int = 720,
    private val fps: Int = 10
) {
    private val preOpt = PreOptionCustomVideo(
        capture = CustomVideoCaptureOptions(width, height, fps, 1200 * 1024),
        publish = VideoPublishOptions(
            desc = TrackDesc.TRACK_CUSTOM.value,
            codec = CodecType.H264,
            maxBitrate = 1200 * 1024,
            width = width,
            height = height,
            maxFps = fps,
            props = null,
            simulcasts = null
        )
    )

    private val pixels = IntArray(width * height)
    private val i420 = ByteArray(width * height * 3 / 2)

    private var track: LocalCustomVideoTrack? = null
    private var thread: HandlerThread? = null
    private var handler: Handler? = null
    private var running = false

    /** Frame source: implemented by your app; returns the image to publish now */
    var frameProvider: (() -> Bitmap?)? = null

    fun start() {
        if (rtcEngine.isAudience()) return

        val customTrack = rtcEngine.getLocalCustomVideoTrack(preOpt)
        track = customTrack

        rtcEngine.publishLocalVideo(customTrack, null, object : RTCResultListener {
            override fun onSuccess() = startLoop()
            override fun onFail(code: Int) {
                // 102202 not joined / 102207 audience / others
            }
        })
    }

    private fun startLoop() {
        if (running) return
        running = true
        thread = HandlerThread("custom-video-pusher").apply { start() }
        handler = Handler(thread!!.looper)
        handler?.post(pushTask)
    }

    private val pushTask = object : Runnable {
        override fun run() {
            if (!running) return
            val startAt = SystemClock.elapsedRealtime()

            frameProvider?.invoke()?.let { bitmap ->
                if (bitmap.width == width && bitmap.height == height) {
                    bitmap.getPixels(pixels, 0, width, 0, 0, width, height)
                    argbToI420(pixels, width, height, i420)
                    track?.inputData(
                        yuv = i420,
                        width = width,
                        height = height,
                        strideY = width,
                        strideU = width / 2,
                        strideV = width / 2,
                        rotation = 0,
                        stamp = System.nanoTime()
                    )
                }
            }

            // Throttle to the target frame rate, subtracting this frame's actual cost
            val cost = SystemClock.elapsedRealtime() - startAt
            val delay = (1000L / fps - cost).coerceAtLeast(0L)
            handler?.postDelayed(this, delay)
        }
    }

    fun stop() {
        running = false
        handler?.removeCallbacksAndMessages(null)
        thread?.quitSafely()
        thread = null
        handler = null

        track?.let { rtcEngine.unPublishLocalVideo(it, null) }
        track = null
    }
}

Usage:

val pusher = CustomVideoPusher(rtcEngine)
pusher.frameProvider = { whiteboardView.snapshotBitmap() }   // Your app provides the image

// Start after joining succeeds
pusher.start()

// Before stopping publishing / leaving the channel
pusher.stop()

8. Troubleshooting table

Symptom	Common causes
Remote video freezes on a frame	The frame loop was interrupted (thread exited, exception swallowed); rejoined after `leave()` without publishing again
Corrupted video, diagonal misalignment	`stride` was passed with padding; U / V planes in the wrong order (NV21 not swapped); array length doesn’t match `width × height`
Wrong colors (purple / green tint)	U and V planes swapped; conversion used the wrong color range (mixing full range and limited range)
Green edge on the right or bottom	Upstream row padding not removed; odd width or height passed
Wrong orientation	`rotation` doesn’t match the actual pixel layout—if the pixels are already rotated, pass `0`; don’t rotate twice
Remote stuttering, bitrate fluctuating	`stamp` not monotonically increasing or not in nanoseconds; frame pacing jitter too large
High local CPU usage, device heating up	Feeding frames on the main thread; feeding faster than `maxFps`; doing 1080p color conversion in pure Java/Kotlin loops
No local preview	Expected behavior; the SDK doesn’t echo custom frames, so your app has to draw them

LocalCustomVideoTrack: full API and parameter definitions
Custom video preset: PreOptionCustomVideo fields and built-in presets
RTCEngine: getLocalCustomVideoTrack / publishLocalVideo / unPublishLocalVideo
Quickstart: the minimal flow for joining, publishing, and subscribing

Overview

Web SDK

Android SDK

Swift SDK (iOS/macOS)

C SDK (server/embedded)

Python SDK (server-side AI)

Server API

1. Overall flow

2. Choose a preset

3. Get the track and publish

4. Prepare frame data (key)

4.1 Memory layout

4.2 Generate I420 from a Bitmap / Canvas

4.3 Generate I420 from YUV_420_888

5. Feed frames continuously

6. Stop and clean up

7. Complete example

8. Troubleshooting table

​1. Overall flow

​2. Choose a preset

​3. Get the track and publish

​4. Prepare frame data (key)

​4.1 Memory layout

​4.2 Generate I420 from a Bitmap / Canvas

​4.3 Generate I420 from YUV_420_888

​5. Feed frames continuously

​6. Stop and clean up

​7. Complete example

​8. Troubleshooting table

​Related docs

1. Overall flow

2. Choose a preset

3. Get the track and publish

4. Prepare frame data (key)

4.1 Memory layout

4.2 Generate I420 from a Bitmap / Canvas

4.3 Generate I420 from YUV_420_888

5. Feed frames continuously

6. Stop and clean up

7. Complete example

8. Troubleshooting table

Related docs