WebRTC (WHIP and WHEP)

The tatolab-webrtc package adds two nodes for exchanging media with WebRTC services:

• WhipPublisher sends encoded video and audio to a WHIP endpoint: a live-streaming service or media server that accepts WebRTC ingest, such as Cloudflare Stream.
• WhepPlayer receives video and audio from a WHEP endpoint, WHIP’s playback counterpart, and outputs them, still encoded, for the rest of your stream.

Use them when a service you already work with speaks WHIP or WHEP. Sharing streams between your own machines, and with browsers and tools, is a different job, done by the runtime’s own MoQ relay; see Across machines.

They’re ordinary Python nodes from an ordinary package. Each runs in its own process, like your own nodes, with its WebRTC work done in native code inside the package.

uv add tatolab-webrtc

Publish the camera and microphone

WhipPublisher takes encoded video (H.264) and encoded audio (Opus) on one input, tracks. Each link into it becomes one track of the WebRTC session:

stream.py

import os

from dotenv import load_dotenv
from tatolab.stream import CameraSource, H264Encoder, MicrophoneSource, OpusEncoder, StreamBuilder, stream
from tatolab.webrtc import WhipPublisher

load_dotenv()  # reads WHIP_URL from .env in the project folder


@stream
def go_live(stream_builder: StreamBuilder) -> None:
    """Publishes the camera and microphone to a WHIP endpoint."""
    camera = stream_builder.add(CameraSource)
    video_encoder = stream_builder.add(H264Encoder, config={"bitrate_bps": 4_000_000})
    microphone = stream_builder.add(MicrophoneSource)
    audio_encoder = stream_builder.add(OpusEncoder)
    publisher = stream_builder.add(WhipPublisher, config={"url": os.environ["WHIP_URL"]})

    stream_builder.connect(camera.output("video"), video_encoder.input("video"))
    stream_builder.connect(video_encoder.output("encoded_video"), publisher.input("tracks"))
    stream_builder.connect(microphone.output("audio"), audio_encoder.input("audio"))
    stream_builder.connect(audio_encoder.output("encoded_audio"), publisher.input("tracks"))

Put the endpoint in a .env file next to stream.py, and add .env to .gitignore:

.env

WHIP_URL=https://customer-xxxx.cloudflarestream.com/<key>/webRTC/publish

Install python-dotenv too (uv add python-dotenv), then tatolab run.

The URL is a password

Services like Cloudflare Stream put the stream key in the URL, so anyone with the URL can publish to your stream. Keep it out of your code and out of version control. If your endpoint uses a bearer token instead, pass it as "bearer_token" in the publisher’s config, read from .env the same way.

Like any node setting, the URL is part of the stream’s graph: tatolab graph and agents connected to the runtime can read it, and a kept stream’s record in the runtime’s state folder contains it. Both are only reachable by your user account on that machine.

Why .env and not an exported variable?

Your stream runs in the runtime, not in your shell. Variables you export in a terminal don’t reach it. The stream function and your nodes start in the project folder, so a .env file there, loaded with python-dotenv, is the simple way to give them settings. See Settings and secrets.

How the session behaves

• A session carries at most one video and one audio track. Connecting a third link is refused when the stream starts; a second link of the same kind is refused when its first bag arrives.
• The publisher doesn’t reconnect. If the session drops, restart the stream. (WhepPlayer does reconnect, with a backoff.)
• The session opens with the first bag, not when the stream starts. If the endpoint is unreachable, the stream still starts, and the error appears in the log when the first bag tries to connect.
• Only H.264 video and Opus audio are accepted, with at most two audio channels.
• Video arrives about one frame behind audio at the far end, around 33 ms at 30 frames per second, because of how RTP timestamps frame durations.
• Logs to watch: the publisher logs when it opens the session and when the endpoint accepts it, then reports every 300 bags. Silence with no error means no bags are reaching it, so check the encoder.

Video needs a hardware H.264 encoder; see Record and encode.

Play a WHEP stream

WhepPlayer connects to a WHEP endpoint and publishes what it receives as encoded video and audio. Decode them to watch and listen:

stream.py

import os

from dotenv import load_dotenv
from tatolab.stream import DisplayWindow, H264Decoder, OpusDecoder, SpeakerSink, StreamBuilder, stream
from tatolab.webrtc import WhepPlayer

load_dotenv()  # reads WHEP_URL from .env in the project folder


@stream
def watch(stream_builder: StreamBuilder) -> None:
    """Plays a WHEP stream in a window, with its sound."""
    player = stream_builder.add(WhepPlayer, config={"url": os.environ["WHEP_URL"]})
    video_decoder = stream_builder.add(H264Decoder)
    audio_decoder = stream_builder.add(OpusDecoder)
    window = stream_builder.add(DisplayWindow, config={"title": "Live"})
    speaker = stream_builder.add(SpeakerSink)

    stream_builder.connect(player.output("encoded_video"), video_decoder.input("encoded_video"))
    stream_builder.connect(video_decoder.output("video"), window.input("video"))
    stream_builder.connect(player.output("encoded_audio"), audio_decoder.input("encoded_audio"))
    stream_builder.connect(audio_decoder.output("audio"), speaker.input("audio"))

The player connects in the background and reconnects on its own if the session drops, waiting a little longer between attempts each time, up to 10 seconds.

Once the video is decoded, it’s ordinary frames: run a model on it, record it, or put it on a virtual camera.

Related

• camera-webrtc-publish, in the repository.
• Record and encode: the encoders and their settings.