---
title: "How Tatolab works"
description: "Streams, nodes, ports and bags; exposing outputs so other streams, tools and agents can read them; the URLs a machine's MoQ relay gives them; and the runtime that runs it all."
type: "docs-page"
url: "https://tatolab.com/docs/how-it-works"
site: "Tatolab"
section: "Start here"
---

# How Tatolab works

> Streams, nodes, ports and bags; exposing outputs so other streams, tools and agents can read them; the URLs a machine's MoQ relay gives them; and the runtime that runs it all.

This page explains the ideas the rest of the docs build on. If you've run the [quickstart](https://tatolab.com/docs/quickstart), you've already met most of them.

## The short version

- A **stream** is a function that adds **nodes** and connects their **ports** with **links**.
- Data moves along links as **bags**: maps of named values, like a dict.
- The **runtime**, `tatolabd`, runs every stream on the machine. You talk to it with the `tatolab` CLI, and agents talk to it over MCP.
- Every output has an **address**, like `motion_watcher/detector/motion`. **Expose** an output, and other streams on the machine can read it too.
- Built-in nodes (camera, window, codecs, recorder) run inside the runtime. **Every Python node runs in its own process**, started from your project's virtual environment.

## Streams

A stream is a function decorated with `@stream`. It receives a `StreamBuilder` and describes a graph: which nodes to create, and how to connect them.

```python title="stream.py"
from tatolab.stream import DisplayWindow, StreamBuilder, TestPatternSource, stream


@stream
def bars(stream_builder: StreamBuilder) -> None:
    """Color bars in a window."""
    pattern = stream_builder.add(TestPatternSource)
    window = stream_builder.add(DisplayWindow)
    stream_builder.connect(pattern.output("video"), window.input("video"))
```

- **The function runs once, when the stream loads.** It doesn't run for each frame or each bag. The runtime records the graph it built and starts it.
- **The function's name is the stream's name** (`bars` here). Load it under another name with `tatolab run --name`. Its docstring documents the stream in your code; agents read node descriptions, not stream docstrings.
- **A project is an ordinary Python project:** a `stream.py`, a `pyproject.toml` and a `.venv`. There's no manifest and no build step. `tatolab run` and `tatolab dev` load the one `@stream` in `stream.py`; for a file with several, name one, as in `tatolab run stream.py:bars`.

## Nodes

A node is one step of a stream. There are two kinds:

- **Built-in nodes** ship inside the runtime: `CameraSource`, `TestPatternSource`, `DisplayWindow`, `MicrophoneSource`, `SpeakerSink`, the H.264, H.265 and Opus encoders and decoders, `Mp4Sink`, and `VirtualCameraSink` on Linux. They're native code, configured from Python with a `config` dict. See [Built-in nodes](https://tatolab.com/docs/reference/built-in-nodes).
- **Python nodes** are classes you write, decorated with `@node`, or classes from a package you install.

A Python node can handle any kind of data. This one watches temperature readings and sends an alarm when one runs too hot:

```python title="nodes/temperature_alarm.py"
from dataclasses import dataclass

from tatolab.stream import RuntimeContextLimitedAccess, node


@dataclass
class TemperatureAlarmConfig:
    threshold_celsius: float = 60.0


@node
class TemperatureAlarm:
    """Sends an alarm for each reading above the threshold."""

    @node.input(delivery_profile="ordered")
    def readings(self) -> dict: ...

    @node.output()
    def alarms(self) -> dict: ...

    def __init__(self, config: TemperatureAlarmConfig) -> None:
        self.threshold_celsius = config.threshold_celsius

    def process(self, ctx: RuntimeContextLimitedAccess) -> None:
        reading = ctx.inputs.read("readings")
        if reading is not None and reading["celsius"] > self.threshold_celsius:
            ctx.outputs.write("alarms", {"sensor": reading["sensor"], "celsius": reading["celsius"]})
```

**Each Python node runs in its own operating-system process**, with its own Python interpreter started from your project's `.venv`. If a node hangs, crashes or leaks memory, the problem stays in that process, the log names the node, and the rest of the stream keeps running. It's also why a node class lives in a module of its own (a file under `nodes/`, say) rather than in `stream.py`: the node's process imports the class by its module path.

[Write a node](https://tatolab.com/docs/build/nodes) covers everything a node can declare.

## Ports, links and bags

A node declares **input ports** and **output ports**, and `stream_builder.connect(output, input)` makes a **link** between them.

- One output can feed many inputs (a **fan-out**). The output isn't split between them: each reader reads it at its own pace.
- Many links can enter one input (a **fan-in**). `Mp4Sink`, for example, records one track per link into its `tracks` input.
- Ports have no types. The annotation on a port method (`-> dict`) is for you and your type checker; the runtime never compares types between nodes.

What travels along a link is a **bag**: a map of named values. There's no schema to register; a node writes a dict and the next node reads it. A bag can hold numbers, strings, booleans, lists, nested dicts and bytes. Built-in nodes use a few well-known shapes. A **video bag** names a frame held in GPU memory, which the runtime calls a **surface**: the bag carries the surface's ID (`surface_id`) with `width`, `height` and `timestamp_ns`, never the pixels. An **audio bag** carries its samples inline.

When you read, you choose how strict to be. `ctx.inputs.read("readings")` gives you the dict. `ctx.inputs.read("readings", into=Reading)` builds a dataclass or pydantic model from it, and raises an error if the bag doesn't fit.

### Delivery profiles

Every input port chooses how it reads:

| Profile | Reads | Use it for |
|---|---|---|
| `newest` | The most recent bag. Older ones are skipped. | Anything that only needs the current state: a window, a detector, a dashboard. |
| `ordered` | Every bag, in the order it was sent, unless the reader falls far behind. | Anything that must not skip: logs, encoders, recorders, audio. |

There's no default; each input must choose. **No link ever slows down the node that sends on it.** If a reader falls far behind, bags are dropped at that reader's input and counted, and `tatolab graph` shows the count in the reading node's metrics. One output can feed a `newest` reader and an `ordered` reader at the same time.

## Exposing outputs

Every output port has an **exposure level**, which decides whether other streams can read it:

| Level | Who can read the port | How to set it |
|---|---|---|
| internal | Only nodes in the same stream. This is the default. | Nothing to do |
| private | Other streams on this machine | `stream_builder.expose(output)`, or `tatolab expose` while it runs |
| public | Readers on other machines too, by URL. | `stream_builder.expose(output, Exposure.PUBLIC)`, or `tatolab expose --public` |

An exposed port has an **address**: `<stream>/<node>/<port>`, such as `motion_watcher/detector/motion`. Node names default to the class name in lowercase; pass `name=` to `stream_builder.add` for an address that reads well. Every part is made URL-safe for you: lowercased, accents dropped, and anything other than letters, digits, `-`, `.`, `_` and `~` turned into `-`. The full form starts with the machine's name, `<machine>/<stream>/<node>/<port>`; the machine's name is its Tailscale machine name, or its hostname when it isn't on a tailnet.

Another stream on the machine reads an exposed port by naming its address:

```python
stream_builder.connect(
    stream_builder.remote_output("motion_watcher/detector/motion"),
    event_log.input("motion"),
)
```

The reading stream owns the link: streams pull from each other, and no stream can push into another's input. The two streams can live in different projects, with different dependencies, and you can stop, change and restart either one without touching the other. [Share between streams](https://tatolab.com/docs/share/between-streams) covers the details.

Looking is different from linking. `tatolab tap`, `tatolab exchange` and an agent's sampling tools take the same addresses, and can sample any output that a link reads, exposed or not, because they're for inspecting your own streams. An output that nothing reads has nothing flowing to sample.

## URLs, over MoQ

Streams on one machine read each other through shared memory. Everything else reads an exposed output by its URL. The runtime runs a **MoQ** (Media over QUIC) relay on the machine, and the relay serves each exposed output at a URL whose path is the output's address.

![The runtime's MoQ relay gives an exposed output a URL, and a browser, a laptop and an agent all read it there.](https://tatolab.com/media/docs/relay-light.jpg)

- **A private output's URL works on its machine.** A browser, a script or any other tool there can read it.
- **A public output's URL works from your other machines too**, over your tailnet, where Tailscale provides the reach, names, identity and encryption. Once the machine joins a MoQ relay on the internet, it works from there as well, and that relay's access rules decide who can read.
- **Nothing is sent for an output nobody reads.** Readers pull, just as streams on one machine pull from each other.

[Across machines](https://tatolab.com/docs/share/across-machines) has the details, including reaching another machine's runtime over `ssh`.

## The runtime

`tatolabd` is the runtime: one program per machine that runs every stream on it.

![Each Python node runs in its own process. When the detector crashes, the camera and the recorder keep going.](https://tatolab.com/media/docs/isolation-light.jpg)

- **It's always on.** The Mac app and the Linux installer start it for you and restart it if it crashes. On a machine where nothing installed it as a service, you run `tatolabd` in a terminal. No `tatolab` command ever starts a runtime.
- **There's one per machine.** Every stream on the machine runs in it. A second `tatolabd` is refused with a message naming the one already running, and so is another user's: one user owns the runtime on a machine at a time.
- **It never runs your Python in its own process.** To learn a node's ports, it asks your project's Python. To run the node, it starts a process from your `.venv`.
- **Control stays on the machine.** The CLI and agents reach the runtime through a local socket that only your user account can open. No network port is opened for control. To work on another machine, you `ssh` to it.

### Attached and kept streams

A stream can be loaded in two ways:

- **Attached:** `tatolab run` or `tatolab dev`. The stream lives as long as the command. Press Ctrl-C, close the terminal, or kill the command, and the stream stops.
- **Kept:** `tatolab run -d`. The runtime records the stream and keeps it running after the command returns. It loads it again every time the runtime starts, including after a crash or a reboot, until you `tatolab stop` or `tatolab rm` it.

[Manage streams](https://tatolab.com/docs/run/manage) covers both.

## Seeing inside

Everything a running stream does can be looked at from outside, from a terminal or by an agent:

- `tatolab graph` prints the live graph as JSON: every node and link, their states, and counters such as dropped bags.
- `tatolab tap` samples the bags an output sends, without slowing it down.
- `tatolab exchange` saves the frame a video bag names as a PNG, with no window needed.
- Each stream writes a **JSON Lines log**, one record per line, in its project's `.streamlib/logs/` folder. `tatolab logs` reads and follows it, and any log tool can ship it, including into OpenTelemetry. See [Logs](https://tatolab.com/docs/run/logs).

[Inspect a running stream](https://tatolab.com/docs/run/inspect) shows each in use.

## Video, audio and the GPU

Streams that carry video or audio get help from the runtime. You don't have to use any of it:

- **Frames stay in GPU memory.** A video bag only names its frame. A model gets it as a GPU tensor through DLPack, with no CPU copy (`torch.from_dlpack(frame)`), a pixel effect is one GLSL function, and CPU code asks for pixels explicitly with `frame.cpu()`. See [GPU effects](https://tatolab.com/docs/build/gpu-effects) and [Use your model](https://tatolab.com/docs/build/models).
- **Hardware codecs and recording.** H.264 and H.265 use the GPU's encoder and decoder, Opus handles audio, and `Mp4Sink` writes a fragmented MP4 that stays playable if the stream stops abruptly. See [Record and encode](https://tatolab.com/docs/build/recording).
- **Audio arrives as you ask for it.** An input can ask for a sample rate, a format and a window size, and the runtime converts and cuts the audio to match. See [Audio](https://tatolab.com/docs/build/audio).

## One clock

Every timestamp in a stream (`timestamp_ns` on frames, `first_sample_timestamp_ns` on audio, `ctx.time` in a node) comes from the machine's monotonic clock, in nanoseconds. Timestamps from different nodes and streams on one machine compare directly, so lining up a camera frame with an audio block is a subtraction.

## What Tatolab leaves to you

| You keep | Tatolab's part |
|---|---|
| Your model runtime: PyTorch, ONNX Runtime, TensorRT, MLX | It hands the model the frame in GPU memory. No inference stack ships with Tatolab. |
| Your Python packages | A node imports whatever your project's `.venv` holds; `uv add` is how a dependency arrives. |
| Your network: Tailscale, ssh | Control is reachable only on the runtime's own machine. Reaching another machine is ssh's job, and the tailnet's. |
| Your message queues and databases | Tatolab doesn't replace them. A node you write sends to them with the client library you already use. |