Inspect a running stream

Four commands show you a running stream from the outside, without changing it or slowing it down. They work on any stream the runtime holds, attached or kept, and you run them on the machine the stream runs on.

Command	Answers
tatolab graph	What nodes and links exist, and are they healthy? Is anything losing data?
tatolab tap	What is this output actually sending?
tatolab logs	What did the nodes say?
tatolab exchange	What does this frame look like?

The examples use the tutorial’s motion_watcher stream.

tatolab graph

Prints the live graph as JSON. Without arguments you get every stream on the machine, under the machine’s name; with --stream, one stream’s graph:

tatolab graph --stream motion_watcher

The parts you’ll use most:

Path	Holds
.nodes[].name, .nodes[].type	Each node’s name in the stream, and its class (nodes.motion_detector:MotionDetector, or tatolab.stream:CameraSource for a built-in)
.nodes[].components.state	The node’s state; Running when healthy
.nodes[].components.metrics	Counters, including bags dropped on each inbound link and writes refused on each output
.links[].state	wired when working; pending while connecting; awaiting_remote while waiting for another stream; error when it failed
.links[].error_reason, .links[].awaiting_remote_reason	Why a link failed, or what it’s waiting for
.exposed	The outputs the stream exposes, and at what level
.machine	This machine’s name, as addresses use it

Some jq recipes:

# Every node and its state
tatolab graph --stream motion_watcher | jq -r '.nodes[] | "\(.name)  \(.components.state)"'

# Links that aren't working, and why
tatolab graph --stream motion_watcher | jq '.links[] | select(.state != "wired")'

# Each node's counters
tatolab graph --stream motion_watcher | jq '.nodes[] | {name, metrics: .components.metrics}'

Losses are always counted. If a slow reader drops bags, the count shows in that node’s metrics, per link (dropped_bags_by_link), so a stream that has lost most of its frames can’t look healthy.

tatolab tap

Samples, for half a second, the bags one output sends, without disturbing it:

tatolab tap motion_watcher/detector/motion --count 5

The output needs a reader: a link in its own stream, or another stream reading it. An output nothing reads has nothing flowing, and the tap is refused with no tappable channel named …. Exposure doesn’t matter.

• The channel is the output’s address: <stream>/<node>/<port>, or <node>/<port> together with --stream.
• --count asks for up to that many bags. A tap collects for a short window and returns what arrived, so a quiet output gives a partial sample instead of hanging.
• The result says how many bags were requested, received and dropped, and gives each bag’s size and its bytes in hex.
• A bag bigger than the per-bag limit comes back cut short (hex_truncated: true) and can’t be decoded. Raise the limit with --max-bag-bytes.

Each bag’s hex is a 76-byte header (a 64-byte port key, an 8-byte timestamp and a 4-byte length) followed by the bag as msgpack. The tutorial has a ten-line script that decodes it.

What you’ll find in common bags:

Bag	Keys
Video	surface_id, width, height, timestamp_ns, and often color_info. The pixels aren’t in the bag; use exchange to see them.
Audio	samples (raw bytes), sample_rate, channels, sample_count, dtype, first_sample_timestamp_ns
Encoded video	bitstream (one compressed frame), codec, is_sync_point, group_index, sequence_index, width, height
Yours	Whatever your node writes

tatolab logs

tatolab logs --stream motion_watcher -f

Reads a stream’s log records from the runtime, Rust and Python alike, and with -f keeps following them. Filters:

Flag	Shows
--level warn	Warnings and errors only (trace, debug, info, warn, error)
--source python	Only your Python nodes (or rust for the runtime and built-ins)
--processor <id>	Only one node, by the ID tatolab graph shows
--intercepted-only	Only output captured from print(), stdout and stderr

The runtime also writes each stream’s log to disk, as JSON Lines, in the project’s .streamlib/logs/ folder. Run tatolab logs --list in the project folder to see the logs, and tatolab logs <name> to read one. That works after the stream has stopped, too. Logs covers the format, and shipping it to other tools.

tatolab exchange

A video bag names a frame that lives on the GPU. exchange fetches frames and writes them as exact, full-resolution PNG files:

tatolab exchange --channel motion_watcher/motiontint/tinted_video --count 3 --every 30 --out frames/

• --channel samples an output and saves the frame each bag names. --count is how many frames; --every 30 takes every 30th bag, about one a second at 30 fps.
• Or pass one surface ID, from a bag you tapped: tatolab exchange '7#42' --out frames/.
• --field names the bag key holding the surface ID, if a node uses a key other than surface_id.
• It prints each file’s path. --out isn’t emptied first, so use the printed paths rather than listing the folder.

It needs no window and no display, so it’s how you see what a stream produces on a machine without a screen.

Symptoms and where to look

Symptom	Look at
The window is black	graph: is every node Running? Then logs --level warn. A decoder or effect that failed to start says why.
A node isn’t running	logs --level error for the node’s error, often from setup
A link shows error	.links[].error_reason in graph
A link shows awaiting_remote	.links[].awaiting_remote_reason: usually the other stream isn’t running, or its port isn’t exposed
Frames are being dropped	metrics in graph shows where; the reader on that link can’t keep up
My node’s output looks wrong	tap its output and decode a few bags
The picture looks wrong mid-stream	exchange the output of each node in turn, to find the first one that’s wrong
The recording is empty or tiny	logs for the encoder: no hardware encoder is the usual cause

Let an agent do it

Every command here is also a tool an AI agent can call. Connect one with claude mcp add tatolab -- tatolab mcp, and ask it to find out why the window is black. See Agents and MCP.