The Tato engine

The part that does the work. It runs your project on the graphics card, keeps the show up when one piece misbehaves, and doesn't need the app. Put it on the show machine and drive it from Studio, from a terminal or from an agent.

Every pin has a type.

A pin's colour is its type: a texture, audio, a float, a colour, a trigger. A wire only joins pins that agree. When one can become the other, like an integer into a float, the wire just works, and a conversion that costs real work shows up as a node of its own, where you can see it.

Texture

Pictures and video. A GPU surface with its size, pixel format and colour.

Audio

Sound. Named sample channels at one rate.

Tensor

Numbers by the million. Named tensors, on the GPU or the CPU.

Encoded video

Compressed video. Encoded units of one codec. Encoded audio has a pin of its own.

Trigger

A moment. It fires once, and whatever it reaches acts once.

Values have pins too: boolean, integer, float, string, menu, colour, vector, rectangle, transform and asset. Each comes with its control, so an input with no wire is set right on the node.

Write your own nodes.

@node.shader wraps a Slang function. Its parameters become the pins and it runs on the GPU. That's the entire file.

@node.function reads the signature and builds the node from it. A parameter is an input pin and its default is the pin's value. Add a Time and a State parameter and it gets the clock and whatever it kept from the last run.

@node.process runs a class in a process of its own, with whatever's in the project's venv: numpy, torch, OpenCV, the SDK for that one odd sensor. This one reads a frame back to the CPU once a second and logs how bright it was.

#[function_node] does the same for a Rust function. #[pin_default] on an argument sets the value on its pin.

ripple.py

from tatolab.stream import ImageFrame, node
 
@node.shader(
    """
float4 Ripple(ImageInput image, float amount, float rings, float time)
{
    float2 from_center = image.uv - 0.5;
    float radius = length(from_center);
    float wave = sin((radius * rings - time) * 6.2831853);
    float2 push = normalize(from_center + 0.0001) * wave * amount * 0.02;
    return image.sample(saturate(image.uv + push));
}
""",
)
def Ripple(
    image: ImageFrame,
    amount: float = 1.0,
    rings: float = 8.0,
    time: float = 0.0,
) -> ImageFrame: ...

lag.py

import math
from dataclasses import dataclass
from tatolab.stream import State, Stateful, Time, node
 
@dataclass(frozen=True)
class LagValue:
    value: float = 0.0
    last_evaluated_at_ns: int = 0
    started: bool = False
 
@node.function
def Lag(
    x: float, seconds: float = 0.1, *, now: Time, previous: State[LagValue]
) -> Stateful[float, LagValue]:
    if not previous.started or seconds <= 0.0:
        return x, LagValue(x, now, True)
    elapsed_seconds = (now - previous.last_evaluated_at_ns) * 1e-9
    followed = x + (previous.value - x) * math.exp(-elapsed_seconds / seconds)
    return followed, LagValue(followed, now, True)

brightness_meter.py

import numpy
from tatolab.stream import Image, Input, RuntimeContextLimitedAccess, log, node
 
@node.process
class BrightnessMeter:
    """Logs the mean brightness of the frames it sees, once a second."""
 
    next_report_at_ns: int = 0
    video = Input(Image, delivery="newest")
 
    def process(self, ctx: RuntimeContextLimitedAccess) -> None:
        frame = ctx.inputs.read(self.video)
        if frame is None or ctx.time < self.next_report_at_ns:
            return
        # The slow door, named so: this reads the frame back from the GPU.
        with frame.cpu() as pixels:
            brightness = float(numpy.mean(pixels[:, :, :3]))
        log.info("brightness", mean=round(brightness, 1))
        self.next_report_at_ns = ctx.time + 1_000_000_000

scale_and_offset.rs

use tatolab_module_sdk::function_node::function_node;
 
#[function_node]
#[meta(description = "`x` scaled by `gain`, then moved by `offset`.")]
fn scale_and_offset(
    x: f64,
    #[pin_default(0.5)] gain: f64,
    #[pin_default(0.5)] offset: f64,
) -> f64 {
    x * gain + offset
}

What's already
in the box.

They're called Core Nodes and every project gets them. The colour, filter, transform and composite ones are Slang shaders. Open any of them and see how it's done.

GeneratorsSolid Color Gradient Radial Gradient Noise Checkerboard Stripes Circle Rectangle

SourcesImage File

ColorLevel Hsv Adjust Invert Monochrome Threshold Tint Duotone Posterize Lookup

FiltersBlur Sharpen Edge Detect Pixelate Displace Rgb Split Vignette Chroma Key Luma Key

TransformTransform Crop Flip Tile Mirror Kaleidoscope

CompositeMix Over Blend Add Blend Multiply Blend Screen Blend Difference Mask Wipe

MathRemap Clamp Add Subtract Multiply Divide Power Minimum Maximum Absolute Modulo Lerp Smoothstep Step Constant Random Perlin Noise

AnimationLfo Timer Lag Ramp Pulse Speed Envelope

LogicCompare Gate Switch Select Trigger Counter Sample And Hold Hold Frame

AudioOscillator

Not in the box yet. There's no node that plays a video or audio file, and no OSC, MIDI, DMX or NDI bridge. We know. For now each of those is a process node you write yourself, against the Python library you'd reach for anyway.

Frames stay on the GPU.

A camera frame lands in GPU memory and reaches torch through DLPack without a CPU copy. Reading pixels on the CPU is something you have to ask for, with frame.cpu().

One process per Python node.

When your pose model hangs at 9 pm, that's one node, and the log names it. The rest of the graph keeps running. You pay for this with a process boundary on every link into Python.

A project is a folder of text.

Graphs are JSON, nodes are Python files, dependencies are in pyproject.toml. So a show goes in git, and somebody can review a change before it's on a forty-foot wall.

Reload on save.

tatolab dev reloads the project every time you save, and keeps its place. Save something broken and it tells you why, then tries again on your next save.

Runs with no network.

The runtime brings its own Python 3.12, uv, wheels, Vulkan loader and Slang compiler. A new project syncs offline, which you'll be glad of at a venue with no Wi-Fi.

A clock you can step.

A stream picks its clock in the graph document: real time, scaled or stepped. A stepped clock won't tick until every node has finished the step. Nothing gets skipped because one node was slow.

Drive it from any AI agent.

There's an MCP server in the runtime. An agent can read the live graph, sample what's travelling down a link, look at a frame, read the logs, and add, connect or remove nodes, all while the stream is running.

It only answers on the machine it runs on. To reach the show machine from your laptop, go through ssh.

claude mcp add tatolab -- tatolab mcpclaude mcp add tatolab -- ssh <machine> tatolab mcp

Runs without the app.

On Linux, one command installs tatolabd (the runtime) and tatolab (its command line) under your home folder. It doesn't start anything. You run tatolabd yourself, in a terminal of its own.

curl -fsSL https://get.tatolab.com/install.sh | shtatolabdtatolab new my-show

tatolab new writes a project with a test pattern in it and plays it. GPL-3.0-only. View the source

Run it on your machine.

Tato Studio is free and open source. You won't be asked to sign in, and there's no licence key to lose.

orbrew install --cask tatolab/tap/tato-studio

The engine is GPL-3.0-only. Tato Studio is AGPL-3.0-only, or a commercial licence from Tatolab LLC. Read the licences