---
title: "The engine"
description: "What runs under Tato Studio: typed pins, your own nodes in Python, and a runtime that works without the app."
type: "page"
url: "https://tatolab.com/engine"
site: "Tatolab"
---

# The Tato engine

The part that does the work. It runs your project on the graphics card, keeps the show up when one piece misbehaves, and doesn't need the app. Put it on the show machine and drive it from Studio, from a terminal or from an agent.

[Read the docs](https://tatolab.com/docs)

[View the source](https://github.com/tatolab/streamlib)

## Every pin has a type.

A pin's colour is its type: a texture, audio, a float, a colour, a trigger. A wire only joins pins that agree. When one can become the other, like an integer into a float, the wire just works, and a conversion that costs real work shows up as a node of its own, where you can see it.

- **Texture.** Pictures and video. A GPU surface with its size, pixel format and colour.
- **Audio.** Sound. Named sample channels at one rate.
- **Tensor.** Numbers by the million. Named tensors, on the GPU or the CPU.
- **Encoded video.** Compressed video. Encoded units of one codec. Encoded audio has a pin of its own.
- **Trigger.** A moment. It fires once, and whatever it reaches acts once.

Values have pins too: boolean, integer, float, string, menu, colour, vector, rectangle, transform and asset. Each comes with its control, so an input with no wire is set right on the node.

## Write your own nodes.

`@node.shader` wraps a Slang function. Its parameters become the pins and it runs on the GPU. That's the entire file.

`@node.function` reads the signature and builds the node from it. A parameter is an input pin and its default is the pin's value. Add a `Time` and a `State` parameter and it gets the clock and whatever it kept from the last run.

`@node.process` runs a class in a process of its own, with whatever's in the project's venv: numpy, torch, OpenCV, the SDK for that one odd sensor. This one reads a frame back to the CPU once a second and logs how bright it was.

`#[function_node]` does the same for a Rust function. `#[pin_default]` on an argument sets the value on its pin.

ripple.py:

```python
from tatolab.stream import ImageFrame, node

@node.shader(
    """
float4 Ripple(ImageInput image, float amount, float rings, float time)
{
    float2 from_center = image.uv - 0.5;
    float radius = length(from_center);
    float wave = sin((radius * rings - time) * 6.2831853);
    float2 push = normalize(from_center + 0.0001) * wave * amount * 0.02;
    return image.sample(saturate(image.uv + push));
}
""",
)
def Ripple(
    image: ImageFrame,
    amount: float = 1.0,
    rings: float = 8.0,
    time: float = 0.0,
) -> ImageFrame: ...
```

lag.py:

```python
import math
from dataclasses import dataclass
from tatolab.stream import State, Stateful, Time, node

@dataclass(frozen=True)
class LagValue:
    value: float = 0.0
    last_evaluated_at_ns: int = 0
    started: bool = False

@node.function
def Lag(
    x: float, seconds: float = 0.1, *, now: Time, previous: State[LagValue]
) -> Stateful[float, LagValue]:
    if not previous.started or seconds <= 0.0:
        return x, LagValue(x, now, True)
    elapsed_seconds = (now - previous.last_evaluated_at_ns) * 1e-9
    followed = x + (previous.value - x) * math.exp(-elapsed_seconds / seconds)
    return followed, LagValue(followed, now, True)
```

brightness_meter.py:

```python
import numpy
from tatolab.stream import Image, Input, RuntimeContextLimitedAccess, log, node

@node.process
class BrightnessMeter:
    """Logs the mean brightness of the frames it sees, once a second."""

    next_report_at_ns: int = 0
    video = Input(Image, delivery="newest")

    def process(self, ctx: RuntimeContextLimitedAccess) -> None:
        frame = ctx.inputs.read(self.video)
        if frame is None or ctx.time < self.next_report_at_ns:
            return
        # The slow door, named so: this reads the frame back from the GPU.
        with frame.cpu() as pixels:
            brightness = float(numpy.mean(pixels[:, :, :3]))
        log.info("brightness", mean=round(brightness, 1))
        self.next_report_at_ns = ctx.time + 1_000_000_000
```

scale_and_offset.rs:

```rust
use tatolab_module_sdk::function_node::function_node;

#[function_node]
#[meta(description = "`x` scaled by `gain`, then moved by `offset`.")]
fn scale_and_offset(
    x: f64,
    #[pin_default(0.5)] gain: f64,
    #[pin_default(0.5)] offset: f64,
) -> f64 {
    x * gain + offset
}
```

## What's already in the box.

They're called Core Nodes and every project gets them. The colour, filter, transform and composite ones are Slang shaders. Open any of them and see how it's done.

**Generators**Solid Color Gradient Radial Gradient Noise Checkerboard Stripes Circle Rectangle

**Sources**Image File

**Color**Level Hsv Adjust Invert Monochrome Threshold Tint Duotone Posterize Lookup

**Filters**Blur Sharpen Edge Detect Pixelate Displace Rgb Split Vignette Chroma Key Luma Key

**Transform**Transform Crop Flip Tile Mirror Kaleidoscope

**Composite**Mix Over Blend Add Blend Multiply Blend Screen Blend Difference Mask Wipe

**Math**Remap Clamp Add Subtract Multiply Divide Power Minimum Maximum Absolute Modulo Lerp Smoothstep Step Constant Random Perlin Noise

**Animation**Lfo Timer Lag Ramp Pulse Speed Envelope

**Logic**Compare Gate Switch Select Trigger Counter Sample And Hold Hold Frame

**Audio**Oscillator

**Not in the box yet.** There's no node that plays a video or audio file, and no OSC, MIDI, DMX or NDI bridge. We know. For now each of those is a process node you write yourself, against the Python library you'd reach for anyway.

**Frames stay on the GPU.** A camera frame lands in GPU memory and reaches torch through DLPack without a CPU copy. Reading pixels on the CPU is something you have to ask for, with `frame.cpu()`.

**One process per Python node.** When your pose model hangs at 9 pm, that's one node, and the log names it. The rest of the graph keeps running. You pay for this with a process boundary on every link into Python.

**A project is a folder of text.** Graphs are JSON, nodes are Python files, dependencies are in `pyproject.toml`. So a show goes in git, and somebody can review a change before it's on a forty-foot wall.

**Reload on save.** `tatolab dev` reloads the project every time you save, and keeps its place. Save something broken and it tells you why, then tries again on your next save.

**Runs with no network.** The runtime brings its own Python 3.12, uv, wheels, Vulkan loader and Slang compiler. A new project syncs offline, which you'll be glad of at a venue with no Wi-Fi.

**A clock you can step.** A stream picks its clock in the graph document: real time, scaled or stepped. A stepped clock won't tick until every node has finished the step. Nothing gets skipped because one node was slow.

## Drive it from any AI agent.

There's an MCP server in the runtime. An agent can read the live graph, sample what's travelling down a link, look at a frame, read the logs, and add, connect or remove nodes, all while the stream is running.

It only answers on the machine it runs on. To reach the show machine from your laptop, go through ssh.

`claude mcp add tatolab -- tatolab mcp`

`claude mcp add tatolab -- ssh <machine> tatolab mcp`

### Runs without the app.

On Linux, one command installs `tatolabd` (the runtime) and `tatolab` (its command line) under your home folder. It doesn't start anything. You run `tatolabd` yourself, in a terminal of its own.

`curl -fsSL https://get.tatolab.com/install.sh | sh`

`tatolabd`

`tatolab new my-show`

`tatolab new` writes a project with a test pattern in it and plays it. GPL-3.0-only. [View the source](https://github.com/tatolab/streamlib)

## Run it on your machine.

Tato Studio is free and open source. You won't be asked to sign in, and there's no licence key to lose.

[Download for macOS](https://tatolab.com/download/macos) · [Download for Linux](https://tatolab.com/download/linux)

or `brew install --cask tatolab/tap/tato-studio`

The engine is GPL-3.0-only. Tato Studio is AGPL-3.0-only, or a commercial licence from Tatolab LLC. [Read the licences](https://tatolab.com/download#licences)
