Pick Cell

A robot cell that only exists in simulation, for now. An arm learns to sort parts off a conveyor while you watch its camera, its decisions and its score. When something looks wrong, you stop the clock and step it forward yourself.

Get Tato Studiotatolab new --sample pick-cell
The Pick Cell project in Tato Studio: a 3D view of the robot arm, conveyor and bins, above the graph that wires the simulator, detector and policy in a loop. The operator panel is on the right.

How it works

1

Operator Panel

A panel you design

A web page with Run, Pause and Step once, a speed slider and the live score. It's the only thing the operator touches.

2

Sim Step

The simulator

Moves the world on by one step: the arm, the belt and every part on it. It runs in Python, on the physics engine you choose.

3

Detect Parts

A vision model

Looks through the simulated camera and boxes every part it finds. It runs on the graphics card.

4

Policy

The decision

Takes the boxes and the arm's joints and picks the next move. The clock waits for it before it steps.

Build the panel as a web page.

The operator panel is a web page you write, plus one small file that lists its pins: what goes in and what comes out, each with its type. It shows up in the stream graph as a node, and you wire its buttons to whatever they should drive.

A panel can show the live picture as well, so the person running the cell never has to look at the graph.

Read the docs

tatolab_view.json

{
  "contract": "1.0",
  "title": "Operator Panel",
  "description": "Run, pause and step a robot cell.",
  "entry": "index.html",
  "placement": "node",
  "inputs": [
    { "name": "score", "pin_type": { "type": "integer" } }
  ],
  "outputs": [
    { "name": "speed", "pin_type": { "type": "float" }, "range": [0, 1] },
    { "name": "step_once", "pin_type": { "type": "trigger" } }
  ]
}

An agent added a node mid-run.

Detect Parts wasn't added by hand. An agent wired it in while the arm was sorting. History lists the edit under the agent's name, and your undo leaves it alone.

Pick Cell with the History view open: edits by you, by an agent, by the command line and by the runtime, newest first.

Run it on your machine.

Tato Studio is free and open source. You won't be asked to sign in, and there's no licence key to lose.

orbrew install --cask tatolab/tap/tato-studio

The engine is GPL-3.0-only. Tato Studio is AGPL-3.0-only, or a commercial licence from Tatolab LLC. Read the licences