Tag Tracker
Loading detector…

Cameras

Blur

Track AprilTags with your camera

Pick a camera and press Start. Show a printed tag36h11 tag to the camera, give it a name, and watch its last second of motion in 3D. Add more cameras to track from several sides.

Video is processed on this computer and never sent anywhere.

Recording

Records every pose the Track tab plots, with your notes.

Press V and speak. “Track”, “Calibrate”, “Stream”, “Settings”, “Start recording”, “Stop recording”, “Tare”, “Clear tare”, “Calibrate robot”, “Fit” and “Reset view” are commands; anything else is a note. Speech stays on this computer; the model downloads once (40 MB, or 120 MB on the graphics card).

Tags

ColourIDNameSizeStatusActions

No tags yet. Show one to the camera and it will appear here to name, or add one by its ID.

Rigid objects

Fix two or more tags to one object, at different angles, and it is tracked as one pose from all their corners: steadier, and free of the mirror-image flips a single flat tag can have.

Motion sensors

Use BLE, a direct USB controller, or a USB-connected XIAO ESB receiver. USB and ESB require TP/1 teleop firmware; its default is 1,000,000 baud. Both routes send motion into Filtered tracking and return TacHammer cues. Use Connect capture USB for the Haptography capture device. Disconnect it from Haptography first. Choose a hand, hold the device still for two seconds, then Set neutral to restore the gripper's base orientation at its current position. Turn about the board's up axis to yaw, tip the cable end to pitch, or twist around the cable to roll the gripper. Wrist rotation stops at its limits; mouse or AprilTags still sets its position. Returning to Tasks resumes orientation automatically. Set neutral again after a stream reset or if heading drifts. Test pulse checks the TacHammer output.

Stream

What is sent

One message for every track in view: its key (a tag's ID, or b: and an object's ID), its name, its position in metres and its orientation as a unit quaternion w, x, y, z. The frame is the 3D view's: the camera, or the tag chosen in Relative to, with any tare.

Format

Last message

Start a camera, or the demo, to see messages here.

Serial or USB

To a microcontroller, or anything on a serial port, one line at a time. Needs Chrome or Edge on a computer.

Not connected

UDP

A web page can't send UDP itself, so a small bridge on your computer passes each message on as one datagram. Download udp-bridge.py (Python 3, nothing else to install) and run it:

python3 udp-bridge.py
Not connected

Tracking

Look through the robot’s wide head camera. Tilt with the slider. With the view focused, ↑/↓ lift/lower the selected gripper and ←/→ move it sideways.
Drag to orbitShift-drag to panScroll to zoomDrag a hand to move itScroll on it to grip

Now

TagPosition, cmOrientation, quaternionDistanceSpeedCorner fit± depth
xyzwxyz
Nothing tracked yet.

Keyboard and mouse

Click the robot scene to focus it before using these keys. Select a gripper, then move or open it. The capture device or wrist tags control gripper orientation; mouse and keyboard control position and grip.

Keyboard layout

Highlighted keys have a use here. Hover, focus or tap one to see it below.

Swipe the keyboard sideways to see every key.

Mouse

Arrow keys

Outside view rotates the camera; Robot eyes moves the selected gripper.

Movement keys step 12 mm × Movement. Use the Movement slider for smaller steps. Mouse and keyboard take over a gripper’s position; choose Resume AprilTags to return position control to your wrist tags. Close with Escape or ×.

Choose a highlighted key or mouse control

This is a reference. Tapping here only shows a control’s use.

Print tags and the board

Paper
AprilTags, tag36h11

Your tags print at their own sizes. Size is the black square; a white border of one cell is added round it.

Calibration board
ChArUco board, 9 × 6 squares

On its own landscape page. Glue it to card so it stays flat, then measure a square and enter it in Calibrate.

Print at 100% scale (turn off “fit to page”), on the same paper as chosen here. Bars of 50 mm and 2 inches on the page let you check the scale with a ruler.

Settings

Kept in this browser.

Camera capture

Start with 720p and the Fast detector. Try 640 × 360 for less processing while keeping the same picture shape. Frame rate depends on the camera and lighting; the browser may deliver less than requested.

Voice commands

Press Voice or V, speak a command, then pause or press again to stop. With a Bluetooth controller, hold its joystick button to talk and release to send. Audio is transcribed locally with Whisper.

Try “Tasks”, “Tracking”, “Calibrate robot”, “Robot eyes”, “Outside view”, “Restart task”, “Next task”, “Fit view” or “Resume AprilTags”. Other speech becomes a note.

Controller audio uses the normal Bluetooth firmware. The separate USB / ESB teleoperation image has no microphone stream. Reconnect with Connect Bluetooth if an earlier connection did not grant the audio service.

Robot

Which track drives what. Auto takes the tag or object named Body (or Chest) as the body, and the others as the arms, each on the side of the body it is on.

Elbows out and below the hands

Detection and pose

Detector Search Pose Motion Compare with raw

Tags view

How much of each tag's motion the 3D view shows when it shows tags rather than the robot.

Printing

Paper

Setup guide

Show the setup guide when the page opens

Show on a phone

Set up

1. A camera

Tag Tracker finds AprilTags in your camera's picture and works out where each one is in 3D. Up to three tags drive a robot: one on your chest and one on each wrist. Video stays on this computer.

Show this when the page opens