A robot that learns the world as motion

It acts, remembers, and even uses tools — without ever needing to know what anything is.

This is a short guide to a small working system. The animated demo shows it step by step; these notes explain what you are watching and why it is unusual. The one idea underneath everything: technology is fundamentally motion, not objects.

01The idea, plainly

Almost every robot today is taught to recognise objects — this is a cup, that is a door — and then to act on them. This system does the opposite. It never stores what a thing is. It stores only what its movements do: touch this, turn that way, and the world changes so. To the robot a dial is not “a knob” — it is a place where a turning motion changes something, reversibly. The object dissolves into the relationship between a motion and its effect. Names turn out to be a story we add on top; underneath, it is all motion.

This comes from a theory with two halves that are mirror images of each other. Using a tool is a motion that produces a trace in the world (a scratch, a turned dial, a knapped stone). Reading a trace — what an archaeologist does — is the same thing run backwards: from the mark, recover the motion that made it. One process, two directions.

The robot in the demo is the forward half, built. The inverse half — reading traces — is what we build next, and it matters for the robot too (see the last section).

02The cycle, step by step

This is the core, and it mirrors the demo exactly. Each line is a stage the robot passes through, the function that runs it, and what it means in plain terms. The first nine are how it discovers a move from scratch; the last three are how it reuses what it learned.

Discover — learning a move with no labels

1. agent.engage()
   Reach toward something. No contact yet — just moving closer.
2. world.contact_begin()
   Touch. The instant of contact opens a “move” — touching is what starts recording.
3. record() · apply_step()
   Move through the contact, measuring force, distance and speed. This trio is Θ — the single currency everything is recorded in.
4. world.contact_end()
   Let go. Releasing ends the move. One clean unit of motion has been captured.
5. segment_stream()
   Cut the continuous stream of experience into moves — at every touch-and-release.
6. cluster_symmetry_first()
   Recognise the kind of move from its shape alone — a turn, a push — by its symmetry. No object label is used.
7. bind_effects()
   Notice what the move did: the dial’s angle changed, and it can be turned back — reversible.
8. memory.add()
   File the experience — as a motion, tied to where and when. It never stores the word “knob”.

Reuse — acting on a goal with what it knows

9. Goal(…) → notice_problem()
   A goal arrives (“nudge that angle”). It searches memory and finds a match: “I already know a move for this — right here.” No need to explore again.
10. execute() · compare() · revise()
    Do the move, check that what happened matches what it expected, and update the record.
11. — result —
    Now the robot “knows” there is a turnable thing here it can operate — learned entirely as motion.

03What makes it unusual

No objects. The whole cycle runs without a single object category. This isn’t an omission — the currency it thinks in (force, distance, speed) simply has no slot for “what a thing is”. So it can meet a dial it has never seen, recognise the turn, and act — because it transfers by what a motion does, not by what a thing is called. A new object that behaves oddly can’t break a category, because there are no categories to break.

One currency. Because every engagement — a turn, a push, a reach — is recorded in the same three measures, they can all be compared, remembered, and reasoned about together. That shared currency is what lets the robot carry a lesson from one situation to another.

The shape of a touch. Instead of just recording how much force a contact used, the system keeps the whole curve of how force changed as it pushed. From that one curve, several things it used to work out separately just fall out: how much effort the move cost, whether it was reversible (does the curve return the way it came?), and — the vivid one — the signature of a material breaking under pressure (force climbs, then suddenly collapses). That break is exactly the moment a camera’s view of “sudden movement” could be bound to, giving the picture its meaning.

04Using a tool

This is the theory’s central claim, and the demo’s companion scene shows it: when something is out of reach, the robot reads the problem as distance, finds a stick, and extends its own body so the same reach now works — then remembers that new ability for next time. That is “technology as boundary-extending motion” in action. Notably, finding-and-using a tool is the same operation as choosing a known move — the robot just searches the environment for something whose length closes the gap, instead of searching its memory for a motion.

05What comes next — reading the world backwards

So far the robot only knows what it did. The next piece to build is the inverse model — reading a trace to recover the motion that made it. This is the archaeological half of the theory, and it is not a side-quest: the robot needs it to reason about a world it did not fully witness. When it returns to a place and finds something changed, or must judge the state of a machine from the marks of past use, it is reading a trace — inferring a motion it never saw. Change-detection in the current system is already a first, tiny version of this. The full inverse model turns the robot from something that only remembers its own actions into something that can read what happened.