From AI-driven software to AI-driven robotics

Teaching AI makes the gap between software and robotics hard to miss. Students can assemble a software application quickly. Giving a machine a useful physical capability brings a much longer list of things to make work together.

Control, sensing, motion planning, vision and hardware all matter. A model can perform well in a controlled example and still fail when the lighting changes, an object moves or a component wears down. Small teams have to decide which parts they can reuse and which assumptions they must test themselves.

More of the stack is becoming reusable

NVIDIA's November 2024 announcement brings together Isaac Lab and new Project GR00T workflows for robot learning. Isaac Lab supports training policies in simulation. The announced GR00T workflows address environment generation, motion, manipulation, control, mobility and perception; NVIDIA described those workflows as coming soon.

Hugging Face's LeRobot is another example of shared infrastructure: an open-source collection of datasets, models and tools for robot learning. For a small team, being able to inspect and reuse parts of a working system is valuable.

Simulation lets a team run experiments before committing every change to hardware. It can make repeated trials cheaper and make failures easier to investigate. The transfer to a real machine still needs evidence. A simulator's assumptions about contact, friction or sensors may be exactly where the deployed system breaks.

Robot data presents its own constraints. Demonstrations take time to collect, and they describe particular bodies, tasks and environments. A larger dataset helps only if it contains the variation the robot needs to handle. I would want to know which conditions the training data leaves out.

What I would build around

The opportunity for a startup is a specific task that becomes practical as more of this infrastructure becomes available. A team can spend more of its effort on the customer's environment, integration and reliability when it can reuse parts of the learning stack.

There is also work in the tools between components. Moving from a successful experiment to a maintained service requires deployment, monitoring and a way to investigate failures. Those are products in their own right when enough teams share the problem.

I would use simulation early, then test the most consequential assumptions on hardware. The question is how quickly the team can find the gap between what the model appears to do and what the customer can depend on.

A demonstration may last a few minutes. The customer needs to know what happens over weeks of use: how often someone intervenes, who maintains the machine, and how work continues when it stops. Those details belong in the business model from the beginning.

Robotics is becoming more accessible to small teams. The teams I would pay attention to are the ones that turn that access into a narrow, dependable capability someone will pay to use.