Flexion is constructing a reinforcement studying and sim-to-real platform for humanoid robots. Supply: Flexion
Prior to now 18 months, humanoid robotics firms have raised billions of {dollars} – a majority of which is quietly funding hiring people to function robots. This implies the robotics trade has a teleoperation and knowledge downside it retains describing as a labor answer.
Teleoperation and human demonstration at scale have develop into the dominant methodology for coaching bodily AI techniques, attracting severe capital, recruiting employees throughout lower-wage economies, and incomes enthusiastic protection as proof of progress. The belief beneath all of it’s that sufficient demonstrations will ultimately produce robots able to generalizing throughout actual environments.
I imagine that assumption deserves much more scrutiny than it’s getting.
Teleoperation hits a structural wall
Language fashions skilled on textual content can draw from many years of writing, articles, and books. With robots, there’s no archive to attract from – somebody has to generate each demonstration, which implies the info can solely develop as quick as human labor permits.
Teleoperation datasets are over 100,000 instances smaller than what’s used to coach right this moment’s language and imaginative and prescient fashions. That hole doesn’t shut by hiring extra operators, as a result of the true world by no means stops altering: A shelf strikes, a door deal with is barely completely different, and a brand new bundle kind exhibits up on the road. Each variation requires a brand new demonstration, which means the issue grows quicker than the workforce can.
Information high quality is the opposite challenge. Operators can’t really feel what they’re touching or decide depth reliably, so that they transfer slowly and overcorrect. This forces the robotic into studying from footage of somebody scuffling with a controller, and that’s what it finally ends up training.
Save the date for RoboBusiness 2026The human value of knowledge technology
The trade’s reply to the info downside has been to recruit extra folks, predominantly employees in lower-wage economies, employed to movie family duties, function robots remotely, or transfer via services carrying digicam rigs.
An entire business ecosystem has emerged round it, with startups throughout China, India, Europe, and the U.S. promoting teleoperation knowledge the identical manner firms as soon as bought labeled textual content for language fashions.
The unique pitch for humanoid robots is that people gained’t have the ability to fill these jobs sooner or later as a result of demographic shifts, labor shortages, and growing older populations. But when what we’re really constructing is infrastructure that requires a everlasting stream of human demonstrations to operate, then we would as nicely have these people do the duty instantly.
A system that may’t deal with something new with out contemporary human enter is basically only a labor system.
A handy protection for robotics enchancment
The usual response is that teleoperation is a bridge – a approach to get began whereas higher robotic mannequin coaching strategies catch up. For slim, repetitive duties in managed environments, that’s truthful.
However what a lot of the trade is definitely constructing is infrastructure for producing demonstrations indefinitely, with no clear account of how or when that adjustments.
The sphere is monitoring what’s simple to depend – demonstrations collected, hours of footage logged, duties accomplished in managed settings – none of which tells you whether or not the robotic can deal with one thing it hasn’t seen earlier than, in a spot that wasn’t arrange for it. Constructing extra of the identical infrastructure deepens that dependency on people somewhat than resolving it.
Flexion’s full autonomy stack features a command layer, a movement layer, and a management layer. Supply: Flexion
The trail that matches the issue
When researchers skilled early language fashions on large quantities of textual content, they received techniques that might loosely imitate the fashion of Shakespeare, however produced phrases that didn’t fairly make sense. Spectacular on the floor, however not but able to reasoning.
The breakthrough got here via reinforcement studying in artificial environments, which produced techniques able to reasoning, coding, and following complicated directions.
The robotics trade is essentially caught in that early second. Scaling teleoperation knowledge is the equal of scaling pre-training textual content on 100,000x much less knowledge. You get robots that considerably transfer their arms, generally seize one thing, generally don’t. They will vaguely imitate what a human operator confirmed them, however they’ll’t motive via a scenario they haven’t seen earlier than.
There are approaches that sit between conventional teleoperation and full autonomy – selfish video seize and units just like the Common Manipulation Interface (UMI), which lets operators reveal duties extra naturally by carrying a handheld gripper somewhat than controlling a robotic remotely.
These strategies scale back the burden on operators and produce considerably extra pure movement knowledge. They nonetheless require people within the loop, however they’re much less invasive, and for slim, well-defined duties, they are often helpful stepping stones. That mentioned, they don’t resolve the trade’s full dependency.
Reinforcement studying is what adjustments this. Quite than imitating what a human operator confirmed it, a system skilled with RL figures issues out via trial and error: making an attempt a activity, failing, adjusting, and attempting once more throughout hundreds of thousands of iterations, with out a human within the loop.
Simulation follows naturally from that; working hundreds of thousands of RL iterations in the true world destroys {hardware} and takes years. In simulation, you reset immediately, run in parallel, and generate variation at a scale no human workforce may match. And in contrast to teleoperation, it scales instantly with compute; extra GPUs imply extra environments, extra variation, and quicker iteration.
The folks doing teleoperation work need to know if autonomy is the precise objective, and so do the folks funding these initiatives. In the event you don’t have knowledge exhibiting the dependency on people reduces over time, teleoperation strikes from a stopgap to the everlasting methodology.
In regards to the writer
Nikita Rudin is co-founder and CEO of Flexion. Rudin accomplished his Ph.D. on the Robotic Methods Lab at ETH Zurich whereas working at NVIDIA, the place he targeted on large-scale reinforcement studying, management techniques, and robotic simulation.
At NVIDIA, Rudin was a part of the crew behind Isaac Health club and Isaac Lab, simulation instruments now extensively adopted throughout the robotics trade. Now he’s main Flexion, which lately raised $50 million from DST/NVentures to construct the general-purpose “mind” for humanoid robots.
