Skild AI has launched its S1 robot foundation model on NVIDIA infrastructure, training robotics systems using single video demonstrations.
The system uses video inputs to execute long-horizon, previously unseen tasks through in-context learning, operating without parameter weight updates or task-specific post-training.
Deepak Pathak, cofounder and CEO of Skild AI, said: “Learning by experience, and not preprogramming, is the step change that has happened in robotics. NVIDIA Isaac Lab and NVIDIA Cosmos technologies help Skild create the scalable, diverse experience its robots need to learn across many scenarios and embodiments.”
Pathak’s company reached a $100 million annual revenue run rate 10 months following its initial commercial rollout. Skild currently maintains over 60 deployment partnerships spanning manufacturing, logistics, inspection, security, and food preparation.
In-context learning from video prompts
Standard industrial deployments require fresh datasets, retraining, and validation whenever product specifications or factory layouts change.
S1 replaces this cycle by taking an operator’s video recording as a prompt. The model interprets intent, identified objects, and operational order, directly mapping them to physical robot actions without additional training runs.
The model handles unencountered tasks lasting up to 10 minutes, including pour-over coffee brewing, pancake preparation, kit assembly, and plant potting. These workflows involve dozens of manipulation steps requiring the system to combine skills in unprogrammed sequences.
During hardware testing for plant potting, Skild transferred an operator recording into autonomous physical execution in 11 minutes. The model also adjusts to moving objects and recovers from physical errors.
In multistep evaluations, S1 achieved an average per-step success rate of approximately 66 percent, compared to nine percent recorded by a baseline comparison system. Skild calculates that a single video demonstration provides operational utility equivalent to roughly 380 manual training examples, which typically require 50–100 hours of human collection.
Hardware deployment on Blackwell production lines
Foxconn, NVIDIA, and Skild are deploying the Skild Brain on dual-arm manipulators for assembly work on NVIDIA Blackwell systems.
The demonstrated assembly sequence requires the robot to install a busbar and limit block, secure 16 screws, track execution order, and adapt to physical disturbances or deviations from the planned environment.
NVIDIA accelerated computing powers the training of the shared model using teleoperation, human video feeds, physical simulations, and permitted deployment data.
Skild applies NVIDIA Cosmos foundation models to convert video into structured records and create varied training inputs, while Cosmos Curator annotates and filters data. Virtual environments from NVIDIA Omniverse and Isaac Sim supply synthetic data and test edge cases.
Isaac Lab provides reinforcement learning via the Newton physics engine to calculate contact, forces, collision, and pressure, reducing variance between simulation and physical hardware. NVIDIA Nsight tools analyse compute performance during training, and the TensorRT SDK handles inference optimisation on physical devices.
Skild and NVIDIA are currently engineering GPU-accelerated simulation solvers for Newton to calculate physical contact, gripping, and solid-object manipulation, with public availability planned for external developers.
Learn more about physical AI during the Physical AI Expo held in Amsterdam, London, and North America.
See also: Anthropic standard lets AI agents run lab and factory hardware
Want to learn more about the IoT from industry leaders? Check out IoT Tech Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events including AI & Big Data Expo and the Cyber Security Expo. Click here for more information.
IoT News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.
