How Robots Learn: Machine Learning, VLA Models, and Industrial Automation
Industrial robots are gaining new ways to adapt. The engineering question is not whether learning will replace automation, but where it can make changeovers, variable tasks, and unstructured work more practical—without giving up accuracy, cycle time, safety, or maintainability.
This guide explains the main robot-learning approaches, the organizations shaping the field, and how training a robot differs from training a large AI model. It also looks at the sensing and motion-feedback requirements that remain essential when a learned skill moves from a demo to a production cell.
Conventional robots still excel at repeatable work
For welding, machine tending, dispensing, and assembly in a controlled cell, a conventionally programmed industrial robot remains an excellent tool. The process is explicit: engineers define coordinate frames, paths, speeds, I/O, interlocks, and recovery behavior. When the part, fixture, and sequence stay consistent, this approach is predictable and straightforward to validate.
Learning-based methods target a different part of the factory: tasks with variable object locations, frequent product changes, awkward contact, or instructions that are difficult to express as fixed waypoints. Learning can reduce some programming effort, but it does not remove the need for process engineering, fixtures, risk assessment, commissioning, or a plan for exceptions.
Deterministic automation
Precise paths, defined fixtures, and known sequences. A strong fit for stable products and high-volume cycles.
Teach and reuse
Capture a person’s demonstration or teleoperation data, then adapt the skill to reduce hand-coded motion.
Embodied intelligence
Combine perception, language, action, force feedback, and safety constraints in a closed loop.
Mobile medical workstations put flexibility in context
A mobile medical workstation is a useful example of a changing environment: people, carts, equipment, and access routes move throughout the day. A robot-assisted workflow may need to navigate around people, locate equipment, and perform a narrowly defined support task. These systems also face strict requirements for hygiene, patient safety, privacy, and predictable behavior. A video demonstration is a starting point for discussion, not evidence of clinical readiness or autonomous operation.
Mobile medical workstation — demonstration video supplied for this article.
What a robot coffee demonstration can—and cannot—show
A robot making pour-over coffee gives visitors an intuitive view of perception and motion working together: finding utensils, moving between task steps, controlling a pour, and responding to a scene. It raises practical questions that matter on a production floor: which variables are fixed, what happens when an object moves, how is task completion measured, and when does the system stop and ask for help?
ABB’s GoFa is a collaborative robot platform. ABB’s OmniCore EyeMotion is a vision toolbox for ABB OmniCore robots. Separately, Universal Robots and Scale AI announced the UR AI Trainer, a data-capture system for robot-learning workflows. These are distinct offerings from different companies; they should not be conflated. Nor should a polished demonstration be taken as proof of general-purpose autonomy in an uncontrolled environment.
ABB coffee robot — short demonstration.
ABB coffee robot — additional demonstration footage.
Four common routes for teaching robots
These approaches solve different problems. Real systems often combine demonstrations, simulation, learned perception, conventional motion planning, and hard safety limits rather than relying on one algorithm alone.
Imitation learning and teleoperation
Demonstrate → record state and action → learn a policy
- A person guides the robot using a controller, leader device, or remote interface.
- Useful for contact-rich manipulation and tasks that are awkward to program point by point.
- Data coverage, operator consistency, and recovery examples strongly affect results.
Reinforcement learning and sim-to-real
Define objective → practice in simulation → transfer → validate on hardware
- The policy improves through repeated trials against a reward or task objective.
- Simulation can produce large volumes of safe, repeatable training experience.
- Transfer gaps in friction, compliance, sensing, timing, and geometry still require attention.
Vision-language-action (VLA) models
Image + instruction + robot state → action sequence
- Connect visual observations and language instructions to robot actions.
- May generalize across objects or task wording when the training data supports it.
- Low-level control, latency, uncertainty, and safety usually need dedicated system layers.
World models and experience-based learning
Observe → predict outcomes → act → compare → update
- Learn a predictive model of how actions change the environment.
- Potentially supports planning and improvement from real operating experience.
- Reliable prediction under rare events and changing conditions remains a hard problem.
Experimental robot — demonstration video supplied for this article.
Who is advancing robot learning?
The field spans robot manufacturers, AI labs, simulation platforms, and specialist data companies. Their announcements point to several complementary routes—not one settled recipe for general-purpose industrial autonomy.
ABB
Combines industrial and collaborative robots with controller, programming, and vision tools. ABB describes OmniCore EyeMotion as a 2D/3D vision toolbox for OmniCore robots.
ABB OmniCore EyeMotion ↗Universal Robots + Scale AI
The UR AI Trainer is aimed at capturing synchronized motion, force, and visual data to support imitation-learning and VLA training workflows.
UR announcement ↗Google DeepMind
Gemini Robotics focuses on vision-language-action behavior, while its Robotics ER models target embodied reasoning and spatial understanding.
Gemini Robotics ↗NVIDIA
Provides simulation, robotics software, and physical-AI models and data intended to support robot learning and evaluation.
Physical AI models and data ↗Figure
Develops humanoid robot systems and a whole-body model approach that links locomotion and manipulation.
Figure Helix 02 ↗Physical Intelligence
Explores generalist robot policies and learning from experience, including reward-driven approaches.
π*0.6 research paper ↗Robot training is not the same as training an AI foundation model
Both use data and optimization, but the data, objective, and consequences of an error are different. A robot-learning system must ultimately produce physical action under timing, sensing, and safety constraints.
| Dimension | Robot skill / policy training | Large language or multimodal model training |
|---|---|---|
| Typical input | Camera, joint position, force/torque, tactile signals, task state, and operator demonstrations. | Large collections of text, images, audio, video, or other tokenized data. |
| Typical output | Actions, trajectories, grasp choices, motion policies, or task decisions tied to a robot and its environment. | Next-token predictions, generated text or media, and learned representations. |
| Training loop | Demonstration, simulation, offline data, and sometimes real-robot trials with resets and safety supervision. | Large-scale self-supervised pretraining, followed by instruction tuning, preference optimization, or task adaptation. |
| Cost of mistakes | May damage a part, tool, robot, or surrounding equipment; can create safety risks and downtime. | Usually a wrong or low-quality digital response, although downstream use can still carry risk. |
| Deployment constraints | Control frequency, latency, calibration, mechanical limits, workspace, cycle time, and fail-safe behavior. | Inference cost, response latency, context, reliability, and policy or product constraints. |
| What transfers? | Skills may transfer across similar robots or tasks, but geometry, payload, sensors, and mechanics matter. | Knowledge can transfer across text or multimodal tasks; physical grounding is not automatic. |
What still makes deployment difficult?
Data that covers the real job
Training data must include meaningful variation in parts, poses, lighting, contact, and failure cases. A narrow demo can produce a narrow skill.
Simulation-to-hardware gap
Small differences in friction, compliance, backlash, sensor alignment, cable routing, or timing can change the result on a physical machine.
Safety and predictable recovery
Plants need defined operating boundaries, fault detection, safe stops, and known behavior when the model is uncertain or the process goes off-nominal.
Position feedback under real loads
Learned perception does not replace accurate joint feedback. Repeatability, resolution, latency, thermal stability, mounting tolerance, and resistance to contamination still shape motion quality.
Likely direction of travel
| Near term | Next step | Longer-term goal |
|---|---|---|
| Teach specific skills with better data capture and validated operating envelopes. | Reuse policies across related parts, workcells, or robot configurations with targeted adaptation. | Systems that can reason about unfamiliar tasks while remaining measurable, controllable, and safe in production. |
Learning still depends on reliable position feedback
Whether the motion policy is programmed or learned, the robot needs to know where its joints and axes are. Encoder choice affects control bandwidth, angular or linear accuracy, integration space, environmental tolerance, interface compatibility, and the ability to detect faults. For robot joints, off-axis layouts and custom magnetic rings can be attractive when a conventional shaft encoder does not fit the mechanical envelope.
For rotary stages and linear axes, the right solution depends on the motion range, accuracy target, speed, mounting geometry, air gap, environment, and controller interface. OTV Sensing develops encoder and magnetic scale solutions for these integration needs. Start with the relevant product collection, then confirm the application details with our engineering team.
Frequently asked questions
Does a VLA model directly control the robot motors?
Usually not by itself. A VLA model may select a task-level action or motion target, while conventional controllers, planners, and safety systems handle real-time execution and enforce limits. The architecture depends on the robot and application.
Can a robot learn a task from ordinary online video?
Video can help teach visual context and task sequence. It often lacks synchronized robot states, force data, precise timing, and recovery behavior, so additional demonstrations or structured data are usually needed for reliable physical execution.
Why do encoders matter if the robot uses AI?
AI can interpret a task or choose an action, but the control loop still needs measured position and motion feedback. Encoder performance contributes to repeatability, stability, speed control, and fault detection.
What should be specified when selecting a robot encoder?
Start with axis geometry, required accuracy and resolution, speed, operating temperature, vibration and shock, mounting tolerances, electrical interface, cable or connector, and the controller’s diagnostics requirements. For magnetic rings, add pole or track configuration, diameter, runout, and sensor gap.
Thank you, ABB, for the coffee
Thanks to ABB for the robot-made pour-over coffee at the event. It tasted great—and made a memorable way to talk about the boundary between programmed motion and learned skills.
We welcome conversations with engineers and robotics teams about robot encoders, precision magnetic rings, and position-feedback solutions. Share the mechanical envelope, accuracy target, speed, environment, and interface requirements, and OTV Sensing can help evaluate a suitable approach.
Product selection and performance depend on the final mechanical, electrical, and environmental requirements. Confirm specifications and validation conditions for the intended application.