Robotics Advances: Memory, Reasoning, and Real‑Time Action Models

The YC Robotics Club discussed the persistent challenge of solving robotics, noting that despite repeated predictions of breakthroughs, fully autonomous and versatile robots like "Rosie the robot" are still not a reality. The discussion highlighted several fundamental hurdles in robotics, presented new research aimed at overcoming these, and explored practical approaches for building successful robotics companies.

The Elusive Promise of Robotics

The speaker humorously pointed out that for the past decade, the robotics community has consistently believed that "next year, robotics will be solved." This sentiment has been fueled by advancements like AlphaGo, Mujoko simulations, and recent developments in diffusion policies and vision-action models (VALAs). Despite impressive demos of robots performing tasks like watering plants, fixing bikes, or making coffee, the widespread deployment of truly autonomous robots remains distant. The speaker noted that even in mid-2026 (humorously implying a future date), consumer robots like Neo 1X or Figure Robot are not readily available, and while some success exists in work cells, a general-purpose household robot is still absent.

Four Walls to Scale in Robotics

The speaker identified four major challenges that need to be overcome for robotics to advance significantly:

1. Physical Real-World Modeling (Sim-to-Real Gap): Current video models and world models, while effective in simulations (e.g., playing Doom), often fail to accurately respect real-world physics. For instance, a simulated car might drive through a grocery store and magically transition to a highway without crashing. This "sim-to-real gap" remains largely unsolved, especially when dealing with deformable objects.
2. Action Space Representation: Estimating the dynamics function (T of ST+1) conditioned on actions requires vast amounts of data. Learning quickly necessitates an effective representation for the action space, which is currently an unsolved problem.
3. Sensory-Motor Issue: Humans possess highly sophisticated sensory capabilities (detecting normal force, tangent force, moisture, temperature, vibration, friction) distributed throughout their bodies. Robots, in contrast, have limited tactile sensors, often just one on each fingertip, and perhaps a wrist camera. This lack of a comprehensive "epidermis" prevents robots from building rich world models without vision, unlike humans who can, for example, identify objects in a backpack purely by touch.
4. Embodiment Drift: This issue, particularly understood by those who deploy real robots for extended periods, refers to the degradation of actuator performance over time due to factors like dust, corrosion, or wear. This drift means that the same action might produce variable force or power, making teleoperation data quickly stale and requiring constant retraining.

Multiscale Embodied Memory (MAM)

Marcel, a PhD student from Chelsea Finn's lab, presented research on "Multiscale Embodied Memory" (MAM), developed during his internship at Physical Intelligence. The goal is to enable robots to perform long-horizon tasks (e.g., manufacturing, cleaning a bedroom, cooking a full meal) that require:

• Tracking task progress.
• Tracking time.
• Reliable and adaptive dexterity.

Marcel argued that memory is crucial for these tasks. Most current policies (like PIO5, Groot) lack memory, meaning the robot receives new observations at each time step without context. This leads to issues like a robot endlessly washing dishes or burning a grilled cheese sandwich because it doesn't remember how long it has been performing the action.

The challenges of adding memory are:

• Effectiveness: Memory can sometimes degrade performance due to distribution shifts and lack of data.
• Efficiency: Increasing context for robot policies is resource-intensive, leading to longer training and inference times.

MAM proposes a solution through compression, decomposing robot policies into:

• High-level policy: Predicts the next step for the low-level policy.
• Low-level policy: Executes robot actions.

Memory is also decomposed:

• Short-context memory: Dense frames for dexterous manipulation, fed into the low-level policy. This uses a new encoder based on a Vision Transformer (ViT) with temporal attention layers, dropping tokens except for the current image to achieve compression and fast inference. This enables tasks like making a grilled cheese (waiting as needed) or unloading groceries (remembering item locations and counts).
• Long-context memory: Compressed language representation of past events, fed into the high-level policy. This is a recurrent "memory scratchpad" that keeps track of what has happened, allowing the policy to modify its memory based on new events. This representation is more compressed than images, physically accurate, and less prone to distribution shifts. This enables tasks spanning tens of minutes, like preparing ingredients for a recipe.

A key benefit of adding memory is in-context adaptation. Policies without memory often get stuck in loops, repeating mistakes. With memory, robots can recognize and react to errors, such as adjusting their grip after failing to pick up a chopstick or switching sides to open a fridge.

A question was raised about how the textual long-term memory ensures it captures the "right" information. Marcel explained that currently, they rely on supervised fine-tuning (SFT) with human-annotated data, where annotators are instructed on what information is important. He suggested future work could involve reinforcement learning in the memory space to discover optimal information to track. Another question addressed the generalizability of textual descriptions for tasks. Marcel noted that the high-level policy, being a VLM trained on internet data, generalizes well with less data, and the low-level policy, while task-specific, only receives a small text description, reducing the need for extensive robot data.

Self-Supervised Bootstrapping of Action Predictive Embodied Reasoning

Milan Gennai, a PhD student working at Waymo, discussed how to move toward robots that teach themselves to reason. He explained that Vision-Action Models (VALAs) are powerful generalist policies, trained by fine-tuning vision-language models (VLMs) on scarce robotics datasets.

The recent trend is to leverage embodied reasoning for better action prediction, similar to "chain of thought" in LLMs. This is valuable because:

• Data scarcity: Reasoning provides richer training signals to augment limited robotics data.
• Interpretability: Textual reasoning traces allow humans to understand a robot's decision-making process.

Milan presented "R&B Encore" (Refine and Bootstrap Embodiment-Specific Chain of Thought Reasoning), a self-improving pre-training cycle. The core idea is to treat reasoning as an unobserved latent variable, leveraging variational inference. R&B Encore has two components:

1. Reasoning Proposer: An annotator model that suggests various types of reasoning (e.g., visible objects, movement, plans, gripper position) for a given demonstration.
2. Reasoning Validator: A scoring metric based on variational inference, evaluating reasoning traces for:
   • Concision: Short and not overly verbose.
   • Non-triviality: Encourages generalization.
   • Action Predictiveness: Ensures grounding in the embodiment.

This process generates a dataset of synthetic, action-aligned, and embodiment-aligned reasoning, which can then be used to retrain and improve embodied reasoning VALAs.

R&B Encore was tested across various embodiments:

• Manipulation: Found that "move" and "gripper position" reasoning were useful, while perceptual reasoning was not, often due to distracting objects. This improved success rates and object criticality rates.
• Hardware Manipulation: Improved out-of-distribution performance for novel objects and cluttered scenes.
• Legged Locomotion: Reasoning about structural affordances and movements was more important than terrains or counterfactuals.
• Self-Driving: Refined human annotations by pruning out unhelpful or "hallucinated" experiences, leading to lower L2 path errors and collision rates.

Key takeaways from R&B Encore:

• Selective reasoning is crucial: Exhaustive reasoning is not always useful.
• Self-supervised bootstrapping works: Addresses the "chicken and egg" problem of model and data sources.
• Generalizes across embodiments and model sizes: From 1 billion to 30 billion parameters.

Milan emphasized that embodied reasoning is not a fixed template but a resource to be discovered, deployed, and budgeted carefully. He also addressed the latency concern in autonomous vehicles, explaining that reasoning can introduce valuable priors during training, and an "action forcing" approach allows dropping reasoning during inference to avoid latency issues while retaining performance benefits.

Sim-to-Real Reinforcement Learning for Dexterous Manipulation

Tyler Lum presented "Sim-to-Real" and "Play to Perfect," focusing on achieving dexterous manipulation through simulation. He showcased a single policy operating at 60 Hz, controlling a 22-degree-of-freedom hand and a 7-degree-of-freedom arm, performing various dextrous tasks (e.g., screwdriver spinning) with novel tools and tasks zero-shot.

The challenge with dextrous hands is that teleoperation is difficult and unreliable for collecting demonstrations at scale due to embodiment mismatch and limited force feedback. Sim-to-Real RL addresses this by:

• Using GPU-accelerated simulation to run tens of thousands of robots in parallel, generating experience much faster than real-time.
• Scaling data collection with compute, not human effort.
• Optimizing for reward-maximizing behaviors, leading to extreme dexterity.

Most prior works learn separate policies for each skill (grasping, reorientation, tool use). Sim-to-Real aims for a single policy that controls both hand and arm through a full sequence (e.g., grasp, reorient, sweep). The key insight is to unify dextrous tool manipulation as goal reaching. The policy doesn't need task labels; it only needs to move an object from its current pose to a desired pose. This goal-conditioned policy is trained with massively parallel RL in simulation using procedurally generated primitive objects and random goals.

At inference time, the policy tracks a sequence of desired goal poses, which can be extracted from human video demonstrations using tools like FoundationPose and SAM. The human video only specifies the desired object trajectory, not robot actions, and is not used for training. The policy takes proprioception, current object pose, a bounding box for grasping, and the current goal pose, outputting joint position targets.

The single policy works zero-shot across 12 unseen tools and tasks, with new tasks simply becoming new sequences of goal poses. Performance is stronger for tools with long handles and weaker for heavier or smaller objects (due to pose tracking issues). Training on generic goal-reaching with primitive objects effectively generalizes to real-world tools and tasks.

Sim-to-Real outperforms baselines like fixed grasp (which causes collisions) and kinematic retargeting (which fails to grasp). Failure analysis showed that pose tracking failures dominate, followed by objects being dropped or failed grasps. The policy demonstrates strong recovery behaviors, such as regrasping a dropped hammer.

Tyler also introduced "Play to Perfect," a follow-up work that uses a pre-training/fine-tuning paradigm. It first learns a shared dextrous prior through task-agnostic play (similar to Sim-to-Real) and then fine-tunes this prior on sparse-reward, contact-rich assembly tasks, enabling zero-shot deployment for tasks like tight insertion and multi-part assembly.

A question about the impressive recovery behaviors revealed that while not explicitly trained for, random forces were applied to objects in simulation, forcing the robot to experience and recover from dropping items. Another question about the generalizability to non-tool objects was acknowledged as a challenge, with simulation limitations for complex objects like water or zippers. The use of LSTMs over transformers was attributed to the finicky nature of RL codebases and the observation that transformers don't always show improvement in this specific data regime.

Building Robotics Application Companies

Nico, co-founder and CEO of Rerun, discussed a successful pattern for robotics companies: focusing on "robotics application companies" or "neo-integrators." These companies take ownership of a full business problem end-to-end, excelling at operations, minimizing custom hardware, and often starting with teleoperation to validate the business model before scaling with models. He believes this category will transform the physical world economy, similar to how SaaS transformed software.

Nico outlined a three-part strategy for starting such a company:

1. Start with a single customer problem: Solve it with teleoperation and off-the-shelf hardware. This helps understand real-world business requirements and failure modes quickly. Examples include learning production quotas, acceptable failure rates, onboarding speed for new designs, and the impact of small design changes (e.g., a paper tray).
2. Set up basics for learning:
   • Fine-tune a model early: Start with a simple model (e.g., PI model) on a few hours of teleop data to understand the training process and how it impacts data collection.
   • Evaluate performance:
     • Have a replica of the customer's environment for testing.
     • Develop repeatable evaluation methods that encode business-critical metrics (e.g., paper plane sharpness, symmetry).
     • Track metadata and failure classifications.
   • Collect effective data: Data that improves the model. This requires constant testing, iterating, and debugging, often leading to changes in data collection instructions. Physical data is multimodal, multi-rate, episodic, and has complex 3D semantics, making traditional databases inefficient.
3. Hill climb, iterate, and scale: Once the foundation is in place, continuously improve intelligence (scaling data, algorithms, advanced data use, adding new sensors) and excel at sales, assembly, shipping, operations, and support. These non-modeling aspects often form the company's moat.

Nico emphasized that successful companies in this space iterate extremely fast, whether through heavy compute investment (frontier labs) or pragmatic, flexible systems with minimal moving parts (startups). He encouraged starting such companies, citing enormous markets, improving base models, inherent friction in the physical world for building moats, and the ability to start with relatively small capital while still requiring great AI and engineering.

He noted that early successes are often in areas that can be teleoperated, such as data centers, warehouse robotics, small-scale manufacturing, and food. The reason for the delayed emergence of these companies is the historical lack of affordable and capable hardware and base models, which are now becoming available. Regarding data scale, he suggested that if a task can be teleoperated, it can likely be learned by a model, and the exact data needs are discovered through iterative training and evaluation, treating autonomy as a scaling factor.

Real-Time World Action Models

Bill and Guan Ming from General Instinct discussed their work on infrastructure for running physical AI models quickly. They highlighted the shift from Vision-Action Models (VALAs) to World Action Models (WAMs).

• VALAs: Use a VLM to predict actions via an action head.
• WAMs: Use a central diffusion transformer to predict future kinematics and frames simultaneously. An example is Nvidia's Dream Zero, which performs well but is computationally expensive, requiring two A100 GPUs (around $70k each) to run, making it economically unscalable for the robotics industry.

The core problem with WAMs like Dream Zero is their reliance on diffusion models to predict frames, which is heavy. The training pipeline involves treating current observations as a condition for flow matching, adding noise to future latents, and training the model to predict future velocity fields. Inference is autoregressive, often taking 50-100 steps for accuracy. Each chunk prediction (e.g., 16 frames) requires running the Diffusion Transformer (DiT) 32 times due to classifier-free guidance (CFG).

To optimize WAMs, several approaches are being explored:

• Image-only prediction: Instead of predicting full video chunks, predict only the future end state (t+n frame).
• Latent world action models (Fast One): Assume world representations are learned in the hidden state of the DiT, eliminating the need for a decoder to reconstruct videos. The hidden state is used directly to condition the action head.

Comparing these, generative WAMs are like a VR Google Maps, while latent WAMs are like looking at Google Maps navigation and inferring the policy's intent. The challenge is to retain rich world representation efficiently. Approaches include:

• Asymmetrical denoising (Japa): Using two encoders (one for current, one for future observations) and an MSE loss to learn future prediction.
• Distributional prediction: Treating the future as a distribution of possibilities and estimating action distributions using flow matching.

General Instinct's infrastructure optimizes WAMs by:

• Distillation: On the VAE encoder/decoder and the DiT, making the DiT smaller.
• Dividing DiTs: Using separate DiTs and cross-attention from a video transformer to an action transformer, avoiding explicit future frame decoding.
• Accelerating flow matching: Reducing autoregressive flow matching sampling from 50-100 steps to 1-2 steps, achieving a 50x speedup without performance drops.
• Modality changes: Exploring more suitable modalities like masks or optical flow to represent and retain world representation.

Their infrastructure allows WAMs to run at 500 milliseconds per chunk (16 actions) on a single Jetson Orin.

A question about inference in WAMs, starting from a noisy space, was answered by explaining that the training process gradually adds noise to clean future latents, teaching the model to reverse this process during inference to reconstruct the clean latent from pure noise. This curriculum learning approach is why it's slow, and their distillation aims to speed it up.

Another question addressed whether WAMs' performance improvement comes from action heads seeing more details or if they develop a "theory of mind" for multiple agents. Guan Ming clarified that WAMs explicitly learn future kinematics by predicting future videos, teaching the model the correlation between physics at the pixel level and generated actions. Regarding multiple agents, the current discussion focused on single-agent scenarios.

The choice of business model (infrastructure for WAMs vs. robotics application company) was explained by Bill, who stated their belief in the importance of understanding the world for models and the need for infrastructure to enable these models to perform well in real-time on robots.