企业官网建设流程全解析

SEES: A SELF-EVOLVING EMBODIED SYSTEM VIA FAILURE-GUIDED VLA POLICY ADAPTATION

Ziwen Li^1,∗Hanlue Zhang^1,∗Zhenyang Ren^2,∗Tianyu Huang^2,∗Runqi Lin³Haoyu Wang¹Zhengqing Gao¹Yandong Guo Fakhri Karray^5,1Tongliang Liu²Chris Russell³Mingming Gong^6,1

¹Mohamed bin Zayed University of Artificial Intelligence²University of Sydney³University of Oxford⁴AI²Robotics⁵University of Waterloo⁶University of Melbourne^∗These authors contributed equally and share first authorship.

ABSTRACT

Recent vision-language-action (VLA) policies demonstrate promising generalization across diverse short-horizon tasks. However, they remain unreliable on longhorizon tasks, partly because the large-scale training data is biased toward singlestage manipulation tasks that are cheaper to demonstrate. A single weak atomic skill can cause failures across multiple multi-stage tasks. To address such failures, existing methods often require experts to identify the bottleneck and provide additional demonstrations, making the improvement costly and potentially impractical after deployment. To this end, we present a Self-Evolving Embodied System (SEES) that learns from failures and improves the VLA policy without additional expert demonstrations. SEES decomposes long-horizon tasks into atomic tasks and routes them to corresponding family policies. Each family consists of related atomic skills that share one VLA adapter. During execution, the system automatically monitors atomic-task outcomes to identify the most frequently failing atomic skills as the current bottlenecks. To overcome these bottlenecks, SEES constructs tailored RL tasks in simulation by restoring previously encountered states and generating task-specific success criteria with an LLM. Online RL updates the shared family adapters to promote positive transfer among related atomic skills and cumulative improvement across evolution rounds. Extensive experiments show that SEES can be integrated with different VLA backbones to progressively improve their long-horizon performance. We also observe continued improvement on unseen tasks, providing evidence of transfer beyond the evolution settings.

SEES：通过失败引导的 VLA 策略适配实现自演化的具身系统

摘要

近期的视觉-语言-动作（Vision-Language-Action, VLA）策略在多种短时程任务上展现出了良好的泛化能力。

然而，这些策略在长时程任务上仍然缺乏可靠性，部分原因在于大规模训练数据偏向于示范成本较低的单阶段操作任务。

单个薄弱的原子技能（Atomic Skill）就可能导致多个多阶段任务失败。

为了解决此类失败问题，现有方法通常需要专家识别瓶颈并提供额外的示范数据，这使得性能改进成本高昂，并且在部署后可能难以实际实施。

为此，我们提出了一个自演化具身系统（Self-Evolving Embodied System, SEES），该系统能够从失败中学习，并在无需额外专家示范的情况下改进 VLA 策略。

SEES 将长时程任务分解为原子任务，并将其路由至相应的技能族策略（Family Policies）。

每个技能族由一组相关的原子技能组成，这些技能共享一个VLA 适配器（VLA Adapter）。

在执行过程中，系统会自动监测原子任务的执行结果，并将失败最频繁的原子技能识别为当前的瓶颈。

为了克服这些瓶颈，SEES 通过在仿真环境中恢复先前遇到的状态，并利用大型语言模型（Large Language Model, LLM）生成特定于任务的成功判据，从而构建定制化的强化学习（Reinforcement Learning, RL）任务。

在线强化学习（Online Reinforcement Learning）更新共享的技能族适配器，以促进相关原子技能之间的正向迁移（Positive Transfer），并实现跨演化轮次的累积改进。

大量实验表明，SEES 可以与不同的 VLA 主干模型集成，逐步提升它们在长时程任务上的性能。我们还观察到模型在未见过的任务上持续取得性能提升，这为其具备超出演化设置范围的迁移能力提供了证据。

1 INTRODUCTION

Recent VLA policies leverage large-scale vision-language pretraining and diverse robot trajectories to perform a broad range of robot tasks (Zitkovich et al., 2023; Kim et al., 2025; Black et al., 2025b;a; NVIDIA et al., 2025; Open X-Embodiment Collaboration, 2024; Walke et al., 2023; Khazatsky et al., 2024). However, long-horizon execution remains unreliable, partly because collecting multi-stage demonstrations is costly and many existing robot datasets focus on short-horizon manipulation (Garrett et al., 2025; Ceola et al., 2024). A household mobile-manipulation task may require multiple navigation and object-interaction stages, and full-task success depends on completing every required stage (Ichter et al., 2023; Gu et al., 2023; Liang et al., 2023). Consequently, weaknesses in a few shared atomic capabilities can cause failures across a wide range of long-horizon tasks.

One way to address these failures is through human-in-the-loop policy improvement. Experts review performance across tasks and inspect failed executions to identify bottlenecks, then collect targeted demonstrations to improve weak capabilities and overall task success. Intervention-based methods aim to use human effort more efficiently by having operators take over only when the policy is at risk of failure (Kelly et al., 2019; Mandlekar et al., 2020). To further reduce the need for constant human monitoring, some methods automatically detect potential problems and request human intervention (Hoque et al., 2022; Liu et al., 2024; 2025). These advances make human supervision more efficient, but collecting sufficient data for supervised fine-tuning (SFT) still requires substantial human effort. Moreover, for commercial robots deployed in users’ homes, repeatedly collecting data for specific tasks and environments may be impractical, limiting continued improvement when expert support is unavailable. We therefore ask whether a robot can autonomously identify its bottlenecks and improve its policy accordingly without additional expert demonstrations.

1 引言

近期的视觉-语言-动作（Vision-Language-Action, VLA）策略利用大规模视觉-语言预训练以及多样化的机器人轨迹数据，来完成广泛的机器人任务（Zitkovich et al., 2023；Kim et al., 2025；Black et al., 2025b;a；NVIDIA et al., 2025；Open X-Embodiment Collaboration, 2024；Walke et al., 2023；Khazatsky et al., 2024）。

然而，长时程任务执行（long-horizon execution）仍然不够可靠，部分原因在于：收集多阶段示范的成本较高，而且许多现有机器人数据集主要关注短时程操作（short-horizon manipulation）（Garrett et al., 2025；Ceola et al., 2024）。

一个家庭移动操作任务可能需要多个导航和物体交互阶段，而完整任务的成功依赖于每一个必要阶段都顺利完成（Ichter et al., 2023；Gu et al., 2023；Liang et al., 2023）。

因此，一些共享的基础原子能力中的薄弱环节，可能会导致大量长时程任务的失败。

解决这些失败的一种方法是采用人在回路中的策略改进（human-in-the-loop policy improvement）。

专家会跨任务评估机器人的表现，并检查失败的执行过程以识别瓶颈，然后收集有针对性的示范数据，以提升薄弱能力和整体任务成功率。

基于干预的方法（intervention-based methods）试图更高效地利用人力：仅在策略面临失败风险时，才由人工操作员接管（Kelly et al., 2019；Mandlekar et al., 2020）。

为了进一步减少持续人工监控的需求，一些方法能够自动检测潜在问题，并请求人工干预（Hoque et al., 2022；Liu et al., 2024；2025）。

这些进展提高了人工监督的效率，但为了进行监督微调（supervised fine-tuning, SFT）而收集足够的数据，仍然需要大量人工投入。

此外，对于部署在用户家中的商业机器人而言，反复收集针对特定任务和环境的数据可能并不现实；当无法获得专家支持时，这也限制了系统的持续改进能力。

因此，我们提出这样一个问题：机器人是否能够在不依赖额外专家示范的情况下，自主识别自身瓶颈，并据此改进其策略？

Figure 1: Human-supervised adaptation versus SEES self-evolution. (a) Related atomic failures can block progress across different long-horizon tasks. (b) Human-supervised adaptation relies on expert diagnosis and additional demonstrations. © SEES automatically identifies recurring bottlenecks, constructs targeted RL tasks by restoring encountered states and generating success criteria, and updates shared family adapters.

图 1：人类监督适配（Human-supervised Adaptation）与SEES 自演化（SEES Self-Evolution）的对比。(a) 相互关联的原子级失败（Atomic Failures）可能阻碍不同长时程任务（Long-Horizon Tasks）的执行进展。(b) 人类监督适配依赖专家诊断和额外的示范数据。© SEES 自动识别反复出现的瓶颈（Bottlenecks），通过恢复执行过程中遇到的状态并生成成功判定标准，构建有针对性的强化学习任务（Reinforcement Learning Tasks, RL Tasks），并更新共享的任务族适配器。

We introduce SEES, a self-evolution framework that can be integrated with different VLA backbones (Fig. 1). It decomposes long-horizon tasks into atomic tasks and executes each using the VLA adapter shared by its skill family. During execution, a learned termination head provides completion signals for timely stopping and outcome monitoring. By aggregating atomic-task outcomes across executions, SEES identifies the bottlenecks that most frequently prevent full-task completion.

For each bottleneck, SEES uses an LLM-based task builder to automatically construct a targeted RL task in simulation. The builder restores previously encountered states as initial conditions and generates task-specific success criteria for computing sparse rewards. Rather than training a separate adapter for each atomic task, SEES jointly trains a shared adapter on bottlenecks from the same family, allowing related tasks to benefit even when they are not selected for RL. We use low-rank adaptation (LoRA) to store these family-level updates without maintaining a full VLA model for each family (Hu et al., 2022). Checkpoints are selected for reuse based on their validation gains on the chosen bottleneck tasks. Repeating this evolution cycle progressively strengthens weak atomic skills and improves overall performance on long-horizon tasks.

We evaluate SEES on RoboCasa365 (Nasiriany et al., 2026) using GR00T N1.5 (NVIDIA GEAR, 2025) and RLDX-1 (Kim et al., 2026), and further examine it across all four LIBERO suites (Liu et al., 2023). On RoboCasa365, both backbones achieve higher atomic and long-horizon success after three evolution rounds. More notably, on Composite-Unseen, which contains entirely new tasks, success increases from 3.5% to 6.1% for GR00T N1.5 and from 6.0% to 7.1% for RLDX-1. On LIBERO, failure-guided policy evolution improves success across all four suites, raising the average from 53.1% to 65.4% after three rounds and benefiting tasks never selected for evolution. Further analyses validate termination-based failure attribution, failure-based bottleneck selection, and cross-task transfer through family-level adaptation. Together, these results demonstrate the effectiveness of SEES across different VLA backbones and benchmarks, as well as its ability to generalize to unseen tasks. This paper makes three main contributions:

• We introduce SEES, a self-evolving embodied system compatible with different VLA backbones. It automatically identifies atomic bottlenecks and constructs targeted simulation RL tasks for policy improvement without additional expert demonstrations.

• We develop a failure-guided family-level evolution mechanism that jointly adapts related bottlenecks selected from long-horizon execution failures. The resulting updates strengthen capabilities shared across the family and can benefit its atomic skills collectively.

• Experiments across VLA backbones and benchmarks demonstrate progressive policy improvement, with gains extending to unseen tasks. Targeted analyses further validate the effectiveness of failure attribution, bottleneck selection, and family-level adaptation.

我们提出了 SEES，一种能够与不同视觉-语言-动作模型骨干网络（Vision-Language-Action Backbones, VLA Backbones）集成的自演化框架（Self-Evolution Framework）（图 1）。该框架将长时程任务分解为原子任务（Atomic Tasks），并使用各原子任务所属技能族（Skill Family）共享的 VLA 适配器执行每个原子任务。在执行过程中，一个经过学习的终止预测头（Termination Head）提供任务完成信号，以支持及时停止执行和结果监控。通过汇总多次执行中的原子任务结果，SEES 能够识别最频繁阻碍完整任务完成的瓶颈。

对于每个瓶颈，SEES 使用基于大语言模型（Large Language Model, LLM）的任务构建器，在仿真环境中自动构建有针对性的强化学习任务。

该构建器将先前遇到的状态恢复为初始条件，并生成任务特定的成功判定标准，以计算稀疏奖励（Sparse Rewards）。

SEES 并非为每个原子任务训练独立的适配器，而是针对同一技能族中的多个瓶颈联合训练一个共享适配器，使相关任务即使未被选中进行强化学习，也能够从中受益。

我们使用低秩适配（Low-Rank Adaptation, LoRA）来存储这些技能族级别的更新，而无需为每个技能族维护一个完整的 VLA 模型（Hu et al., 2022）。

根据模型在选定瓶颈任务上的验证性能提升情况，选择可供复用的检查点（Checkpoints）。通过重复这一演化循环，系统逐步增强薄弱的原子技能，并提升长时程任务的整体执行性能。

我们在 RoboCasa365（Nasiriany et al., 2026）上使用 GR00T N1.5（NVIDIA GEAR, 2025）和 RLDX-1（Kim et al., 2026）评估 SEES，并进一步在全部四个 LIBERO 任务套件（Liu et al., 2023）上进行研究。

在 RoboCasa365 上，经过三轮演化，两种骨干模型的原子任务成功率和长时程任务成功率均有所提高。

更值得注意的是，在包含全新任务的 Composite-Unseen 测试集上，GR00T N1.5 的成功率从 3.5% 提升至 6.1%，RLDX-1 的成功率从 6.0% 提升至 7.1%。

在 LIBERO 上，失败引导的策略演化（Failure-Guided Policy Evolution）提高了全部四个任务套件的成功率，经过三轮演化，平均成功率从 53.1% 提升至 65.4%，且从未被选中参与演化的任务也从中受益。

进一步的分析验证了基于终止信号的失败归因（Failure Attribution）、基于失败的瓶颈选择（Bottleneck Selection），以及通过技能族级别适配（Family-Level Adaptation）实现的跨任务迁移（Cross-Task Transfer）。

这些结果共同证明了 SEES 在不同 VLA 骨干模型和基准测试上的有效性，以及其对未见任务的泛化能力。本文的主要贡献有以下三点：

• 我们提出了 SEES，一种兼容不同 VLA 骨干模型的自演化具身系统（Self-Evolving Embodied System）。该系统能够自动识别原子任务瓶颈，并构建有针对性的仿真强化学习任务，在无需额外专家示范的情况下改进策略。

• 我们开发了一种失败引导的技能族级别演化机制，对从长时程任务执行失败中选出的相关瓶颈进行联合适配。由此产生的更新能够增强技能族内共享的能力，并使该技能族中的各项原子技能共同受益。

• 在不同 VLA 骨干模型和基准测试上的实验表明，该方法能够实现渐进式策略改进，且性能提升能够扩展至未见任务。针对性分析进一步验证了失败归因、瓶颈选择和技能族级别适配的有效性。

2 RELATED WORK

Autonomous Robot Self-Improvement. A shared goal is to let robots improve from execution experience without repeatedly collecting expert demonstrations. ASPIRE repairs robot programs and accumulates reusable skills, but the behaviors it can express remain constrained by the available perception and control primitives (Lu et al., 2026). Q-Planning learns from deployment rollouts to improve action selection, but relies on its frozen policy to propose useful actions (Giridhar et al., 2026). Direct VLA adaptation offers a complementary route by using execution feedback to improve action generation, rather than only program structure or action selection.

Other systems improve policies through autonomous data collection. SOAR selects feasible tasks to diversify experience, while RoboClaw repeatedly collects trajectories to refine manipulation policies (Zhou et al., 2025a; Li et al., 2026b). These approaches reduce the effort needed to acquire training data. Broader experience, however, does not necessarily prioritize the skills that most frequently prevent long-horizon tasks from succeeding. This distinction matters when limited training resources must be allocated across many skills.

The cost of repeated practice is another consideration. RISE and VLAW reduce physical interaction through world-model rollouts, which require learned models that reliably predict interactions and their outcomes (Yang et al., 2026; Guo et al., 2026). Given an existing simulator, SEES records states during execution and restores them later for targeted training based on accumulated failure statistics. The resulting updates improve subsequent executions.

Targeted RL Task Construction. RLinf-VLA and SimpleVLA-RL fine-tune VLA policies with online RL on given tasks and reward definitions (Zang et al., 2026; Li et al., 2026a). Eureka and Text2Reward generate reward programs, RoboGen and OMNI-EPIC generate tasks and environments, and ReinforceGen combines long-horizon task decomposition, automated data generation, and RL (Ma et al., 2024; Xie et al., 2024; Wang et al., 2024; Faldor et al., 2024; Zhou et al., 2025b). SAFE detects failures, SeqVLA predicts completion, and BATON localizes stage failures without updating the policy (Gu et al., 2025; Yang et al., 2025; Xu et al., 2026). SEES aggregates stage-level outcomes across rollouts to choose which atomic skill to train, restores states encountered before that skill, and generates a Boolean success predicate to define its sparse reward.

Skill-Structured Policy Adaptation. Modular adaptation balances preserving existing capabilities against sharing improvements. CORAL isolates task-specific LoRA experts to prevent parameter interference, but an update to one expert does not directly improve the others (Luo et al., 2026). CLARE reuses and expands adapters, while Stellar VLA organizes shared knowledge through task– skill structure (Romer et al.¨ , 2026; Wu et al., 2025). OrthoSkillVLA constrains parameter updates and maintains skill-specific experts to protect previously learned skills (Wang et al., 2026). These methods primarily address how to incorporate new task or skill data while retaining previous abilities. SEES builds on structured parameter sharing but determines its training targets from recurring execution failures rather than incoming demonstration datasets. Related bottlenecks jointly train one family adapter, and the updated adapter is reused by all tasks assigned to that family. This allows targeted practice to benefit related tasks not selected for RL, without maintaining and training a separate adapter for every atomic task.

2 相关工作

自主机器人自我改进（Autonomous Robot Self-Improvement）。一个共同的目标是让机器人能够从执行经验中持续改进，而无需反复收集专家示范。

ASPIRE 通过修复机器人程序并积累可复用技能来实现改进，但其能够表达的行为仍受到现有感知与控制原语（Perception and Control Primitives）的限制（Lu et al., 2026）。

Q-Planning 从部署阶段的执行轨迹（Deployment Rollouts）中学习，以改进动作选择，但它依赖其冻结策略（Frozen Policy）来提出有用的动作（Giridhar et al., 2026）。

VLA 直接适配（Direct VLA Adaptation）提供了一条互补路径，即利用执行反馈改进动作生成，而不仅仅是改进程序结构或动作选择。

其他系统通过自主数据收集（Autonomous Data Collection）来改进策略。SOAR 选择可行任务以增加经验的多样性，而 RoboClaw 则通过反复收集轨迹来优化操作策略（Zhou et al., 2025a; Li et al., 2026b）。这些方法减少了获取训练数据所需的工作量。然而，更广泛的经验并不一定能够优先针对那些最频繁阻碍长时程任务成功的技能。当有限的训练资源必须分配给多种技能时，这一区别尤为重要。

重复练习的成本是另一个需要考虑的因素。RISE 和 VLAW 通过世界模型推演（World-Model Rollouts）减少物理交互，但这要求学习得到的模型能够可靠地预测交互过程及其结果（Yang et al., 2026; Guo et al., 2026）。在已有仿真器的条件下，SEES 在执行过程中记录状态，并在之后恢复这些状态，根据累积的失败统计信息进行有针对性的训练。由此产生的更新能够改进后续执行表现。

针对性强化学习任务构建（Targeted RL Task Construction）。RLinf-VLA 和 SimpleVLA-RL 在给定任务和奖励定义的条件下，使用在线强化学习（Online RL）对 VLA 策略进行微调（Zang et al., 2026; Li et al., 2026a）。

Eureka 和 Text2Reward 生成奖励程序，RoboGen 和 OMNI-EPIC 生成任务与环境，而 ReinforceGen 则结合长时程任务分解、自动化数据生成和强化学习（Ma et al., 2024; Xie et al., 2024; Wang et al., 2024; Faldor et al., 2024; Zhou et al., 2025b）。

SAFE 检测失败，SeqVLA 预测任务完成情况，而 BATON 在不更新策略的情况下定位阶段性失败（Gu et al., 2025; Yang et al., 2025; Xu et al., 2026）。

SEES 汇总多次执行轨迹中的阶段级结果（Stage-Level Outcomes），以决定训练哪项原子技能，恢复执行该技能之前遇到的状态，并生成布尔成功谓词（Boolean Success Predicate）来定义其稀疏奖励。

技能结构化策略适配（Skill-Structured Policy Adaptation）。
模块化适配（Modular Adaptation）需要在保留现有能力与共享改进成果之间取得平衡。CORAL 隔离任务特定的LoRA 专家模块（Task-Specific LoRA Experts），以防止参数干扰（Parameter Interference），但对某个专家模块的更新不会直接改进其他专家模块（Luo et al., 2026）。

CLARE 复用并扩展适配器，而 Stellar VLA 则通过任务—技能结构（Task–Skill Structure）组织共享知识（Romer et al.¨ , 2026; Wu et al., 2025）。OrthoSkillVLA 约束参数更新，并维护技能特定的专家模块（Skill-Specific Experts），以保护先前学到的技能（Wang et al., 2026）。

这些方法主要解决如何在保留已有能力的同时，整合新的任务或技能数据。SEES 以结构化参数共享（Structured Parameter Sharing）为基础，但其训练目标由反复出现的执行失败决定，而不是由新引入的示范数据集决定。

相关瓶颈共同用于训练一个技能族适配器，更新后的适配器由分配给该技能族的所有任务复用。

这使得有针对性的练习能够惠及未被选中进行强化学习的相关任务，而无需为每个原子任务分别维护和训练独立的适配器。

3 METHOD

SEES follows a repeated execution-and-adaptation loop (Fig. 2). It first decomposes long-horizon tasks, executes them with family policies, and aggregates atomic outcomes to identify recurring bottlenecks (Sec. 3.1). For each selected bottleneck, an LLM-based task builder restores encountered states and generates a success function to construct a targeted simulation RL task (Sec. 3.2). Online

Figure 2: The SEES execution–evolution loop. In the top row, SEES decomposes long-horizon instructions, executes atomic tasks with family policies, and identifies bottlenecks using learned termination and outcome statistics. In the bottom row (right to left), restored states and LLMgenerated success criteria define targeted simulation RL tasks. Online RL jointly trains each family adapter on its selected bottleneck tasks. The checkpoint with the largest positive gain on bottlenecktask validation is registered and returned to execution.

RL then updates the shared adapters of the affected skill families, and the checkpoint with the largest positive target-task gain is registered for subsequent execution (Sec. 3.3).

3.1 EXECUTION AND BOTTLENECK IDENTIFICATION

Given a long-horizon instruction c and initial-scene observations, a frozen VLM planner produces an ordered sequence of executable atomic tasksAc.A _ { c } .Ac​.A language agent maps each atomic task i to a predefined skill familyg(i)g ( i )g(i)according to the capability it requires. Tasks within a family retain their individual instructions but share the same VLA adapter, allowing related tasks from different composite plans to reuse the family’s current policy. We denote the frozen pretrained VLA parameters by θ and the adapter shared by family f byϕf\phi _ { f }ϕf​. The policyπθ,ϕg(i)\pi _ { \boldsymbol { \theta } , \boldsymbol { \phi } _ { g ( i ) } }πθ,ϕg(i)​​executes task i conditioned on its atomic instruction and current observations. Before an updated adapter becomes available for a family, its tasks are executed by the pretrained base VLA. Appendix A provides the planner configuration and skill-family definitions.

To stop completed subtasks promptly and provide stage boundaries for failure attribution, we attach a termination head to the pretrained VLA and train it to predict atomic task completion, while keeping all original parameters frozen. During long-horizon task execution, a predicted termination ends the current motion and allows the next atomic task to begin. A subtask is marked as failed if no termination signal is emitted before the step limit or unsuccessful environment termination. The audit in Appendix B supports using termination signals for failure attribution. We then aggregate these outcomes across rollouts to identify recurring bottlenecks.

For each failed rollout of composite task c, we assign the failure to the first reached atomic stage i that prevents execution from continuing. Downstream stages blocked by this failure are not counted. LetFc,iF _ { c , i }Fc,i​denote the number of failed rollouts assigned to stage i. At each evolution round, we select the stage with the highest failure count as the bottleneckbcb _ { c }bc​for each composite task:

bc=arg⁡max⁡i∈AcFc,i.(1) b _ {c} = \arg \max _ {i \in \mathcal {A} _ {c}} F _ {c, i}.\tag{1}bc​=argi∈Ac​max​Fc,i​.(1)

We select one bottleneck per composite task each round to focus the fixed adaptation budget on its most frequent blocker. Selecting bottlenecks before grouping them by skill family keeps the decision sensitive to specific weak stages, even when other tasks in the family are reliable.

3.2 BOTTLENECK-TARGETED RL TASK CONSTRUCTION

For each selected bottleneck task i, the RL task builder supplies two components: initial states that reproduce its execution context and a success predicate for sparse rewards. We collect candidate states immediately before i begins in composite-task rollouts, whether the attempt subsequently succeeds or fails. Restoring the scene configuration and task-dependent episode state allows training to start directly at the bottleneck without replaying earlier stages (Appendix C.1).

An LLM generates the success predicateCi(s)C _ { i } ( s )Ci​(s), a Boolean success-checking function, from the task instructionli,l _ { i } ,li​,the identifiers of relevant objects and fixtures, and documentation for simulator functions that query their states and relations. The function returns whether the atomic task is complete and is used to compute sparse rewards. States that cannot be restored or already satisfyCiC _ { i }Ci​are excluded, leaving the initialization setTi\mathcal { T } _ { i }Ti​. Appendix C.2 provides further details and separately evaluates agreement with official success predicates. Reward construction therefore relies on the generated function and documented simulator interfaces rather than privileged task oracles, which supports extending the same pipeline to simulators reconstructed from real-world execution traces.

We keep the simulator dynamics unchanged and define the RL task for atomic task i as

Ti=(li,μi0,ri),μi0=Uniform(Ii),(2) \mathcal {T} _ {i} = (l _ {i}, \mu_ {i} ^ {0}, r _ {i}), \qquad \mu_ {i} ^ {0} = \mathrm{Uniform} (\mathcal {I} _ {i}),\tag{2}Ti​=(li​,μi0​