RSIGym: An Environment Built for Agents to Improve AI Systems

Existing research and engineering experiments on Recursive Self-Improvement (RSI) show that agents can improve AI systems autonomously. Yet much of their compute is consumed by environment setup, data preparation, deployment, and evaluation, while the model and its execution harness are difficult to optimize together.

Our team approached the problem through research infrastructure and built RSIGym. It packages training, inference, rollout, evaluation, and sandbox execution as callable services under Everything as a Service (EaaS). The environment supports Data, Harness, and Joint improvement tracks, allowing agents to focus limited compute on the decisions that matter most. This article explains the motivation, experimental scope, and selected high-level results.

Paper: https://arxiv.org/abs/2610.10310

1.Motivation: Directing Reasoning Compute Toward the Work of Self-Improvement

The vision of Recursive Self-Improvement (RSI) is for AI to act as an agent that can sustain its own research: identify capability bottlenecks in itself or a target system, propose hypotheses for improvement, design and run experiments, and use the results to decide what to try next. Formulating useful hypotheses, constructing training data, improving learning methods, refining execution frameworks, and analyzing results all require reasoning.

In practice, however, agents often have to complete substantial engineering work first: prepare data, build training pipelines, deploy checkpoints, launch task environments, and collect evaluation results. Dependencies, GPU memory allocation, and service scheduling consume the research agent's time and context. Every deployment error investigated, dependency conflict resolved, or environment rebuilt uses reasoning compute that could otherwise support experiment design and capability improvement.

Many of these capabilities can already be packaged as mature, reusable services. Having every research agent rebuild and integrate this infrastructure repeatedly consumes valuable reasoning tokens and experimental time. Our starting question is therefore: can we make the research environment more agent-native, so agents can readily call existing experimental capabilities and focus their limited reasoning compute (FLOPs) on research questions that require their judgment?

RSIGym is designed around agents' needs. Experimental capabilities should be easy to understand and invoke; lengthy jobs should run asynchronously; results should support diagnosis; and the platform should manage the infrastructure. These choices aim to reduce the engineering burden of experimentation, helping agents sustain a cycle of proposing hypotheses, running experiments, analyzing feedback, and revising their approach.

Reducing that burden also needs to preserve freedom to explore. Whether a model completes a task depends on its weights and on how it receives observations, calls tools, maintains context, handles failures, and decides that its work is finished. For example, the same model may perform quite differently when each command runs independently versus when it uses a shell that preserves state. Training data and settings influence what the model learns, while the execution framework determines how those capabilities are used in a task.

We therefore bring training data, training settings, and the execution framework into one research environment, allowing agents to decide where to intervene based on the problem they find. Researchers can allow changes to a single component to study its room for improvement, or open these components together to study how they interact and evolve.

RSIGym combines research infrastructure that agents can call directly with a configurable space for improvement. Within this environment, we ask: given the same starting system, explicit permissions, and a limited budget, how much can a research agent improve an AI system?

2.How the System Works

RSIGym adopts Everything as a Service (EaaS). Training, inference, data generation, evaluation, and sandbox computation are exposed as services, with accompanying skill documents. Agents use these services to conduct experiments autonomously, inspect the results, and decide what to try next.

The target of improvement is a system consisting of a model and its harness. The harness is the execution framework that organizes tool calls, manages context, and handles task execution. The research agent can revise training data, adjust training settings, and modify the harness, allowing these components to evolve together.

Budgets and permissions define the scope of each research run. The same environment can support experiments that change only data, only the harness, or both alongside training settings. At the end of a run, the agent submits its selected system, and a common evaluation protocol measures the improvement.

The research agent conducts experiments; the target system performs tasks. They can use different models, or the same model can attempt to improve itself.

3.Experiments and Findings

1.Comparing Autonomous Improvement Across Five Types of Tasks

We let several frontier research agents start from the same open-source base model and a simple harness. Within a given budget, they autonomously generate data, train models, and modify execution code. The tasks cover repository repair, terminal operations, mathematics, scientific questions, and skills in science, office work, and finance.

To summarize performance across domains, we introduce RSI-Index: the fraction of the remaining room for improvement closed by a research run in each domain, averaged across domains.

The results show that multiple agents achieve overall gains, with different strengths across tasks. To give a sense of scale, systems produced by one research agent improve repository repair from under 20% to around 50%, and mathematics from around 30% to nearly full marks. These systems are selected through independent research runs in each domain. The paper will present the full comparison.

2.Comparing Different Scopes for Improvement

We test changes to data alone, changes to the harness alone, and joint changes to data, training settings, and the harness.

On the repository repair and terminal tasks tested, joint optimization performs substantially better than changing data alone. Revising the execution framework while keeping model weights fixed also produces gains. These results show that research agents can find opportunities for improvement in both training and execution.

3.Autonomously Refining an Existing Framework

Alongside experiments starting from a simple harness, we give an agent an existing execution framework and let it read the code, analyze failure logs, make changes, and evaluate them.

With model weights held fixed, the mean success rate on terminal tasks increases by about ten percentage points, and more tasks are solved consistently across repeated attempts. Autonomous improvement can therefore also build on an existing system.

4.Varying Budgets, Network Access, and Research Models

We also run experiments with larger research budgets, restricted external network access, and an open-source model serving as both the researcher and the model being improved.

The outcomes vary. A larger budget does not always produce a higher score. An agent can complete the process of training, comparing candidates, and submitting a system without achieving an overall gain. The research records let us examine which experimental decisions produced improvements and which attempts fell short.