Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments

Overview§

Change2Task automatically generates executable training and evaluation tasks for coding agents from real repository history. Unlike prior approaches that rely on static or synthetic tasks, it anchors each task in a merged pull request (PR), ensuring that the task state, specification, and verification all reflect genuine developer intent and repository evolution.

Core Methodology§

The goal is to produce a task state $S$ on a modern healthy revision $M$ such that an agent solving $S$ should produce a patch equivalent to the original PR change $C$. To bridge historical changes with current code, the system uses three complementary strategies:

• Patch Reversal: Reverts $C$ on $M$ when the historical patch applies cleanly to the modern codebase.
• Code Mapping: Maps the historical diff to the corresponding modern code sections when direct reversal conflicts occur, preserving semantic alignment.
• Agent Reconstruction: When mapping fails, an LLM agent reconstructs the intended change by analyzing the original PR context and adapting it to the modern code state.

Each constructed task is validated through a lifecycle check: starting from a healthy base $B$, the system applies the reversal/mapping to create $S$, then runs an agent or test harness to produce a restored state $R$. The task passes if $R$ matches the target state (e.g., original tests pass, or the expected code change is recovered). This verification is formalized as:

$$ \text{Success}(S) = \mathbb{1}[\text{Test}(\text{Agent}(S)) \approx M] $$

Implementation Details§

The pipeline integrates with Git history and repository state reconstruction. It identifies eligible merged PRs by filtering for files related to coding tasks (e.g., Python, Java, TypeScript) and excludes PRs with insufficient test coverage. For each eligible PR, it attempts the three reconstruction strategies in order of fidelity, falling back to the next only when validation fails.

A key contribution is modern base reuse: multiple tasks sharing the same modern revision $M$ can be built simultaneously, reducing repeated environment setup and storage costs. The system also selects meaningful test commands from the repository's existing test suites, ensuring verification is reliable and practical.

The following simplified Python snippet illustrates the reconstruction workflow:

def build_task(repo, pr, strategy):
    if strategy == "reversal":
        task_state = repo.revert_patch(pr.patch)
    elif strategy == "mapping":
        task_state = repo.map_and_apply(pr.patch)
    elif strategy == "reconstruction":
        task_state = agent_reconstruct(repo, pr)
    else:
        raise ValueError(f"Unknown strategy: {strategy}")

    if repo.verify_lifecycle(task_state, pr):
        return task_state
    return None

Evaluation§

Change2Task is evaluated across five coding-agent task families: Bug Fix, Feature Addition, Test Generation, API Migration, and Security Repair. From 1,130 source changes, it achieves 79.6% verified task construction success. On a matched candidate set, it recovers 29.2% more verified tasks than a PR-based baseline. Historical and reconstructed cases reach up to 98.0% matched outcome agreement under agent evaluation, and reusing modern bases reduces measured pipeline expenditure by 10.8%.

These results demonstrate that repository history can serve as a scalable, high-quality source of executable tasks for coding-agent training and continuous evaluation, directly addressing the data scarcity bottleneck in agent development.