Budapest · Oct 24–29 emnlp.org

EMNLP 2026 Industry Track

Repo2ROCm

An agentic system for reproducing research code on AMD ROCm GPUs.

Paper Poster GitHub

Gaurav Sarkar^1,† Umang Pandey² Pratik Prabhanjan Brahma² Zicheng Liu² Emad Barsoum²

¹IIIT Hyderabad ²Advanced Micro Devices, Inc. ^†Work done while at AMD

gaurav.sarkar@research.iiit.ac.in
{upandey, pprabhan, Zicheng.Liu, ebarsoum}@amd.com

59/60repositories verified on AMD GPUs

3.3×over 18/60 running out of the box

77.3Portability Fidelity Score, vs. 61.4 for Claude Code

18–20 minGitHub URL to a verified GPU run

01 — Abstract

Public ML repositories are how industry teams decide whether a method is worth adopting. Running them is rarely a clone-and-execute exercise, and it gets harder when the code assumes CUDA and the machine in front of you is an AMD GPU.

Repo2ROCm takes a repository URL and migrates that CUDA-centric codebase onto ROCm with no manual steps. It rebuilds the environment, adapts device assumptions and custom kernels, runs a real workload, and accepts success only when runtime evidence shows the workload executed on the AMD GPU. Installation, an import, or a --help screen is not enough.

We evaluate on ResearchBench, 60 repositories from recent ICLR, ICML, ACL, and NeurIPS papers. Eighteen run out of the box. Repo2ROCm verifies functional reproduction for 59. Against general-purpose coding agents that share the same Claude Sonnet 4 backbone, it also preserves the accelerator path more faithfully.

A repository that runs only after disabling FlashAttention, falling back to CPU, or shrinking the workload is no longer the experiment in the paper.
What Mode 1 refuses to count

02 — System

Plan outside Docker. Execute behind hard guards.

A GitHub URL goes in. A verified AMD-GPU run comes out, with a Dockerfile, a source patch, command logs, verification evidence, and the full agent trajectory.

Repo2ROCm system diagram. Planning phase: input a GitHub URL and optional paper, clone and fingerprint the repository, then a static planner ranks ROCm base images and writes a plan. Execution phase: a kernel agent converts CUDA to HIP, a guarded ReAct executor runs inside a Docker sandbox with hard guards, an observer sidecar, and causal memory, then a deterministic verifier checks GPU execution and, in Mode 2, paper metrics. Outputs are a Dockerfile, a unified diff, command logs, a SuccessReport, and the agent trajectory. — **Figure 1.** Planning fingerprints the repository and ranks ROCm base images. Execution is a guarded ReAct loop inside a GPU-pinned Docker sandbox. The verifier, not the language model, owns the success claim. Select the figure to enlarge it.

03 — Method

Evidence-gated migration, in three modes.

Mode	What it asks	Outcome
Mode 0	Clone and follow the repository’s own instructions. No agent.	18/60
Mode 1	Functional reproduction. Real workload, on the AMD GPU, with the accelerator path intact.	59/60
Mode 2	After Mode 1, compare measured metrics with the paper’s targets and tolerances.	1 of 2 cases

Planning · outside Docker

Fingerprint, then one plan.

Clone the repository and detect frameworks, dependency files, entry points, CUDA packages, and custom .cu kernels.
Rank candidate ROCm base images. Package overlap is a Jaccard score over normalized dependency tokens, with filters for architecture, GPU family, and images that have already failed.
Emit a single plan: install path, migration hazards, workload command, and validation checks. Mode 2 also records a metric and a tolerance when both are unambiguous.

Execution · inside the sandbox

One action, then proof.

A ReAct executor follows the plan one action at a time in a GPU-pinned container. Successful milestones are checkpointed; failures roll back.
Custom CUDA extensions go through a kernel agent: CUDA-to-HIP conversion, including hipify, then a build-and-load check.
Hard guards block an image switch, an accelerator-library swap, a critical source edit, or a success claim until deterministic tools supply runtime evidence.

A run counts only if all four hold.

01An official workload finishes: inference, training, evaluation, or a forward pass.
02Runtime evidence shows that workload ran on AMD hardware through ROCm.
03Accelerator-critical pieces stay in place. Attention backends, custom kernels, and GPU libraries are not disabled, stubbed, or swapped for a degraded path such as SDPA in place of FlashAttention.
04No unreported CPU fallback, backend substitution, disabled module, or toy-scale run.

The language model cannot override that verdict. An observer sidecar breaks repeated failure loops, and causal memory offers hints from earlier repositories. Distilled hints never bypass the guards.

04 — Results

ResearchBench, on eight MI350X GPUs.

18/60Mode 0, out of the box

3.3×improvement

59/60Mode 1, Repo2ROCm

The misses out of the box are the usual research-code failures plus ROCm-specific ones: CUDA-only dependencies, the wrong container, hard-coded CUDA checks, silent CPU fallback, stale APIs, and small source bugs. The single Mode 1 miss, DiffusionVeteran, also fails out of the box. It is not a regression from a working baseline.

Host: eight AMD Instinct MI350X GPUs (gfx950, 270.6 GB HBM3E), ROCm Runtime 1.18. Backbone: Claude Sonnet 4, capped at 100 turns. A typical run uses about 719K tokens (median 608K) and 18–20 minutes. Repeating three repositories five times each, all 15 of 15 runs matched the original verification outcome.

Same model. Higher fidelity.

Binary “it ran” is too coarse. Agents often get a green run by replacing FlashAttention with SDPA, disabling a GPU module, or shrinking the workload. The Portability Fidelity Score charges for that. On a ten-repository diagnostic set chosen before any scores were known, every agent used Claude Sonnet 4, the same revision, hardware, budget, and public ROCm docs.

Repo2ROCm77.3

Claude Code61.4

Mini-SWE-Agent58.7

Cursor Agent40.3

PFS = Ran × (15 + 35 F_path + 20 A_work + 20 A_env + 10 D_proc)

Ran is 1 only when the run produced runnable evidence. Path fidelity grades each accelerator-critical component as preserved, degraded, or disabled. The ordering holds under equal component weights and when any single component is removed. This comparison is diagnostic, not a score over all 60 repositories.

Mode 2 is a case study, not the main claim.

Paper result reproduced

TurboQuant

After a verified ROCm environment, the quantization command’s compression, similarity, and match rate all landed inside the stated tolerance.

Not reproduced

dKV-Cache

Mode 1 succeeded: the intended workload ran on AMD GPUs. The reported hardware-sensitive speedup did not fall within tolerance, so Mode 2 refused the claim.

One of two preliminary cases matched. Missing datasets, gated checkpoints, and incomplete evaluation scripts still block result-level checks even when the code runs. Causal memory, ablated on 20 GPU-dependent repositories, moved the functional-run rate from 15/20 to 18/20 and gave a 1.16× speedup on repositories both settings finished.

05 — What every run ships

An audit bundle, not a chat log.

Dockerfile: The ROCm environment the planner selected and the executor built.
Unified diff: Source changes against the original repository, including kernel ports.
Command logs: What ran inside the sandbox and what ran outside it.
Verification: ROCM_ENV_VERIFIED, or a Mode 2 metric report with a match or a mismatch.
Plan and trajectory: The static plan and the step-by-step agent trace used for audit and distillation.
Code: System, benchmark artifacts, and reproduction instructions on GitHub.

github.com/Gaurav7888/Repo2ROCm

06 — Citation

Cite the paper.

@inproceedings{sarkar2026repo2rocm,
  title     = {Repo2{ROCm}: An Agentic System for Reproducing Research Code on {AMD} {ROCm} {GPU}s},
  author    = {Sarkar, Gaurav and Pandey, Umang and Brahma, Pratik Prabhanjan and Liu, Zicheng and Barsoum, Emad},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track},
  year      = {2026},
  address   = {Budapest, Hungary}
}