$ cat wiki/papers/2026/2610.12569-swe-duel.md
SWE-Duel — coding agents generate and solve each other's tests
TL;DR
SWE-Duel generates coding tasks and their tests during evaluation. Two agents alternate between constructing a repairable defect and reviewing a pull request without being told that a defect exists. (source)
Authors & Org
Wei Minn, Zhensu Sun, Lwin Khin Shar and David Lo. Submitted 2026-10-08. The abstract page does not state affiliations. (source)
Method
Red uses three context-isolated sessions to add a feature, supply feature tests and embed a subtle bug with sealed triggering tests. A task is admitted only if feature and existing repository tests pass, bug tests fail, and a fresh author session repairs it blind. Blue must make its revision pass all three test suites; the agents then swap roles. (source)
Results
The tournament covers 11 model-harness participants, 15 repositories, five languages and 1,500 turns. GPT-5.5 with Codex CLI ranks first as solver, repairing 98.3% of reviewed defects. Claude Opus 4.8 with mini-swe-agent ranks second overall and is the strongest task author, with a 27.3% Red win rate. Within one model family, newer authors defeat a fixed-reference solver on 42.9% of tasks versus 4.0% for older authors. The authors describe the last result as preliminary evidence of scaling task difficulty. (source)
Significance
The study separates creating a difficult task from solving one and creates evaluation content at runtime. For Eval Harness Configuration, the participant is explicitly a model-harness pair. Its ranking should not be read as a context-free ranking of model weights. (source)
Open Questions
The abstract does not provide all participant configurations, full costs or independent replications. Runtime task generation addresses fixed-task leakage but does not by itself establish that every generated test suite captures all valid implementations. (source)
Cite
arXiv:2610.12569, submitted 2026-10-08. Captured abstract.