AI Safety via Debate

Debate is a proposed training method in which competing AI agents argue before a judge, who decides which provided the most useful, truthful information. It aims to improve Scalable Oversight by exposing errors the judge could not find unaided.^[1]

How competition could help

Agents choose answers, exchange arguments under a turn limit and receive opposing rewards based on the judge’s decision. The intended incentive is to expose a rival’s mistake rather than persuade through a misleading explanation. This requires errors to be findable, evidence to be accessible and the judge to evaluate the decisive claim correctly.^[1][2]

The proposal’s computational analogy assumes optimal play, arbitrarily powerful agents and an appropriately chosen polynomial-time judge; polynomially many turns can then decide problems in PSPACE. It does not show that learned agents reach that play or human judges implement the required decision rule. Persuasion and unstable training remain material concerns.^[1]

Evidence and error visibility

The 2018 paper’s restricted MNIST experiment uses search agents to reveal genuine pixels to a fixed classifier. It gives evidence about adversarial evidence selection under those constraints, rather than unrestricted language debate with human judges.^[1]

The 2020 human protocol writeup reports unreliable early judging on tricky physics questions, even with motivated judges. Ambiguous claims let a dishonest debater change interpretations; proposed cross-examination of noncommunicating copies aims to expose inconsistencies. The target of over 90% correct judging within ten minutes was an objective, not an achieved result.^[2]

That writeup also distinguishes finding an error from checking one. A long argument can hide a flaw neither debater can isolate, leaving the judge without a decisive objection. Dividing discussion into smaller disputes does not itself supply the missing understanding.^[2]

Debate’s promise rests on turning difficult verification into an assessable disagreement. Whether a particular protocol achieves that depends on the task, judge, debaters and training incentives. Who Supplies Alignment Feedback? connects this change in evidence to the broader reliability of supervision; the paper entries retain the formal setup and study details.