Debating and Voting Agents

A single model's answer is one sample from a distribution that includes errors and over-confidence. Running multiple agents and aggregating — by voting or by structured debate — exploits independence and disagreement to surface mistakes and converge on more reliable conclusions.

• Voting smooths out independent random errors.
• Debate exposes flawed reasoning to scrutiny.

The simplest aggregation: sample several independent reasoning paths for the same question and take the majority final answer. Diverse chains often reach the same correct answer while errors scatter, so the mode is more reliable than any single path.

Works best when the answer space is discrete and the paths are genuinely independent.

Voting only helps if errors are independent. If all agents share the same model, prompt, and bias, they fail together and the vote merely amplifies a shared mistake. Introduce diversity: different temperatures, different prompt framings, different decompositions, or different models.

Correlated voters give false confidence — the danger of consensus without independence.

In debate, agents produce initial answers, then read each other's reasoning and revise across rounds. Exposure to opposing arguments lets agents catch their own errors and abandon weak positions, often converging on a better answer than any started with.

• Round 1: independent answers.
• Round 2+: critique and revise given peers' arguments.

Debate can collapse into agreement for the wrong reason: agents capitulate to a confident peer rather than to a better argument. Counter this by instructing agents to change their mind only on the merits, to defend correct positions under pressure, and to state the specific argument that moved them.

Productive disagreement is the goal; premature consensus is failure.

Rather than hoping debaters converge, add a neutral judge that reads the final positions and arguments and renders a decision with justification. The judge is not a debater and has no stake, which reduces the pull of the loudest voice.

Separating advocacy from adjudication is a powerful, auditable pattern.

Not all votes are equal. Agents can report calibrated confidence, and you can weight the aggregate accordingly — or discard low-confidence votes. Be cautious: model confidence is often poorly calibrated, so validate that weighting actually improves accuracy before trusting it.

• Weight by confidence only if confidence is calibrated.
• Otherwise, plain majority is safer.

Aggregation multiplies cost linearly with the number of agents and rounds, but accuracy gains diminish. The first few extra samples help a lot; the tenth rarely does. Tune the count empirically and stop where the marginal accuracy no longer justifies the marginal spend.

Adaptive stopping — halt early when consensus is strong — saves budget on easy items.

Voting suits tasks with a clear discrete answer and independent samples. Debate suits tasks where reasoning is the failure point — multi-step proofs, plans with subtle flaws — because the value is in critiquing the chain, not tallying conclusions.

Match the method to where the errors live: in the answer (vote) or in the reasoning (debate).

Define what happens when agents do not converge: a tie in votes, a deadlocked debate. Have a deterministic tie-breaker (judge ruling, lowest-risk option, or escalation to a human) rather than letting the system loop or pick arbitrarily.

• Cap debate rounds; on deadlock, judge decides or escalate.
• Surface 'no consensus' as a valid, actionable outcome.

For reliable consensus: ensure voter independence through real diversity, choose voting for discrete answers and debate for reasoning-heavy tasks, guard against sycophancy with merit-only revision, add a neutral judge for adjudication, weight by confidence only when calibrated, stop adaptively to control cost, and define deterministic tie-breaking with escalation. Disagreement is the resource; manage it deliberately.

You run seven agents that all use the identical model, prompt, and temperature, then majority-vote their answers, expecting higher reliability.

Aggregating multiple agents improves reliability by exploiting independence and disagreement. Use self-consistency voting for discrete answers and multi-agent debate for reasoning-heavy tasks, but only when voters are genuinely diverse — correlated agents amplify shared bias. Guard against sycophancy with merit-only revision, add a neutral judge to adjudicate, weight by confidence only when calibrated, stop adaptively to control cost, and define deterministic tie-breaking with escalation.