Multi-Armed Bandit vs A/B Testing: When to Use Each

The short answer

An A/B test and a multi-armed bandit solve different problems.

• An A/B test splits traffic in fixed proportions until the end, then tells you which variant is better and how confident you can be. Its goal is learning.
• A bandit keeps shifting traffic toward the variant that is performing best right now. Its goal is earning during the test itself.

If the result guides a long-term decision, run an A/B test. If the window is short and every lost conversion hurts, use a bandit.

The explore-exploit trade-off

Picture a gambler facing several slot machines ("one-armed bandits") with unknown payouts. Every pull is a choice:

• Explore — try a machine you know little about to learn its payout.
• Exploit — pull the machine that looks best so far to collect the reward.

Pure exploration wastes money on bad arms. Pure exploitation risks locking in an early leader that was simply lucky. A classic A/B test is pure exploration followed by pure exploitation: equal split until the end, then 100% to the winner. A bandit blends the two continuously.

How bandit algorithms reallocate traffic

Three common approaches, from simplest to most adaptive:

A minimal Thompson sampling step for conversion rates looks like this:

import random

def choose(arms):
    # arms: {"A": {"conv": 30, "views": 1000}, "B": {...}}
    draws = {
        name: random.betavariate(1 + a["conv"], 1 + a["views"] - a["conv"])
        for name, a in arms.items()
    }
    return max(draws, key=draws.get)

As one variant accumulates evidence of being better, it wins more draws and gets more traffic automatically.

Where bandits beat A/B tests

• Short promotions. A weekend sale has no "after the test" — the opportunity ends with the campaign. Waiting for significance means losing most of the window.
• Headlines and thumbnails. Content sites test many short-lived titles. A bandit quickly concentrates traffic on the strongest one.
• Ad creatives. Ad platforms already rotate creatives adaptively; the same logic works when you control rotation yourself.
• Many variants. With five or ten options, an equal split spends a lot of traffic on obvious losers. A bandit drops them fast.
• Ongoing optimization. Recommendation slots or subject lines where the best option drifts.

Why bandits are worse for learning causal effects

A bandit optimizes reward, not knowledge. That has costs:

• Unequal samples. Losing variants receive little traffic, so their estimates stay noisy. You know the winner is likely better, but not by how much.
• Biased estimates. Because allocation depends on past results, naive conversion rates per variant are biased. Standard t-tests and confidence intervals no longer apply directly.
• Time effects get mixed in. If the leader gets most traffic on weekends and the others on weekdays, day-of-week effects can masquerade as variant effects.
• Delayed conversions break it. If purchases happen days after the click, the algorithm reallocates traffic on incomplete data.
• Secondary metrics are hard to read. Checking retention, refunds or average order value across arms is unreliable with lopsided samples.

For pricing, checkout redesigns or anything you will keep for months, these weaknesses outweigh the conversions saved during the test.

How to choose

Ask four questions:

1. Will the decision last long after the test? Yes — A/B test. No — consider a bandit.
2. Do you need the size of the effect for a business case or forecast? Yes — A/B test.
3. Is feedback fast (clicks, sign-ups within the session)? Bandits need quick rewards.
4. How many variants? Many variants and limited traffic favor a bandit.

A common hybrid: use a bandit to screen many creatives, then confirm the top two with a proper A/B test before a long-term rollout.

Common mistakes

• Using a bandit and then reporting "variant B lifted conversion by X%" as if it were an A/B result.
• Optimizing for clicks when the real goal is revenue — the bandit will happily find clickbait.
• Running a bandit on a metric with long delays.
• Setting epsilon or priors so aggressively that the algorithm commits after a handful of conversions.
• Ignoring guardrail metrics such as bounce rate or refunds.