A/B test
An A/B test compares versions of an ad strategy by showing each one to a separate, randomly assigned part of your audience, so nobody sees more than one. In Meta Ads Manager the versions are whole campaigns, ad sets or ads that differ in one variable, such as the image, text, audience or placement, and the winner is usually the one with the lowest cost per result.
What does Meta's A/B test compare?
Meta's About A/B testing page:
A/B testing lets you compare two versions of an ad strategy by changing variables such as ad images, ad text, audience or placement. We show each version to a segment of your audience and ensure nobody sees both, then determine which version performs best.
Meta's About Experiments page gives a higher number for the same tool: "An A/B test enables you to compare up to 5 versions of an ad". Its comparison table says "Compare up to 5 versions" and "Available to any advertiser". The two pages disagree on two versus five; the Experiments page is the more specific one.
The split is the point. Meta "Divides your audience into random, non-overlapping groups who are shown different versions of your ads." Testing informally can't promise that: Meta says it "can lead to overlapping audiences".
How does Meta pick the winner?
By cost per result, for a test without a holdout. Meta's winner page for those tests says the winner "is determined by comparing the cost per result of each campaign, where the result is the event or events you selected during test creation". It adds that "you can add a holdout to an A/B test to compare cost per incremental purchase of each campaign", and the About A/B testing page says a test can measure each strategy "on a cost per result basis or cost per conversion lift basis." Back on the winner page:
Based on the activity observed during the test, Meta simulates possible outcomes tens of thousands of times to determine how often winning outcomes would have won.
The result comes with a confidence percentage, which Meta describes as the "chance of similar results if your test was repeated."
How long should a Meta A/B test run?
| Setting | Meta's guidance |
|---|---|
| Recommended minimum | "a minimum of 7-day tests" |
| Allowed schedule | "between 1 and 30 days" |
| Longer sales cycles | Run longer, "such as 10 days", if customers take more than 7 days to convert |
From Best practices for A/B Testing, which adds that "tests shorter than 7 days may produce inconclusive results."
What it means for your ads
- Test one variable. Meta: "You'll have more conclusive results for your test if your ad sets are identical except for the variable that you're testing."
- Write the hypothesis first. Meta's example: "My cost per result will be lower when I optimize for landing page views."
- Keep the test audience to itself. Don't use it "for any other campaign that you're running" at the same time; "Overlapping audiences may result in delivery problems and contaminate test results."
- Give it the same budget per version. Meta recommends "using the same budget for both versions in a test to ensure a fair comparison."
- For new creative inside a running campaign, Meta has a separate tool: creative testing.
Common mistakes
- Testing by switching ads on and off. Meta: "We do not recommend testing informally, such as by turning ad sets or campaigns on and off manually. This can lead to inefficient ad delivery and unreliable test results."
- Changing several things and naming a cause. You can test more than one variable, but "you won't know exactly which variable led to the winning ad set."
- Calling it after two days. Under seven, Meta's own guidance says the result may be inconclusive.
Sources
- About A/B testing — Meta Business Help Center: the definition, "nobody sees both", equal budgets, the cost per conversion lift basis, the warning against informal testing (rendered in a browser, accessed 2026-10-16)
- About Experiments — Meta Business Help Center: "up to 5 versions", the random non-overlapping split, availability (rendered in a browser, accessed 2026-10-16)
- How winning campaigns are determined in A/B tests without a holdout — Meta Business Help Center: cost per result, the holdout option and the simulated confidence (rendered in a browser, accessed 2026-10-16)
- Best practices for A/B Testing — Meta Business Help Center: one variable, the hypothesis example, audience overlap, the 7-day minimum and 1–30 day schedule (rendered in a browser, accessed 2026-10-16)
- Testing more than one variable in A/B tests — Meta Business Help Center: multi-variable tests can't isolate the cause (rendered in a browser, accessed 2026-10-16)
Meta's help pages are client-rendered and show no date; they were read in a real browser on 16 October 2026. Meta revises them without notice, and two of them already disagree on how many versions a test can hold, so check the source before acting on any of it.
Terrads makes ads that differ in concept and layout, so the versions you put in a test are actually different.