A/B testing: how to test ads and landing pages properly
An A/B test shows which variant works better. Here is how to set the hypothesis, duration and measurement, and how to avoid typical false conclusions.

An A/B test compares two variants that differ in exactly one thing. Each variant is seen by a random half of the visitors. The variant that does better on a metric set in advance wins, for example the number of enquiries per hundred visitors. That applies to ads just as much as to landing pages. Three things are decisive: a clear hypothesis, enough data and clean measurement of conversions.
What can you check with an A/B test?
You can test anything that visitors see or do, as long as you can separate the variants cleanly. For ads that means the headline, text, image or audience. For a landing page, meaning the page where visitors arrive after clicking the ad, it means the headline, value proposition, layout, form length or button label. What you cannot test is anything you cannot distribute randomly, such as a comparison of January with February. That would be a comparison over time and not an A/B test.
A practical example: a heating engineer runs ads for servicing contracts. Variant A names the price of the service in the headline, variant B the benefit (“Get your heating checked before the cold arrives”). Everything else stays the same: the same landing page, the same form, the same budget. After the test it is clear which headline brings more enquiries per click.
How does an A/B test work step by step?
A clean test always follows the same order. The order protects you from reading out of the result only what you wanted to see.
- Formulate a hypothesis: “If we change the headline to the benefit, more visitors will submit the form because they understand more quickly what they get.”
- Set the success metric: a single main conversion, for example a submitted contact form
- Change exactly one thing and distribute the variants randomly across visitors
- Note a stopping rule before you start: duration in full weeks or number of visitors per variant
- Evaluate only after the end, then decide and document the result
→ Cookie banner with Consent Mode v2: compliant, with no monthly licence
How much data and time does an A/B test need?
The smaller the difference between the variants, the more data you need to separate it from chance. There is no fixed minimum that applies to every page. A worked example shows the problem: suppose variant A brings 10 enquiries from 200 visitors and variant B 14. That looks like 40 percent more. With so few enquiries, though, the difference can be pure chance, because three or four enquiries more or fewer are enough to turn the picture.
Two rules help in everyday work. First: let the test run for full weeks, because behaviour differs from Monday to Sunday. Second: do not look at the result after every day in order to stop. Anyone who keeps looking until one variant is ahead almost always finds an apparent winner. For calculating whether a difference is reliable, there are significance calculators. The platforms do this calculation themselves in their testing features.
Which tools are there for A/B tests?
For ads, the platforms offer testing features. For landing pages you need a tool on the website or two separate pages. Google Optimize, which many used for website tests in the past, no longer exists.
| Where | What you test | What to watch out for |
|---|---|---|
| Google Ads: custom experiment | Settings, bids or ads of a campaign against a copy | The experiment gets a share of the campaign’s traffic and budget. According to Google Help, a 50/50 split gives the most comparable results. You choose up to two conversion goals as success metrics. |
| Meta: A/B test in Meta Ads Manager | One variable such as ad, audience or placement | Meta divides the audience into groups that do not overlap. Change only one variable, otherwise you cannot attribute the effect. |
| Landing page | Headline, layout, form | Two separate pages with their own link each, or a testing tool. Both variants need the same conversion tracking. |
For experiments in Google Ads, limits apply according to Google Help: each campaign can have up to five experiments, but only one runs at a time. The original campaign has to stay active during the experiment. Changes to it can distort the comparison. For Search campaigns, Google recommends the split by cookies, in which each user always sees the same variant.
→ Setting up Google Consent Mode v2 correctly: no warning in Tag Assistant
How do you measure which variant won?
What is always measured is a conversion, not the click. An ad with more clicks is not better if the clicks lead to fewer enquiries. So decide before you start which event counts as success, for example the submitted form, counted only after the success message. In Google Analytics 4 (GA4), the recommended event for this is generate_lead.
So that you can separate the variants in GA4, you label them in the links. You can do that with UTM parameters, for example utm_content=headline-benefit and utm_content=headline-price. Both variants must use the same tracking. If a page lacks the conversion or counts it twice, the measurement error decides in the end and not the headline. If many visitors come without consent, their conversions are missing in both variants, the sample gets smaller and the test takes longer.
Which false conclusions are typical with A/B tests?
- Stopping too early because one variant is ahead after a few days
- Changing several things at once and then not knowing what worked
- Comparing clicks instead of enquiries
- Changing budget, audience or bids during the test
- Testing too many variants at once, so that each gets too little data
- Transferring a result from one campaign to all the others without checking
A test with no clear winner is also a result. It shows that the thing you changed makes no big difference for your visitors. The next step is then to test something bigger.
Why the measurement has to be right first
An A/B test only delivers usable results if conversions are counted completely and exactly once. Trackum sets up conversion tracking for Google Ads and Meta at a fixed price and tests it in the preview before it goes live, with and without consent. Whether your page loads tags and cookies the way you think is shown in two minutes, without sign-up, by the free check at trackum.app/check.
Frequently asked questions
How long should an A/B test run?
There is no fixed duration that suits every page. Before you start, decide how many visitors or conversions you will wait for. Let the test run for at least full weeks so that days of the week are weighted equally. Do not stop just because one variant is ahead after two days.
What is the difference between an A/B test and a multivariate test?
In an A/B test, two variants differ in exactly one thing. A multivariate test changes several elements at once and tests all the combinations. That takes far more visitors, so it is rarely suitable for small sites.
Can I run an A/B test with few visitors?
Yes, but only big differences become visible. Test bold changes such as a completely different headline or a shorter form, not the colour of a button. With very few enquiries a month, talking to customers about what convinced them of the offer often brings more than a test.
Do I need consent for an A/B test?
That depends on the tool. If it stores information in the browser, such as a variant assignment, consent under § 25 TDDDG may be needed. Check the tool and your privacy policy. This article is not legal advice.
Sources
- Google Ads Help: Set up a custom experiment support.google.com
- Meta Business Help Centre: About A/B tests facebook.com
- Google Analytics Help: Campaign URL builders support.google.com
- Google for Developers: GA4 event reference (generate_lead) developers.google.com
Read more
FeaturesCookie banner with Consent Mode v2: compliant, with no monthly licence
Reject on the first layer, settings per purpose, Consent Mode v2 with every consent type and a record of each choice. No monthly licence.
FeaturesSetting up Google Consent Mode v2 correctly: no warning in Tag Assistant
The most common warning in Tag Assistant says a tag read consent before a default was set. How to avoid it, and what else matters.
FeaturesMarketing dashboard with GA4 and Google Ads: every number on one page
Visitors, leads, orders, ad spend and ROAS per campaign on one page, compared with the previous period. One link, nothing to set up.
FeaturesRevenue from your CRM, shop and ERP in one dashboard: HubSpot, Shopify, Odoo and more
Google Analytics only sees what happens on the website. Orders by phone or email are missing. With a data connection, the real revenue from your system appears in the dashboard.
FeaturesWebsite leads with their source in your CRM: HubSpot, Pipedrive, Brevo, Salesforce or webhook
Where did the lead come from? The CRM handover writes source, campaign and click ID into the CRM with every enquiry, without a form plugin.
FeaturesOffline conversions: report sales from your CRM to Google Ads and Meta
Google Ads and Meta learn from what you report to them. If they only get form fills, they optimise for form fills. With offline conversions, they optimise for sales.