Flight log / Transmission 009

Incrementality testing for marketers who hate math

Every ad platform is sure it deserves the credit. Incrementality testing is how you check, and since November 2025 Google says an experiment that once cost upwards of $100,000 can be run for $5,000.

Attribution answers "who gets credit?" Incrementality answers a ruder question: "would this have happened anyway?" Those are different questions, and most paid media budgets are set using only the first one.

No equations ahead. Just the three test formats, what Google and Meta will run for you today, when a test is worth the withheld reach, how to read the report and the traps that turn a good test into an expensive opinion.

What is incrementality testing?

Incrementality testing is a controlled experiment for ad spend. You split people, or places, into two groups. One group can see your ads (the test group). The other can't (the control group, often called the holdout). If the groups are similar and nothing else differs, the gap in conversions between them is what your ads caused.

That gap is the lift. Everything else your dashboard reports, including people who were already going to buy, brand searchers who would have found you organically and retargeted visitors who had a contract in hand, is credit, not cause.

The honest way to know what would have happened without your ads is to build a timeline where they didn't run. No time machine or flux capacitor required: a holdout group is that alternate timeline, running in parallel.

How is incrementality different from attribution?

Attribution, whether last click or data-driven, starts from conversions that already happened and divides credit among the touchpoints it can see. It never asks whether the conversion needed any of them. A retargeting campaign that shows ads to people already on their way to checkout can look brilliant in attribution and add close to nothing in a holdout.

That's why the two work as a pair, not as rivals. Attribution is cheap, daily and directional. Incrementality is slower, costlier and closer to the truth. We go deeper on how they fit together in cross-channel attribution: what to trust and what to ignore.

What are the main types of incrementality tests?

How does a holdout test work?

A holdout test randomly assigns eligible users to see your ads or not. On platforms that support it, the control group is usually shown nothing from you, or a placeholder ad from someone else, so you can compare behavior without your message in the mix.

1. Pick one question. "Does retargeting drive demo requests we wouldn't otherwise get?" is testable. "Is Meta working?" is not.
2. Pick one primary outcome. The conversion closest to revenue that still has enough volume. For B2B, that's often a qualified lead rather than a closed deal.
3. Decide the holdout size before launch. Bigger holdouts give a cleaner read and cost more in withheld reach.
4. Freeze everything else. No new creative, budget swings or landing page changes mid-test.
5. Run past your conversion delay. If leads typically convert 10 days after the first click, a 7-day test misses most of the effect.

How do geo tests work?

A geo test splits by location instead of by person. You pick comparable regions, keep ads running in some and switch them off, or scale them up, in others, then compare outcomes. Because you measure regions rather than tracked users, geo tests can read outcomes ad platforms can't see directly, such as CRM pipeline or offline sales.

The trade-off is noise. Regions differ in size, seasonality and sales coverage. For B2B, one enterprise deal landing in a test region can swamp the signal. Use more regions rather than fewer, match them on historical performance and avoid splits that line up with sales territories.

What lift tools do Google and Meta offer?

Google Ads Conversion Lift. Google describes Conversion Lift as an incrementality tool that splits audiences into an exposed group and a control group. It comes in two forms:

• Based on users: a user-level split for Video, Discovery and Demand Gen campaigns (Google's docs still list Discovery, the former name of Demand Gen), with results by conversion action, age, gender and country. Metrics include incremental conversions, relative lift, incremental CPA and incremental ROAS.
• Based on geography: a regional split for App, Demand Gen, Discovery, Display, Video, Search, Shopping and Performance Max campaigns. Campaigns must target a single country, and offline conversions at ZIP or city level are supported. Google notes this split is noisier and doesn't slice results by demographics.

Access is the catch. Google's help center says Conversion Lift isn't available for all accounts and directs advertisers to their account representative. On cost, Google announced in November 2025 that "what once might have cost upwards of $100,000 for a single experiment can now be done for $5,000," and says updated models deliver conclusive results up to 50% more often.

Meta conversion lift. Meta's developer documentation lists conversion lift studies and A/B tests as its two ways to measure the incremental impact of ads, and includes an eligibility check before you create a test. The practical details come from practitioner guides, not Meta: you need enough expected conversions in the test window and working tracking through the pixel, Conversions API or offline conversions, and a 10% to 20% holdout is the commonly suggested range. Neither source lists a fixed budget minimum.

If Meta is your measurement problem, the first fix is usually signal quality, not experiment design. Our guide to Meta Conversions API setup for B2B lead gen covers getting CRM stages back to the platform.

When do you actually need an incrementality test?

Not every channel needs one. A test costs time, withheld reach and a few awkward weeks of explaining to sales why the ads are off in Ohio. Run one when the answer could move real budget.

The pattern that shows up in most B2B accounts: branded search, retargeting and lookalike audiences built from site visitors get the most credit in platform reporting, and they're the first places worth testing.

How do you read incrementality test results?

Most lift reports give you a handful of numbers. Here's what each one means in plain English:

• Incremental conversions: conversions the test group got that the control group didn't, scaled to the same size. This is the "caused" number.
• Relative lift: incremental conversions as a percentage of what the control group did on its own. A 20% lift means the ads added about one conversion for every five that would have happened anyway.
• Incremental CPA (iCPA): spend divided by incremental conversions. Expect it to be higher than the CPA in your dashboard, sometimes much higher.
• Incremental ROAS (iROAS): incremental revenue divided by spend. The number your CFO actually cares about.
• Confidence or probability: how sure the model is that lift is real. Higher confidence needs more volume.

Say a 20-person SaaS team runs a hypothetical holdout on retargeting. The platform reports 50 demo requests at $200 each. The lift study finds 15 of those were incremental. The real cost per incremental demo is about $667. That doesn't mean retargeting is bad. It means you now know what you're paying for, and you can compare it fairly with channels that create new demand.

What are the most common incrementality testing traps?

• Calling it early. Day 4 looks amazing. Day 21 looks normal. Decide the end date before launch and keep it. The same discipline applies to how long to run an A/B test.
• Reading "no significant lift" as "no lift." Inconclusive means the test couldn't separate the effect from noise. Check volume and duration before you cut a channel.
• Changing other things mid-test. New creative, a pricing change or a landing page redesign during the window contaminates both groups unevenly.
• Picking the wrong outcome. A test on form fills may show lift that disappears at the qualified-lead stage. Measure the deepest outcome with enough volume.
• Ignoring conversion delay. B2B buyers take weeks. A test that ends before they convert under-reads the effect.
• Treating one result as permanent. Lift changes with seasonality, creative and competition. Retest the big channels at least once a year.
• Leaking the control group. If the same people can see your ads on another campaign, platform or device you didn't hold out, the control isn't clean.

Where this breaks

Incrementality testing is the best tool most marketers have for cause and effect, but it has limits worth saying out loud:

• Low-volume B2B. If a channel produces a handful of opportunities a month, no test design will produce a confident answer in a reasonable time. Test a higher-volume proxy and be honest that it's a proxy.
• Platform-run studies. A lift study run by the platform you're spending on is still useful, but you can't inspect every modeling choice. Treat the methodology note as required reading.
• Long-term brand effects. A 4-week test measures short-term response. It won't capture what happens when a category hears your name for a year.
• Access and minimums. Google's lowered $5,000 threshold helps, but access still runs through account teams, and requirements vary by campaign type.

For the bigger picture of where incrementality sits in a paid program, start with the B2B PPC playbook.

FAQ