How to Do Incrementality Testing: A Plain Guide

To do incrementality testing, you split a comparable audience or set of regions into a group that sees your marketing and a group that does not, run both over the same period, and compare the results. The gap between them is the extra sales your marketing caused, rather than sales that would have happened anyway. This guide walks through the steps in plain terms.

What an incrementality test actually measures

Incrementality is the extra sales or conversions your marketing causes that would not have happened without it. Platform reports usually count every conversion that follows an ad, including customers who were already on their way to buy. A test separates those customers from the ones your spend genuinely persuaded.

The core idea is a fair comparison. One group is exposed to the campaign, and a matching group is deliberately not. Because the groups are alike in every other way, any difference in outcomes can be credited to the marketing and not to chance, seasonality or a promotion running at the same time.

The steps to run a test

Start with a single clear question, such as whether a particular channel is adding sales or only claiming credit for them. Choose one outcome to judge it by, such as orders or revenue, and write it down before the test begins. Then decide how you will divide the audience: by randomly splitting people, or by splitting regions that behave similarly.

Hold back the campaign from the control group, keep everything else steady, and let the test run long enough to cover a normal buying cycle for your product. Avoid changing budgets, creative or offers mid-test, because any change muddies the comparison. Once the period ends, compare the outcome in the exposed group with the outcome in the held-back group.

Common test designs

A holdout test withholds ads from a random slice of your audience and compares it with everyone else. A geographic test switches a channel off, or changes its spend, in selected regions and watches how sales move compared with regions left alone. Some ad platforms also offer built-in lift studies that handle the split for you.

Each design suits different situations. Holdouts work well where you can control who sees an ad. Geographic tests suit channels where you cannot target individuals, such as broadcast or out-of-home. Choose the design that matches how the channel is bought and how your customers are spread.

Reading results and avoiding mistakes

Look at the difference between the groups, then ask whether it is large enough to trust or small enough to be noise. A result with a wide margin of uncertainty is a prompt to run the test longer or with a bigger audience, not a reason to act. Compare the lift against what you spent to judge whether the channel paid for itself.

Typical mistakes include groups that are not truly comparable, tests that end too early, and overlapping campaigns that leak ads into the control group. Treat one test as one data point about one channel at one moment, and repeat it when conditions change.

Where testing fits with marketing mix modelling

Tests give a direct answer about a specific channel, but you cannot test everything at once. Marketing mix modelling uses historical sales and spend data to estimate what each channel contributes, and test results can be used to check whether those estimates hold up.

Seeda’s platform covers marketing mix modeling, incrementality testing, channel-level ROI and budget optimisation, and connects to your ad platforms, analytics and commerce data. If you want to see how that works with your own numbers, you can book a no obligation demo.

Frequently asked questions

How long should an incrementality test run?

Long enough to cover a normal buying cycle for your product, so that the result reflects typical behaviour and not a quirk of a few days. Products with longer decisions generally need longer tests.

Do I need to switch off a channel to test it?

Not always. You can hold back a random slice of your audience, or reduce spend in selected regions, which limits lost sales while still giving a fair comparison.

Is incrementality testing the same as marketing mix modelling?

No. A test runs a controlled experiment on one channel, while marketing mix modelling estimates each channel’s contribution from historical sales and spend. They work well together, and each can check the other.