As advertisers slowly veer away from last-click methods of attribution, the popularity of lift testing keeps growing. For all its popularity, however, there are some common misunderstandings that can lead to inaccuracies when measuring costs associated with lift testing. 

What Is a Lift Test?

Some people might call lift testing the marketing equivalent of a randomized controlled trial. The idea is to show your ads to a small segment of your target audience, prevent the rest of the audience from seeing the ads, and then measure the long-term difference in conversions between the two groups.

This difference between the two groups reflects the success of your ads. When we divide the cost of the ads by the difference in conversions between the groups, we end up with a cost per incremental conversion metric. 

This metric shows the true cost of driving one incremental conversion in the lift tested campaign.  

Lift testing is more powerful than the standard cost per conversion (CPC) metric on Google Ads or Facebook Ads; these platforms simply tell you the cost for every conversion attributed to your ads — they don’t make any attempt to determine incrementality. That’s where lift testing comes in clutch. 

Despite their benefits, lift tests are far more complicated than many advertisers expect. That’s why we’re going to focus on one key misstep marketers make when lift testing — and how to correct it.

Not Just Incremental Conversions, but Incremental Cost

The generic formula for cost per action (CPA) is:

In this equation, “conversions” is usually understood as “conversions that have been attributed to a channel.”.

You can think of ift testing as a way to modify the denominator (the bottom half) of the equation. That is, instead of only looking at attributed conversions and getting an attributed CPA, lift testing looks at incremental conversions and gives you an incremental CPA.

This is a good start to breaking down the lift test, but the equation doesn’t tell the whole story. To understand why, we must first define what “cost” means in the context of a lift test.

Typically “cost” represents the media cost of running ads that are part of the lift test. For example, if our lift test involved running $5,000 worth of ads to a certain audience, then most people would assume that $5,000 is the cost.

The problem with that assumption is that it oversimplifies how channels interact with one another. If our lift test is run on an upper-funnel channel like display or Facebook, it’s important to recognize that these ads will also affect our lower-funnel channels.

There are two ways this can happen: 

• Paid search: If you run significant volumes of mid-to-upper-funnel activity, you’re likely to also see an increase in paid search volume. This could be through brand search, where more users start searching your brand specifically, or non-brand search, when users become more likely to click your ads on a results page. 
• Remarketing: If you’re driving more people to your brand, your remarketing campaigns are likely going to grow in step with this traffic. Provided that your remarketing campaigns aren’t strictly budget-capped, running higher-funnel channels can fuel growth in your remarketing pools, and in turn, can cause your remarketing campaigns to spend more.

The takeaway here is that both of these examples of lower-funnel channels will incur extra costs because of the upper-funnel activity. 

Typical lift testing doesn’t account for these extra costs — it only looks at the cost of ads on the channel being lift tested. This is because a lift test only measures whether users convert at all during the duration of the test; it doesn’t matter which channel the conversion happened on. This makes it an attractive way to track long-term conversions. 

If we refer back to our earlier equation, we can see that this leaves us in a tricky situation.

Typical lift test approaches underestimate costs by not considering the incremental cost accrued on other channels. This, in turn, causes an underestimate of our incremental CPA. Ultimately, most lift tests give an overly optimistic picture of efficiency. 

How Do We Make Sure to Count All Costs?

The easiest way to incorporate the extra costs into our calculations is to measure the incremental costs incurred on other channels as part of the lift test.

This can get a little technical, but it’s worth it to measure the true efficiency of your lift test. 

Let’s say we want to measure how much extra we have to spend on brand search due to running a lift-tested campaign on Facebook Ads. Just as we choose which conversion events we want to measure as part of our lift test, we can also choose to measure events like brand search visits. 

To do this, we set an event to fire on our website each time someone starts a session from a brand search ad (as determined by a UTM parameter, for example). Once this event is established, we then include it in our lift test measurement. We use the lift test to measure how many incremental brand search visits we get by running ads on Facebook.

From here, we multiply this number by the average brand search CPC during the lift test, which gives us an incremental cost figure — the extra money we spent on brand search because of the Facebook ads.

Now, when we go back to our original equation, we get a more accurate representation of our lift test’s incremental CPA:

This is a more accurate picture of our true incremental CPA from running our Facebook campaign because it also incorporates the incremental spend accrued on brand search.

The best part about this is you don’t need to stop here. You can easily add in events for any other channels influenced by your Facebook campaigns.

Start Slow; Ramp Up

For brands just starting with lift tests, the typical approaches can suffice in the beginning. However, as advertisers begin to scale and run larger, more complex tests, these methods become necessary to accurately measure incremental CPA. Without considering these extra costs during the lift test, any measurement of efficiency is going to be inaccurate. 

Want to talk more about incremental CPA, lift tests or Performance Branding? Get in touch with us any time.