Lift Tests and Incrementality Experiments

Well-run experiments are a wonderful complement to Recast's media mix model. We recommend that brands using Recast run incrementality experiments as frequently as they can and then use those results to help to calibrate the Recast MMM model.

Lift tests can be used to calibrate the Recast MMM model results in a fairly straightforward way. Technically, the lift test information is included as a special type of prior in the model (distinct from the broadly uninformative channel-level priors you might set).

In order to use a lift test to calibrate Recast, you need a few pieces of information:

• The channel name. This needs to map to a single channel configured in Recast
• The start date and end date of the test. We only want the lift test result to apply to when the experiment was run.
• The point estimate for the test result
• The standard error of the test result

So in practice, this might look something like this:

• Channel: Meta Prospecting
• Dates: April 15 to May 15
• Test result: incremental CPA of $75, with standard error of $10

Other things to review:

• Calculating Uncertainty and Standard Errors for Lift Tests
• Details on Priors for Lift Test Calibrations

Additional reading:

• This blog post by Gufeng Zhou (of Meta and Robyn fame) on the intricacies of using lift tests to calibrate MMM results is very helpful.