⏏️ Interpreting the Outputs

This is an interesting question! While it may be best practice when using tools like Robyn to use metrics like clicks and impressions, we don’t believe it is best practice generally. There are a few things to consider:

• Consistency: We want to get an apples-to-apples comparison of channel performance that isn’t biased in favor of certain channels. That means we want to use measures that are *consistent.* We think of this in a few ways:
  • Consistency across channels: Not all media channels have clicks, and most don’t actually measure impressions the same way. What do you use for linear TV, radio, and podcast? If you’re using different “impression” metrics for these different channels, is that biasing your results?
  • Consistency over time: What happens if a channel changes how they measure and record impressions? How was measuring clicks impacted by iOS 14.5? Additionally, are all impressions within a given platform the same?
• Interpretation and actionability: Marketers don’t directly control the number of clicks or impressions they are getting, but rather control investment. If you measure the effectiveness of “clicks” and “impressions” then you add another step to actionability (converting to dollars) and it makes forecasting more challenging (because you have to somehow forecast the number of clicks you’ll get and what the cost-per-impression will be on some platform)

Unfortunately, the practice of including clicks and impressions is likely to bias the model in favor of digital channels that most easily measure clicks and impressions (like Facebook) so we would recommend against it.