I don’t believe these results! What do I do?

You can’t take action on results you don’t trust. If something seems off in the Recast model, it makes it difficult to trust its findings and make decisions based on the results. Below is a list of our recommended next steps when you encounter something in your results that contradicts what you know about your business:

• Talk through the interpretation - it’s very common for doubt to be caused simply because the way you’re interpreting the model doesn’t line up with the way the model interprets your data. For example, if you use a different definition of “organic” than the intercept, the intercept may look way too high, but getting a better understanding on how to interpret the intercept may resolve the discrepancy.
• Consult how we configured your model - When we first set up your model, we had to establish “reasonable ranges” (priors) for the outputs (ROIs, time shift, saturation, etc). Translating marketing lingo into complicated mathematical modeling is not simple, and for a variety of reasons what we had originally put in as your “reasonable ranges” might not be so reasonable to you, and we need to revisit those to make sure they match your ideas of reasonable.
• Remember the uncertainty - Many times doubt is caused by a small channel having a very large ROI. However, when channels are small there is still a lot of uncertainty in the result. Consider the lower end of the confidence interval. If that is what we showed as the ROI, would it still be unreasonably high?

If we’re aligned on interpretation, confident in our configuration, checked the uncertainty, and still don’t feel confident in the results, there’s two more paths to be able to gain confidence in the ability to take action on model results:

• Lean into experimentation - The model gets better the more you use it! If the model is very confident in a channel, experiment with growing the channel. Either the model will learn that it was wrong and update its results, or you will, but even more often it will be somewhere in between.
• Identify limitations of the model - the model isn’t perfect (yet!). Each business has its own unique marketing challenges, not all of which are going to be fully captured by Recast. Talk with the Recast team to identify what the model might be missing (so we can work on it) and help you know where the deficiencies are. Certain channels may be structured in a way that make it difficult to get an accurate read on performance. We can give advice about how to handle and interpret these channels, as well as what other parts of the results may be effected.

Visit the "How to" section for more information on How to Improve and Validate your model.