Home

🧑🏽‍🏫 Insights

Learn all about how to interpret the charts and graphs in the Recast Insights Dashboard.

Suggest Edits

The first place many people start is on the Insights tab. The Insights tab tells us about what’s been happening in the past and shows the results of a statistical model trained on your company’s historical data.

💰Spend Summary

The spend summary tab shows some high-level statistics on your historical spend and business KPI over the last 12 months. This view is useful for:

  1. Understanding high-level patterns in the data
  2. Validating that there’s no data issues that might impact the model results

📷 Last Week Snapshot

This is probably the most important view on the insights tab. The last week snapshot view shows us a breakdown of channel performance over the last complete 7 days that the model has seen. You can see the last-complete-data-date in the top left-hand corner of the screen where it says “data through”.

Waterfall Plot

The waterfall plot shows a breakdown of your KPI in the last 7 days and how much each marketing channel contributed to that KPI (according to the statistical model).

Notes:

  • The “intercept” is all of the KPI that’s not attributed to marketing activity. Some marketers call this “base sales” or “organic”.
  • Spikes are things like promotional events, store closures, or new product launches. You can see more detail on the “Spike Summary” tab
  • “Unexplained Variation” is due to the random variation where sometimes the model estimates miss high or low. This should average out to zero over long time frames, but within a given week you may randomly have more (or less) unexplained variation just due to randomness.

Marketing Effectiveness

Here we see two charts side-by-side. The left-hand side just shows how much you spent on every marketing channel in the last week, and the right hand side shows the relative effectiveness of each channel. Longer bars are always better.

The blue bars show the ROI or CPA of all of the dollars invested in the channel over the time period. The red bars show the marginal ROI or CPA of the last dollar invested into the channel.

The marginal ROI will always be less than the average ROI due to diminishing returns. You can see each channel’s diminishing returns curve on the Channel-Level Deep Dive tab.

Note:

  • These are estimates of the total return earned (but not necessarily realized) in the last 7 days. The model expects that additional conversions will come in the future.
  • The lines on top of the bars represent uncertainty in the model and are showing the interquartile range of effectiveness estimates found by the model.

Share of Spend vs Share of Effect

This chart is a nice summary of which channels you’re most invested in versus which channels drive the most impact for your business. Channels where the % effect is higher than the % spend are relatively over-performing and channels where the % spend is higher than the % effect are relatively under-performing.

↕️ Lower Funnel Spend

The Lower Funnel Spend tab shows how your spend in certain marketing channels drives spend into channels in which you have less control over what you spend. Recast has a notion of “lower funnel channels” and “upper funnel channels”. This terminology likely doesn’t match your internal categorization of channels (and that’s okay!) but in Recast “lower funnel channels” are generally channels that are driven primarily by other channels. These are generally channels like “branded search” and “affiliates”.

The lower funnel spend section of the dashboard has a tab for each lower funnel channel and it shows a waterfall chart of what other marketing channels drove that spend. That is, it answers questions like “how much of our google branded search spend was driven by TV spend?”

The lower funnel channels also include an “intercept” which basically just includes all of the spend in that channel not driven by other marketing channels.

🌊 Channel-Level Deep-Dive

The channel-level deep dive tab has three sub-tabs: Channel Performance, Shift Curves, and Saturation Curves

Channel Performance

The channel performance section has one tab for each paid marketing channel your brand has spent money on in the last year. In this tab, we can see a few different views of how channel performance changes over time.

Each of these views has two time series lines on it. One line (in green) is the amount of spend which is simply shown for context. The other line is blue with a shaded region and is the time-series of the results found by Recast.

The views are:

  • ROI: What is the total earned ROI for this channel for every day in the last 12 months. Note that the ROI is earned, not necessarily realized on that day.
  • Marginal ROI: What is the ROI of the last dollar invested into the channel on that day.
  • Impact: This is the effect earned on that day. It is just the ROI multiplied by the spend.
  • Impacted Shifted: This is the effect realized on that day. It is the impact after the time-delay schedule (”shift curves” in Recast parlance) has been applied.

Shift Curves

Shift curves show how long it takes for marketing to have its effect. Sometimes people use the term “adstock” which is technically a different functional form then what Recast uses but the idea is the same: the model is learning how long it takes for marketing spend to have its effect.

How to interpret the chart:

  • The horizontal axis is the number of days since the spend occurred
  • The vertical axis the percent of effect realized after N number of days since the spend occurred.

So, on the left side of the chart we can read off how much of the effect is realized on the same day the spend happens.

So if we spend $1,000 on this channel, and the channel has a total ROI of 2x, we expect to get 25% of $2,000 = $500 on the day the spend happens (day 0).

And then we can see on the chart what percent of effect is realized after any number of days. Here we can see that after 10 days we expect to realize 87% of the total effect.

Saturation Curves

Saturation curves show how Recast expects the ROI of a channel to change at different levels of spend. Sometimes these are referred to as “diminishing marginal returns” curves.

You can view each of these curves in different units, showing either the ROI, marginal ROI, or total impact at different levels of daily spend.

📘

By default, the curves all show the expected saturation as of the last day modeled, and those curves might change depending on seasonality or holiday or promo schedules.


🌾 Intercept

The “intercept” shows you your brand’s “baseline” sales. That is, how many sales would you have in the absence of any marketing activity (or at least the marketing activity included in your Recast model). You can view the intercept on its own or you can view the intercept combined with the “spikes” (holidays, promo events, etc) all in one chart.

✳️ Spike Summary

“Spikes” includes information about certain promotional events that are included in your Recast model. These types of events are generally things like:

  • Time-limited promotional or discounting events (e.g., 20% off Memorial day sale)
  • Holidays (BFCM, Christmas, New Years, Mother’s Day, etc.)
  • Other important events (conference day, partner email blast, etc.)

The Recast model shows the true full impact of a spike over time. This might include positive effects on the day of the spike, but then negative effects before and after due to “pull forward” or “pull backward” effects. You can compare the size and shape of these effects directly in the tool:

🖼️ Context summary

The Context Summary is a useful tool for companies who have models with additional factors affecting their marketing outcomes. For example, if you find that after a price change or new offer, your sales are performing differently, our model will be able to measure these changes. The context summary is where you can see the effects of these contextual variables on your sales and marketing spend.

❗️

Contextual variables must be pre-configured in your Recast model before the report will be available

Interpreting the Graphs

The first graph in this report is the Effect of Contextual Metrics graph. This allows you to compare the % change in your marketing effectiveness and organic effectiveness as a result of changes in your contextual variables.

The next graph is the variable over time graph. This shows the change in the contextual variable we are measuring over time. In the graph below, we can see that the contextual variable was a one time price increase and a one time decrease.

The final graph is the effect over time graph. This shows the relative change in the marketing and organic effectiveness.

  • When the price was low, marketing and organic effectiveness was ~9% better than now
  • When price was high, marketing and organic effectiveness was ~4% lower than it is now.

⬅️ Backtests

The backtests tab is probably the most important tab in the entire Recast platform. The backtests tab helps us understand how well your Recast model does at predicting the future on data the model has never seen before. We believe this is the most important way to evaluate the accuracy of an MMM model.

The way the Recast backtests tab works is that it shows you how well models trained on data in the past do at predicting the future. So if we trained a model 60 days ago, how well does that model do at predicting the next 60 days (data that we now have, but that the model never saw).

↪️ Prior vs Posterior

The Prior vs Posterior tab shows you a comparison between the model’s time shift and intercept priors and the posteriors. This helps you to see how your initial assumptions about your business compare to Recast’s measurement.

What is the posterior?

The posterior is the calculated estimate the model makes based on your past spend and return data. These include estimates of the intercept and timeshifts (channel ROI coming soon).

How to use the Prior vs Posterior report?

This is useful to help you check your assumptions about the effects of your marketing spend. It can help you visualize where the data disagrees particularly strongly with your prior estimates, and can be useful in identifying areas where revisiting the priors may be helpful.

❓Insights FAQs

What is the unexplained variation on the waterfall chart?

The simple linear regression formula you may have learned in school is: y = x_slope + residual error. Our model is like a really complicated regression. It produces a prediction for your sales in the last seven days, and the difference between the prediction (x_slope) and what actually happened (y) is the residual error, which is what we show in the waterfall chart.

The unexplained variation number will change every week because it’s just a random deviation specific to that week. One week sales may be a little higher than our prediction, the next week a little lower. Over the long run, the average of these deviations should be close to zero (you’ll have as many positive residuals as negative). The unexplained variation is not a measure of how confident we are in any particular channel (we provide confidence intervals for that), and it’s not a prediction of how far off the model will be in the future (we recommend the forecaster to see how big the range of possible outcomes is for a given scenario).

Can you give me an overview of the interpretation of the dashboard?

We sure can!

How do we interpret the intercept chart?

What does the intercept mean?

📘

If you turned off all your marketing spend, and then waited for all the effect of your marketing spend to wear off (e.g. 3 months) how much of your business would still remain?

The intercept can be thought of as the amount of revenue (or conversions for a customer acquisition model) not attributable to marketing efforts.

It is often called “organic” although that can mean different things in different organizations so we avoid the term.

What is reflected in the intercept?

The intercept varies over time, so it can capture the effect of actions you take outside of marketing. For example, if you release a popular new product that brings lots of new customers, the intercept will trend up. If you introduce new email outreach programs that are successful, that will also be reflected in the estimate of the intercept.

We’re often asked if “word of mouth” is included in the intercept, and that depends on some of the details of your business. The ROI estimates Recast provides are estimates of true incrementality (how many additional sales would you have lost if you didn’t spend this money?). For some marketing efforts, the advertisement causes the person to purchase your product, and then they tell their friend who also purchases the product. This means that if the marketing spend was absent two customers would have been lost, so the incremental benefit was two new customers. To the extent possible, Recast will attribute these to the marketing channel ROIs; however, if the time lapse between the person who was driven by marketing and the person who purchased by word of mouth is large, the model will be unable to identify the causal link back to the marketing channel, and the intercept will absorb that effect.

❓Have a question not covered here? Email us at [email protected].

Updated 2 days ago