Mix Models Mislead Marketers?

By Pat LaPointe

Being a huge proponent of structured measurement and decision modeling (and alliteration too), it’s with mixed emotions that I raise the topic of the enormous dangers facing marketers tied to their mix models. If you are using a mix model to help you refine your marketing mix resource allocation, listen up…

The vast majority of mix models out there are regression models which seek to find correlations in historical data, and then apply those correlations to forecasting what would happen next “if…”. The problem is the historical data part. In a discontinuous environment like our current economy, the old “norms” of consumer/customer behavior no longer hold true. Consequently, all those model correlations painstakingly developed over 2 to 5 years of data are now largely invalid as predictors of the future.  Using a mix model developed over the past 5 years or so, even if it’s been regularly updated, would be like using last year’s Wall St. Journal to pick stocks. Bad idea.

So what’s a marketer to do?

First of all, don’t abandon modeling as a practice. On the whole, modeling is a terrific exercise that gets us to clearly state and test our assumptions and capture our experience in structured ways. However, your mix model needs supplemental assessment right now.  Continue to feed it new data and refine the inner correlations, but be more diligent in challenging the outputs. Form a team of your best internal and external experts to A) predict how you think the model will perform in the new reality of today; B) test it by flexing the most sensitive driver variables to see how it responds; C) challenge and interpret the possible error ranges in the outputs; and D) explore alternatives to the current model structure in anticipation of managing in an era of greater uncertainty and discontinuity.

This is no time to stick your head in the analytical sand. Your model may have gone bad, but your efforts to tune/evolve/replace it will surely pay hansom dividends moving forward.
________________________   
Pat LaPointe is Managing Partner at MarketingNPV – specialists in measuring the payback on marketing investments, and publishers of MarketingNPV Journal.

Comments

[John Dawson]

Nice points but I'd add the following;

1) you should have been doing all of the above for the last 5 years anyway - now's a good time to learn

2) finding a "broken" model isn't a bad thing - that's why refreshing models frequently is important since is highlights changes in the market

3) Now is a great time to decide whether you need models accurate to 3 decimal places and created on an annual basis or models with less accuracy but available almost on demand. There are providers of the latter out there.

John