Expert Opinion Beats Predictive Analytics
A few entries back I posted an entry about Information Builders' bold prediction that the Los Angeles Dodgers would win the World Series. Well, that didn't work out so well. The Dodgers didn't even make it to the series and the Yankees easily beat the Phillies as 9 out of 10 ESPN experts had predicted.
Is this a black-eye for predictive analytics just when its getting all the attention?
I think it is. Think about it, this was exactly the type of scenario that predictive analytics is supposed to excel in. Forty years of data, a consistent yearly outcome (a World Series winner), and tons of readily available statistics that can be correlated to previous outcomes to predict a future outcome. Surely, that would be more accurate than a bunch of guys who based their predictions almost completely on this years performance and who's hot going into the playoffs. But it was the experts who got it right.
To me it proves a major point about predictive analytics. It should not replace expert opinion, it should supplement it. I think expert opinion is more accurate for short-term plans and unforeseen anomalies while predictive analytics is better for determining correlation across performance indicators, longer term forecasting, and what-if scenario modeling.
In the coming age of widespread predictive analytics, don't get too caught up in the hype. Don't ignore the experts in your business, the ones that know it inside and out, from many levels of the organization.
Without these experts to put a credibility check on predictive analytics your company might just have bet the bank on The Los Angeles Dodgers.
Predictive analytics will be game changing but in most cases this should not be an "either or" choice. Both predictive analytics and expert opinion need to be involved in decision making. A balance between predictive analytics and expert opinion certainly would have predicted the Yankees in six...
... and Hideki Matsui as the Series MVP
Is this a black-eye for predictive analytics just when its getting all the attention?I think it is. Think about it, this was exactly the type of scenario that predictive analytics is supposed to excel in. Forty years of data, a consistent yearly outcome (a World Series winner), and tons of readily available statistics that can be correlated to previous outcomes to predict a future outcome. Surely, that would be more accurate than a bunch of guys who based their predictions almost completely on this years performance and who's hot going into the playoffs. But it was the experts who got it right.
To me it proves a major point about predictive analytics. It should not replace expert opinion, it should supplement it. I think expert opinion is more accurate for short-term plans and unforeseen anomalies while predictive analytics is better for determining correlation across performance indicators, longer term forecasting, and what-if scenario modeling.
In the coming age of widespread predictive analytics, don't get too caught up in the hype. Don't ignore the experts in your business, the ones that know it inside and out, from many levels of the organization.Without these experts to put a credibility check on predictive analytics your company might just have bet the bank on The Los Angeles Dodgers.
Predictive analytics will be game changing but in most cases this should not be an "either or" choice. Both predictive analytics and expert opinion need to be involved in decision making. A balance between predictive analytics and expert opinion certainly would have predicted the Yankees in six...
... and Hideki Matsui as the Series MVP
Posted by Dave Kasabian at 11/10/2009 10:18 PM 
Categories: Vendor News,General Musings
Tags: SAS Predictive Analytics Business Intelligence Performance Management Information Builders
Categories: Vendor News,General Musings
Tags: SAS Predictive Analytics Business Intelligence Performance Management Information Builders
Dave, you are absolutely correct. As a die-hard Phillies fan, I was ecstatic watching the Phillies shatter the predictive analytics crystal ball. However, as a believer in listening to forecasts from people who are closest to 'the market', like sales people in sales forecasting or commentators in baseball, I also had a sinking feeling the evil-empire Yankees would beat my beloved Phillies. From a business perspective, I’ve witnessed (see my blog) great companies utilize both statistical (predictive) forecasts and sales people forecasts together to get a complete trusted view of the future.
Reply to this
Southard. Thanks for the comment. I read your blog entry on the topic and particularly liked this statement:
"Unfortunately, no easy answer to this conundrum exists. The fact is that a statistical forecast is a valuable part of an overall forecast. It’s not the complete forecast, but it cannot be left out of the forecast"
It is the blended approach that we are both touting that will bring predictive analytics "to the masses". It needs to be seen as a part of the process, not the whole process. Otherwise, business users will resist it as a "big brother" type threat to their ability to manage the business.
Reply to this