IWSM 2018 Forums Estimating Reference Class Forecasting

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
  • #2003
    Frank Vogelezang

    In 2002 Daniel Kahnemann shared the Nobel Prize for Economics “for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty.” A few years ago I read his best-selling book, Thinking Fast and Slow. In this text Kanhemann describes how planning typically falls into one of two perspectives – an Inside View and an Outside View.
    Those that prepare estimates using an “Inside View” quickly focus on specific circumstances and search for evidence using their own experiences (“Expert Judgment”). This is a significant contributor to what he calls a “Planning Fallacy” (a bad estimate).
    Kahnemann explains that preparing an estimate using an Outside View requires the estimator to use a reference class; some empirical evidence from a similar activity. This use of the Outside View has become known as “Reference Class Forecasting.” For an IT estimate, this Outside View may take the form of a large/international database with plans and outcomes from a myriad of projects – provided they can provide statistical information about the likely outcome. The selection of a reference class is simple: (paraphrased from Thinking Fast and Slow):

    1. Identify an appropriate reference class (IT projects, painting a house, building a race car, etc.).
    2. Obtain the statistics of the reference class (in terms of number of servers, square meters of walls, go-cart or formula-one race car). Statistics are used to generate a baseline estimate.
    3. Use specific information about the case to adjust the baseline prediction, if there are particular reasons to expect an optimistic bias to be more or less pronounced in this project than in others of the same type.

    Unfortunately, the “Inside View” is generally the first choice selected when making an estimate. There is ample evidence that reveals that most estimators (of anything) are overly optimistic of their capabilities and hence make bad estimates – in other words they are comfortable with their “Inside View”. But still the “Inside View” seems indestructible.

    How can we stop people from making this mistake over and over again?

    Frank Vogelezang

    Many thanks to Dave DeWitt by raising the subject in his blog Fool the Guesser.

Viewing 2 posts - 1 through 2 (of 2 total)
  • You must be logged in to reply to this topic.