Investors often select project whose estimated performance meets or exceeds a hurdle value. At the time of decision making, the true performance of a project is unknown but uncertain forecasts are available. Decision makers (DM) often ignore the prediction errors when they use these forecasts to choose projects. To the disappointment of the DM, many selected projects result in smaller actual yields than the ones that were forecasted.
Some have attributed the cause of this to the optimistic bias of the predictions. This paper shows that this disappointment can occur even if the prediction is unbiased. In this case, a bias can be introduced by the selection process that will allow more unattractive (over-estimated) projects to be accepted than attractive (under-estimated) ones. Although a similar phenomenon has been noted in statistics and finance research, it is not well understood in the context of project selection by decision analysts.
We present a solution method based on Bayesian updating, and demonstrate its effectiveness in eliminating the disappointment in project selection with realistic data from oil E&P projects.
Capital investment decisions often involve selecting good projects from all available alternatives based on some criterion that measures or is closely related to project performance such as NPV, IRR, reserves of an oil well, etc. In many applications, the decision makers set a minimum value and a project will be undertaken if its performance measure meets or exceeds it; otherwise it will be rejected. For example, NPV of an investment must be greater than zero; IRR should be at least equal to the cost of the capital plus a risk premium; an oil well must have a reasonable reserve size to be profitable to develop.
However, at the time of decision making, the performances of the alternatives are usually unknown. For example, the NPV or IRR of a project will not be known with certainty before the project is undertaken. To resolve the difficulty, it is common to base the decisions on predicted values that can be evaluated before the project is undertaken.
A problem arises because predictions are subject to errors and will seldom equal to the actual values that are realized. If all alternatives would be carried out, the actual performance could be measured upon the completion of the projects, some of which would turn out to be better than predicted while some others would turn out to be worse. Nonetheless, decision makers often ignore the uncertainty because of prediction errors and proceed as if predicted values were true; i.e., they choose "optimal" projects by comparing the predictions with the hurdle threshold. A possible justification is that if the estimates are not systematically biased, then it seems that the positive and negative errors would cancel each other. Is this practice entirely satisfactory?
Empirical surveys (Pruitt and Gitman, 1987; Statman and Tyebjee, 1985 and references therein) show that many projects selected based on predicted performance actually have smaller yields than expected. Here is an example. Fig. 1 plots the actual versus the estimated sizes (in log scale) of oil reserves in the Norwegian sector of the North Sea reported by the Norwegian Petroleum Directorate in 1997. About 70% are overestimates and several reserves with very large forecasts turned out to be very poor, making the estimate much higher than the size of discovery.
The presence of prediction errors and the problems they cause have long been recognized. Some consider systematic biases inherent in the project estimation procedures to cause the erros. For example, prediction involves technical imperfections like systematic underestimations of costs, over-optimism in predicting cash flows, etc.; sometimes there are economic and political incentives to exaggerate the predictions, like the tendency to inflate estimates to compete for limited resources (see Pinches, 1982).