Some of the underperformance encountered in the petroleum industry is attributed to cognitive biases in project assessments (estimates or forecasts). It has been shown in the literature that if biases are reduced, portfolio profitability is increased. We will demonstrate that biases can be measured quantitatively, and through using this information the reliability of subsequent assessments can be improved.
To improve reliability of probabilistic assessments, we utilize calibration curves and statistical de-biasing methods. Calibration curves show actual occurrence frequency of assessed events as a function of the estimated probability. Calibration curves can be generated by tracking probabilistic assessments and comparing them to actual performance. Typically, similar biases exist in subsequent assessments and, therefore, statistical de-biasing methods coupled with the calibration curve can be used to de-bias and improve these later assessments.
In this work, we assess the benefits of applying a continual process of forecast tracking, lookbacks, probabilistic calibration, and external adjustments to improve the reliability of probabilistic production forecasts. We present a hindcast (backtesting) case study for a set of 197 Barnett shale gas wells. We compared the impact of not updating the estimates (base case) with updating estimates using new production data and externally adjusting them using calibration data. At 12 months of production data, the externally adjusted assessments showed marked reduction in overconfidence bias (underestimation of uncertainty) compared to the base case and compared to updating forecasts using production data only. The original unadjusted probabilistic assessments generated with the Markov-Chain-Monte-Carlo (MCMC) method coupled with Arps decline-curve-analysis (DCA) had a moderate overconfidence bias equal to 0.27 on a scale of 0 (no overconfidence) to 1 (complete overconfidence). De-biasing the assessments lowered overconfidence bias significantly to 0.11.
Since Capen proposed similar bias-correction methodology in 1976, only a few papers have been published in the petroleum literature demonstrating application of calibration methodology to reduce biases in probabilistic assessments. Implementing a continual process of tracking assessments, checking calibration, quantifying biases, and adjusting subsequent assessments as we propose should result in more reliable assessments, better identification of superior and inferior projects, better investment decisions, and more profitability over the long run.
