This paper presents methodology to analyze the reliability of reserves estimates filed with regulators in the U.S. and Canada. Using this methodology, we measured biases in the technical revisions presented in reserves reconciliation reports for 34 companies filing in Canada and 32 companies filing in the U.S from 2007 to 2017.
Filers in both Canada and the U.S. overestimated 1P reserves, and U.S. filers overestimated 1P reserves (51% positive technical revisions instead of 90%) more often than Canadian filers (72% positive technical revisions). Canadian filers underestimated 2P reserves slightly (54% positive technical revisions instead of 50%). Considering the entire reserves distribution, Canadian filers were moderately overconfident (underestimated uncertainty) and slightly pessimistic. U.S. filers, who report only 1P, were somewhere between (1) extreme overconfidence and neutral directional bias and (2) moderate overconfidence and extreme optimism.
Three groups of professionals can benefit from this study: (1) estimators, who can use the methodology to track their technical revisions over time, calibrate them, and use this information to improve future estimation procedures; (2) investors, who can analyze reported reserves estimates to compare volumes fairly; and (3) regulators, who will have quantitative methodology to suggest to filers to help them ensure that they are complying with appropriate criteria for 1P and 2P reserves and avoid significant reserves write-downs later.
Estimating reserves is an important process in most companies, as these estimates can have a large impact on company valuation. An estimate of reserves is inherently a probabilistic assessment; the different reserves categories (1P, 2P, 3P) quantify the uncertainty in this estimate. Unfortunately, humans are poor at assessing uncertainty; i.e., we are biased. Several authors have reported on the tendency for overconfidence and optimism in the petroleum industry (Capen 1976; Welsh et al. 2005; McVay and Dossary 2014).
Reliability in reserves estimates requires that, over a large number of these estimates, the frequency of outcomes would correspond to the probabilities of reserves stated by reserves definitions. Reserves volumes should be as reliable as possible so that investors can be confident they are comparing volumes fairly: “Tightly controlled and audited reserves volumes are meant to provide investors with the confidence that a barrel of reserves at Company A bears the same uncertainty as a barrel at Company B” (Beliveau and Baker 2003).