"Fast is fine, but accuracy is everything." Xenophon (BC 431 - 350) "You can't always get what you want … but if you try sometimes you just might find you get what you need" Mick Jagger (1943 -)
On a simple level accuracy is well understood: the closer the results of a model match reality, the more accurate the model. Indeed, the most common definition of "error" is a measurement of the deviation of observed behaviour from expected behaviour. However, no model is perfect and so there will always be errors. Errors come from many different sources: from assumptions made when formulating the problem, through numerical error endemic in any non-analytic solution of the model equations to errors in the (measurement) observations against which the model results are being judged. Ultimately, model accuracy is affected by many mechanisms, some of which are only apparent when considering transient phenomena. In the pursuance of accuracy some of the sources of errors and issues of complexity can be addressed and their effects mitigated. This comes, perhaps, at the expense of accuracy elsewhere or with increased data requirements. Almost always it comes at the expense of computational performance.
Recent investigations by the authors into the comparison of accuracy of various numerical methods using various levels of modelling complexity have highlighted some interesting accuracy issues and sources of error that are common to many simulators. Interesting numerical artefacts have been discovered that have caused us to revise our thoughts about the metrics we typically apply to describe accuracy. This paper discusses some of the complexities around the seemingly simple concept of accuracy whilst describing some of the different sources of "errors" in a simulator.
Through some relatively simple examples and case studies the nature and importance of errors influencing the results of a model are discussed starting with a number of very relevant questions:
What is accuracy?
Why are we concerned about accuracy?
What level of accuracy is required and acceptable?
Understanding the nature of errors is one thing but how do we consider accuracy when building a model?
Finally, we provide some suggestions as to what could be done to help the modeler to build better models taking into account requirements for functionality, accuracy, speed and ease-of-use.
The simplest definition of accuracy is the degree of conformity of a measured or calculated quantity to its actual (true) value1. This is not to be confused with "precision" which is a measure of the degree of repeatability of the result. A widely used analogy to help distinguish the two concepts is that of firing arrows at a target and attempting to hit the bulls eye: Accuracy is a measure of how close the arrows are to the bulls eye whereas precision is how closely grouped the arrows fall.
