Cohen's Kappa and Classification Table Metrics 2.0: An ArcView 3x Extension for Accuracy Assessment of Spatially Explicit Models: USGS Open-File Report 2005-1363

In the field of spatially explicit modeling, well-developed accuracy assessment methodologies are often
poorly applied. Deriving model accuracy metrics have been possible for decades, but these calculations
were made by hand or with the use of a spreadsheet application. Accuracy assessments may be useful
for: (1) ascertaining the quality of a model; (2) improving model quality by identifying and correcting
sources of error; (3) facilitating a comparison of various algorithms, techniques, model developers and
interpreters; and, (4) determining the utility of the data product in a decision-making context. When
decisions are made with models of unknown or poorly-assessed accuracy, resource managers run the
risk of making wrong decisions or drawing erroneous conclusions. Untested predictive surface maps
should be viewed as untested hypotheses and, by extension, poorly tested predictive models are poorly
tested hypotheses. Often, if any accuracy measure is provided at all, only the overall model accuracy is
reported. However, numerous accuracy metrics are available which can describe model accuracy and
performance. Because issues concerning data quality and model accuracy in landscape analyses have
received little attention in the management literature, we found it useful to develop a systematic and
robust procedure for assessing the accuracy of spatially explicit models. We created an ArcView 3.x
extension that provides end users with a packaged approach for accuracy assessment, using Cohen's
Kappa statistic as well as several other metrics including overall accuracy, overall misclassification rate,
model specificity and sensitivity, omission and commission errors, and positive and negative predictive
power. Collectively, these metrics may be used for gauging model performance. When multiple models
are available, these metrics offer end users the ability to quantitatively compare and identify the "best"
model within a multi-criteria model selection process.