Certification and evaluation: A security economics perspective

There has been some discussion in the industrial control system security community of evaluation and certification. There are already at least two independent third party evaluators, and some have advocated common criteria certification of products used in critical systems. The broader IT security community has considerable experience of evaluation and certification, which we seek to summarise and share in this paper. Certification is not a silver bullet, and can very easily end up as spin rather than substance: as `security theatre' designed to reassure customers or regulators rather than a genuine risk-reduction mechanism. It can also be very expensive, and once entrenched it can impose deadweight costs on industry that are difficult to eliminate even when certification processes are widely seen as failing. We discuss a number of further issues such as perverse incentives, usability and liability and argue that the industry should proceed with great caution.