Monday Keynote:
Measuring Security Outcomes

Monday, 25 April 2022
9:20am – 10:20am

Security is measurable. Or rather, security outcomes (broadly defined) are measurable. In the most basic form the measure of insecurity is the sum total of harm experienced. Success of security efforts can be measured based on the rate of decrease of this harm. If harm is decreasing, then we are succeeding. If harm is increasing, then we are failing. While such measurement of outcomes is essential for post-hoc assessment of overall progress, it does not directly help In guiding new decisions. For that we need predictive heuristics

In order for these predictive metrics to be valid their correlation with actual outcomes must be continuously empirically validated. Unfortunately, most of the predictive security metrics in use today, plausible as they seem, have never been empirically correlated with harm reduction. Thus, establishing reliable measurement of security outcomes is essential for developing better predictive metrics and driving the evolutionary progress of our field. Our domain is not unique in this challenge and we can learn a lot from how other fields have approached it.