Foreword |
|
xi | |
Preface |
|
xiii | |
Acknowledgments |
|
xv | |
Office Memorandum |
|
xvii | |
|
|
1 | (12) |
|
1.1 Why Have We Written This Book? |
|
|
2 | (1) |
|
1.2 What's Different about This Metrics Book? |
|
|
3 | (2) |
|
1.3 Who Are We Writing This For? |
|
|
5 | (1) |
|
|
5 | (3) |
|
|
5 | (2) |
|
|
7 | (1) |
|
1.5 What We'll Be Talking about |
|
|
8 | (1) |
|
1.6 Defining Our Terminology |
|
|
9 | (1) |
|
1.7 What We Expect of You, the Reader |
|
|
10 | (1) |
|
|
11 | (2) |
|
2 Why Measure Information Security? |
|
|
13 | (16) |
|
2.1 To Answer Awkward Management Questions |
|
|
15 | (3) |
|
2.2 To Improve Information Security, Systematically |
|
|
18 | (2) |
|
2.3 For Strategic, Tactical, and Operational Reasons |
|
|
20 | (2) |
|
2.4 For Compliance and Assurance Purposes |
|
|
22 | (1) |
|
2.5 To Fill the Vacuum Caused by Our Inability to Measure Security |
|
|
23 | (1) |
|
2.6 To Support the Information Security Manager |
|
|
24 | (1) |
|
|
25 | (1) |
|
2.8 For Various Other Reasons |
|
|
26 | (1) |
|
|
27 | (2) |
|
3 The Art and Science of Security Metrics |
|
|
29 | (22) |
|
3.1 Metrology, the Science of Measurement |
|
|
30 | (1) |
|
3.2 Governance and Management Metrics |
|
|
30 | (2) |
|
3.3 Information Security Metrics |
|
|
32 | (1) |
|
3.4 Financial Metrics (for Information Security) |
|
|
33 | (2) |
|
3.5 (Information Security) Risk Management Metrics |
|
|
35 | (1) |
|
3.6 Software Quality (and Security) Metrics |
|
|
36 | (1) |
|
3.7 Information Security Metrics Reference Sources |
|
|
37 | (9) |
|
3.7.1 Douglas Hubbard: How to Measure Anything (Hubbard 2010) |
|
|
37 | (1) |
|
3.7.2 Andrew Jaquith: Security Metrics (Jaquith 2007) |
|
|
38 | (1) |
|
3.7.3 NIST SP 800-55: Performance Measurement Guide for Information Security (NIST 2008) |
|
|
39 | (1) |
|
3.7.4 Debra Herrmann: Complete Guide to Security and Privacy Metrics (Herrmann 2007) |
|
|
40 | (1) |
|
3.7.5 W. Krag Brotby: Information Security Management Metrics (Brotby 2009a) |
|
|
41 | (1) |
|
3.7.6 Lance Hayden: IT Security Metrics (Hayden 2010) |
|
|
41 | (1) |
|
3.7.7 Caroline Wong: Security Metrics: A Beginner's Guide (Wong 2012) |
|
|
42 | (1) |
|
3.7.8 ISO/IEC 27004: Information Security Management-Measurement (ISO/IEC 27004 2009) |
|
|
42 | (1) |
|
3.7.9 CIS Security Metrics (CIS 2010) |
|
|
43 | (1) |
|
|
44 | (2) |
|
|
46 | (2) |
|
3.9 Metrics Catalogs and a Serious Warning about SMD |
|
|
48 | (1) |
|
3.10 Other (Information Security) Metrics Resources |
|
|
49 | (1) |
|
|
50 | (1) |
|
4 Audiences for Security Metrics |
|
|
51 | (8) |
|
4.1 Metrics Audiences Within the Organization |
|
|
52 | (5) |
|
|
53 | (1) |
|
4.1.2 Middle and Junior Management |
|
|
54 | (1) |
|
4.1.3 Security Operations |
|
|
55 | (1) |
|
4.1.4 Others with Interest in Information Security |
|
|
56 | (1) |
|
4.2 Metrics Audiences From Without the Organization |
|
|
57 | (1) |
|
|
58 | (1) |
|
5 Finding Candidate Metrics |
|
|
59 | (16) |
|
5.1 Preexisting/Current Information Security Metrics |
|
|
60 | (1) |
|
5.2 Other Corporate Metrics |
|
|
61 | (5) |
|
5.3 Metrics Used in Other Fields and Organizations |
|
|
66 | (1) |
|
5.4 Information Security Metrics Reference Sources |
|
|
67 | (1) |
|
5.5 Other Sources of Inspiration for Security Metrics |
|
|
68 | (2) |
|
|
68 | (1) |
|
5.5.2 Vendor Reports and White Papers |
|
|
69 | (1) |
|
|
70 | (1) |
|
5.6 Roll-Your-Own Metrics |
|
|
70 | (1) |
|
5.7 Metrics Supply and Demand |
|
|
71 | (1) |
|
|
72 | (3) |
|
6 Metametrics and the PRAGMATIC Approach |
|
|
75 | (40) |
|
|
76 | (2) |
|
6.2 Selecting Information Security Metrics |
|
|
78 | (3) |
|
|
81 | (14) |
|
|
82 | (3) |
|
|
85 | (1) |
|
|
86 | (1) |
|
|
87 | (1) |
|
|
88 | (2) |
|
|
90 | (1) |
|
|
91 | (2) |
|
|
93 | (1) |
|
|
94 | (1) |
|
6.4 Scoring Information Security Metrics against the PRAGMATIC Criteria |
|
|
95 | (9) |
|
6.5 Other Uses for PRAGMATIC Metametrics |
|
|
104 | (1) |
|
6.6 Classifying Information Security Metrics |
|
|
105 | (8) |
|
6.6.1 Strategic/Managerial/Operational (SMO) Metrics Classification |
|
|
106 | (2) |
|
6.6.2 Risk/Control Metrics Classification |
|
|
108 | (1) |
|
6.6.3 Input-Process-Output (Outcome) Metrics Classification |
|
|
108 | (1) |
|
6.6.4 Effectiveness and Efficiency Metrics Classification |
|
|
109 | (1) |
|
6.6.5 Maturity Metrics Classification |
|
|
109 | (1) |
|
6.6.6 Directness Metrics Classification |
|
|
110 | (1) |
|
6.6.7 Robustness Metrics Classification |
|
|
110 | (1) |
|
6.6.8 Readiness Metrics Classification |
|
|
111 | (1) |
|
6.6.9 Policy/Practice Metrics Classification |
|
|
112 | (1) |
|
|
113 | (2) |
|
7 150+ Example Security Metrics |
|
|
115 | (130) |
|
7.1 Information Security Risk Management Example Metrics |
|
|
118 | (12) |
|
7.2 Information Security Policy Example Metrics |
|
|
130 | (10) |
|
7.3 Security Governance, Management, and Organization Example Metrics |
|
|
140 | (20) |
|
7.3.1 Information Security Financial Management Metrics |
|
|
141 | (1) |
|
7.3.2 Information Security Control-Related Metrics |
|
|
141 | (1) |
|
7.3.3 Metrics for Business Alignment and Relevance of Controls |
|
|
142 | (1) |
|
7.3.4 Control Monitoring and Testing Metrics |
|
|
143 | (13) |
|
7.3.5 Financial Information Security Metrics |
|
|
156 | (4) |
|
7.4 Information Asset Management Example Metrics |
|
|
160 | (4) |
|
7.5 Human Resources Security Example Metrics |
|
|
164 | (15) |
|
7.6 Physical Security Examples |
|
|
179 | (9) |
|
7.7 IT Security Metric Examples |
|
|
188 | (15) |
|
7.8 Access Control Example Metrics |
|
|
203 | (5) |
|
7.9 Software Security Example Metrics |
|
|
208 | (9) |
|
7.10 Incident Management Example Metrics |
|
|
217 | (8) |
|
7.11 Business Continuity Management Examples |
|
|
225 | (7) |
|
7.12 Compliance and Assurance Metrics Examples |
|
|
232 | (12) |
|
|
244 | (1) |
|
8 Designing PRAGMATIC Security Measurement System |
|
|
245 | (22) |
|
8.1 Brief History of Information Security Metrics |
|
|
246 | (2) |
|
8.2 Taking Systems Approach to Metrics |
|
|
248 | (1) |
|
8.3 Information Security Measurement System Lifecycle |
|
|
249 | (17) |
|
|
266 | (1) |
|
9 Advanced Information Security Metrics |
|
|
267 | (12) |
|
9.1 High-Reliability Metrics |
|
|
268 | (3) |
|
9.2 Indicators and Proxies |
|
|
271 | (1) |
|
|
272 | (3) |
|
9.3.1 Key Goal Indicators (KGIs) |
|
|
272 | (1) |
|
9.3.2 Key Performance Indicators (KPIs) |
|
|
273 | (1) |
|
9.3.3 Key Risk Indicators (KRIs) |
|
|
274 | (1) |
|
9.3.4 Critical Success Factors (CSFs) |
|
|
275 | (1) |
|
9.4 Targets, Hurdles, Yardsticks, Goals, Objectives, Benchmarks, and Triggers |
|
|
275 | (2) |
|
|
277 | (2) |
|
|
279 | (10) |
|
10.1 Numbers Don't Always Tell the Whole Story |
|
|
279 | (2) |
|
10.2 Scoring Political Points through Metrics |
|
|
281 | (1) |
|
10.3 Implausible Deniability |
|
|
282 | (1) |
|
|
283 | (1) |
|
10.5 On Being Good Enough |
|
|
284 | (1) |
|
|
285 | (2) |
|
|
287 | (2) |
|
11 Using PRAGMATIC Metrics in Practice |
|
|
289 | (32) |
|
|
290 | (7) |
|
|
290 | (1) |
|
11.1.2 Automated Data Sources |
|
|
291 | (2) |
|
11.1.3 Observations, Surveys, and Interviews |
|
|
293 | (1) |
|
11.1.4 Online or In-Person Surveys |
|
|
294 | (1) |
|
|
295 | (1) |
|
11.1.6 Audits, Reviews, and Studies |
|
|
296 | (1) |
|
11.2 Data Analysis and Statistics |
|
|
297 | (5) |
|
|
302 | (14) |
|
11.3.1 General Considerations |
|
|
302 | (1) |
|
11.3.2 Analytical Tools and Techniques |
|
|
303 | (2) |
|
11.3.3 Reporting Tools and Techniques |
|
|
305 | (2) |
|
11.3.4 Presentational Tools and Techniques |
|
|
307 | (3) |
|
11.3.5 Graphs, Figures, Diagrams, and Illustrations |
|
|
310 | (5) |
|
11.3.6 Drawing Attention to Specific Issues |
|
|
315 | (1) |
|
11.4 Using, Reacting to, and Responding to Metrics |
|
|
316 | (3) |
|
11.4.1 Periodic versus Event-Driven Reporting |
|
|
318 | (1) |
|
|
319 | (2) |
|
|
321 | (42) |
|
12.1 The Context: Acme Enterprises, Inc. |
|
|
322 | (1) |
|
12.2 Information Security Metrics for C-Suite |
|
|
323 | (35) |
|
12.2.1 Information Security Metrics for the CEO |
|
|
328 | (11) |
|
12.2.2 Information Security Metrics for the CIO |
|
|
339 | (3) |
|
12.2.3 Information Security Metrics for the CISO |
|
|
342 | (6) |
|
12.2.4 Information Security Metrics for the CFO |
|
|
348 | (1) |
|
12.2.5 Information Security Metrics for the VP of Production |
|
|
349 | (3) |
|
12.2.6 Information Security Metrics for the VP of Marketing |
|
|
352 | (6) |
|
12.3 Information Security Metrics for Management and Operations |
|
|
358 | (1) |
|
12.4 Information Security Metrics for External Stakeholders |
|
|
359 | (1) |
|
12.5 Acme's Information Security Measurement System |
|
|
360 | (1) |
|
|
361 | (2) |
|
|
363 | (14) |
|
13.1 Take-Home Lessons from This Book |
|
|
364 | (3) |
|
13.1.1 On Pragmatism and Being PRAGMATIC |
|
|
364 | (1) |
|
13.1.2 On Giving You the Confidence and Skills to Have a Go |
|
|
365 | (1) |
|
13.1.3 On Improving the Quality of Your Management Information through Metametrics |
|
|
366 | (1) |
|
13.1.4 On Improving Metrics of All Sorts |
|
|
367 | (1) |
|
13.2 Your Chance to Advance the Profession and the Practice of Metrics |
|
|
367 | (2) |
|
13.3 An Action Plan to Take Away |
|
|
369 | (1) |
|
|
370 | (7) |
|
Appendix A PRAGMATIC Criteria |
|
|
377 | (4) |
|
Appendix B Business Model of Information Security (BMIS) |
|
|
381 | (4) |
|
Appendix C Capability Maturity Model (CMM) |
|
|
385 | (4) |
|
|
385 | (1) |
|
|
386 | (1) |
|
|
386 | (1) |
|
|
386 | (1) |
|
|
387 | (2) |
|
Appendix D Example Opinion Survey Form |
|
|
389 | (2) |
|
Security Awareness Survey on Malware |
|
|
389 | (2) |
|
Appendix E SABSA Security Attributes Table |
|
|
391 | (20) |
|
Appendix F Prototype Metrics Catalog |
|
|
411 | (16) |
|
Appendix G Effect of Weighting the PRAGMATIC Criteria |
|
|
427 | (4) |
|
Appendix H ISO27k Maturity Scale Metrics |
|
|
431 | (44) |
|
Appendix I Sample Management Survey |
|
|
475 | (2) |
|
|
477 | (4) |
|
Appendix K Observer Calibration |
|
|
481 | (2) |
|
|
483 | (4) |
Index |
|
487 | |