13Nov/090
Application Security Metrics from the Organization on Down to the Vulnerabilities
This presentation was by Chris Wysopal, the CTO of Veracode. My notes are below:
"To measure is to know." - James Clerk Maxwell
"Measurement motivates." - John Kenneth Galbraith
Metrics do Matter
- Metrics quantify the otherwise unquantifiable
- Metrics can show trends and trends matter more than measurements do
- Metrics can show if we are doing a good job or bad job
- Metrics can show if you have no idea where you are
- Metrics establish where "You are here" really is
- Metrics build bridges to managers
- Metrics allow cross sectional comparisons
- Metrics set targets
- Metrics benchmark yourself against the opposition
- Metrics create curiosity
Metrics Don't Matter (Mike Rothman)
- It is too easy to count things for no purpose other than to count them
- You cannot measure security so stop
- This following is all that matters and you can't map security metrics to them:
- Maintenance of availability
- Preservation of wealth
- Limitation on corporate liability
- Compliance
- Shepherding the corporate brand
- Cost of measurement not worth the benefit
Bad metrics are worse than no metrics
Security Metrics Can Drive Executive Decision Making
- How secure am I?
- Am I better off than this time last year?
- Am I spending the right about of money?
- How do I compare to my peers?
- What risk transfer options to I have?
Goals of Application Security Metrics
- Provide quantifiable information to support enterprise risk management and risk-based decision making
- Articulate progress towards goals and objectives
- Provides a repeatable, quantifiable way to assess, compare, and track improvements in assurance
- Focus activities on risk mitigation in order of priority and exploitability
- Facilitate adoption and improvement of secure software design and development processes
- Provide and objective means of comparing and benchmarking projects, divisions, organizations, and vendor products