For my last session of the first day of the TRISC 2009 Conference, I made the mistake of attending Ricky Allen and Randy Holloway’s presentation on “The Importance of Log Management in Today’s Insecure World”.  I say “mistake” because out of all of the presentations I attended over the entire three days of the conference this was by far the most vendory, the least security oriented, and the worst presentation.  Both of these guys work for ArcSight and while they certainly know their log managment, it was just a lame excuse for a presentation and if I was able to go back in time I would have attended Chip Meadows’ presentation on “Pocket protectors, Purple hair and Paranoia” instead as I heard he did a fantastic job.  Anyway, my notes from this presentation are below and the actual slides can be found here:

What is log management?

  • Ensuring your enterprise log data is accessible, easily retrievable and forensically sound
  • Properly dealing with mammoth amounts of event data stores in thousands of vendor generated log files
  • Achieving compliance (SOX, HIPAA, PCI, FISMA), Security and IT operation usage of log data that does not break the bank
  • Log data now represents over 30% of ALL data generated by enterprises – creating a real need for log management
  • Dominant uses for log data include:
    • IT operations – systems/network health and availability
    • Security monitoring – perimeter or insider threat detection
    • Compliance monitoring – for regulations and industry standards

Why should I care?

  • Overwhelming flood of logs
  • Islands of defense
  • Week long manual investigations
  • Massive false positives
  • Heterogeneous consoles
  • Many different formats
  • Regulations and their commonly used frameworks impose various requirements when it comes to log management
  • Regulatory mandates have further increased log retention requirements
  • Increased need to store both security and non-security
  • There continues to be an increased emphasis on audit quality data collection
  • Regulatory requirements
    • SOX: 7yrs
    • PCI: 1yr
    • GLBA: 6yrs
    • EU DR Directive: 2yrs
    • Basel II: 7yrs
    • HIPAA: 6/7yrs
  • Compliance requirements
    • More logging
    • More types of devices
    • Higher volumes of log data
    • Extensive reporting requirements
    • Broader user access
    • Long term retention requirements
    • Audit quality data

What can effective log management do for me?

  • Self-managing & scalable
  • Automated & cost-effective audits
  • IT Operations SLA Efficiency
  • Compliance
  • Simplified Forensic Investigations

Best Practices – NIST 800-92

  • Common log management problems
    • Poor tools and training for staff
    • Laborious and boring
    • Reactive analysis reduces the value of logs
    • Slow response
  • Solutions
    • Establish log management policies & procedures
    • Prioritize log management appropriately
    • Create and maintain a secure log management infrastructure
    • Provide proper support for all staff with log management responsibilities
    • Establish standard log management processes for system-level admins
  • The directive to only log and analyze data this is of the greatest importance helps provide sanity to the logging process
  • Collecting and storing all data regardless of its usefulness increases complexity and deployment costs
  • Secure storage and transmission guideline directly points to the importance of secure and robust capture, transmission and storage of logs
  • Organizations should carefully review the collection architecture, transmission security and access control capabilities of SEM solutions to ensure support of this section of the standard
  • Filtering and aggregation are recommended as a means to only capture logs of security and compliance value based on the corporate retention policy
  • Guideline helps organizations support a “reasonableness” position in not collecting useless log data

Developing a Log Management Program

  • Understand your log management needs (regulatory and operational requirements)
  • Review NIST 800-92
  • Understand your environment
    • Lots devices to collect logs from
    • Multiple locations with no IT staff
    • Collection agents are not an option
    • Network time settings
    • Low bandwith links
  • Devices
    • Firewalls/VPN
    • IDS/IPS
    • Servers and desktop OS
    • Network equipment
    • Vulnerability assessment
    • Anti-virus
    • Applications
    • DBs
    • Physical infrastructure
  • Establish prioritized log management policies & procedures

Log Management Checklist

  1. Scalable architecture
  2. Minimal footprint at remote sites
  3. Transaction assurance
  4. Audit and litigation quality data
  5. Universal event collection
  6. Ease of manageability
  7. ….