I’ve been focusing a lot of my time lately on our PCI initiatives.  One sub-topic that I’ve spent a particularly large amount of time on has been Requirement 11.2 which says that you need to have internal and external network vulnerability scans performed by a scan vendor qualified by PCI.  We already employ one such tool, but I’ve been working to evaluate several other vulnerability scanning tools to see where our current tool is at in comparison.  I’ll post my evaluations of each of these tools in time, but for now I’ll start with my evaluation of Rapid7 NeXpose.

First off, I had never heard of the company before, but they were among the cheaper options of what I evaluated and apparently are doing some good things.  They got the SC Magazine recommendation for the month of August 2008 and they received a 5-star overall rating in said magazine.  The problem came as soon as I started talking to their salesperson.  From the start, the guy was coming off like a used car salesman asking questions like “What would it take to get you to buy by the end of this month?”  This was before I even saw an evaluation of the product.  From that point forward, I don’t think a week went by where I didn’t hear from the salesperson.  “How’s the evaluation going?  Do you think you’re going to buy?”  It got annoying very quickly.

The evaluation of the product went fairly smoothly.  My biggest gripe was that the company claimed that they did everything that Qualys does and more (they even forwarded me a press release on it), but ultimately failed to deliver on that promise when I found something rather large that Qualys finds and NeXpose does not.  To their benefit, Rapid7 had engineers and developers calling me and asking about the issue trying to get it into their system for me.  That was pretty cool, but ultimately they’re getting paid to find these vulnerabilities for us.  You would think that they’d at least have all of the CVE items in their scanning tool.

My missing Qualys vulnerability aside, the NeXpose tool found plenty of issues.  This was both a positive and a negative since a lot of what it found had to do with a single vulnerability being exposed over and over through our site’s faceted navigation.  It would have been nice if the scanner recognized that since it made the results look a lot worse than it actually is.  Also, when going through the results, I noticed quite a few false positives.  It seemed like most of these were due to the scanner just looking at a version number in a header instead of actually trying to test the vulnerability.  It found issues with Apache modules that we didn’t even have enabled.

My favorite thing about the Rapid7 NeXpose vulnerability scanning tool was the reporting.  They provide some very good reports in there by default.  I found the “Remediation Plan Report” to be particularly interesting as it provided you with their suggested path to remediate our vulnerabilities most effeciently and effectively.  Was it better than the reporting that I’ve seen in other products?  Maybe, maybe not.

Anyway, my evaluation of Rapid7 NeXpose was coming to a close when I got a call from the salesperson last week.  It went something like this…

Salesperson: “Did you hear we got a recommendation from SC Magazine?  Yeah, things are busy here.  Your evaluation is taking longer than normal and I know you’ve had several issues with the product, do you think you’re going to buy it?”

Me: “Nope, hadn’t heard about the SC Magazine thing.  We’ve definitely worked through some issues.  Overall, the evaluation went well and I like the product.  Once I finish the other evaluations I’m working on, I’ll let you know our decision.”

Salesperson: “Well, with the amount of business we’re getting with the SC Magazine article, I don’t have time for you.  Feel free to call me back if you decide to buy our product, otherwise, good luck.”

What do you say to that?  I got dumped by a salesperson, who I kept dropping hints to leave me alone to do my evaluation, because I was taking up too much of his time?  It’s a little difficult to do an unbiased review after that, but I tried my best.