I absolutely love my job and one of the coolest things about what I do is getting to do proof-of-concepts with bleeding edge technology.  I feel very privileged that many companies out there respect me enough to provide me with these opportunities and I feel that engaging on this level enables me to be a better security practitioner because I routinely have my finger on the pulse of the latest and greatest tools out there.  The problem that I run into, however, is when vendors present me “enterprise ready” tools that are clearly not enterprise ready.  Maybe it’s a cool concept, a working prototype, or even a functional system.  The problem is that “enterprise ready” assumes so much more than just a product that does some stuff as advertised.  To me, at least, it assumes a product that can be easily transitioned to the company’s IT team for long-term support of the platform.   Here are some signs to look out for that will tell you if the tool is truly ready for the big show:

  1. Installation Process: This one could honestly go either way.  Personally, I prefer to have a product that I can install and configure myself.  I cringe every time I hear a vendor mention to me that professional services are involved in an installation.  I get it, sometimes a tool is so highly customized to your environment that you need help, but the majority of the products I use on a daily basis aren’t that way.  If installing a product requires a day of professional services time, then this should probably be your first signal to at least start looking out for the following additional signs.
  2. Initialization Script: I honestly feel a bit silly even having to mention this as I would assume this to be a standard part of any enterprise product, but it’s not.  If I have to poke around in the installation directory looking for the right script to run to start or stop your product, then it’s not enterprise ready.  Even worse, if it’s a more complex product that requires starting multiple pieces and you don’t have a single init script to handle the startup and shutdown in the proper order, then your product is not enterprise ready.  If you’re trying to sell me something to make my life as a security professional easier, then I should spend my time using your tool instead of figuring out how to start and stop it.
  3. Release Notifications: If I buy a product from you and I’m paying you for support, then, I’m typically doing so with the intention that I will be able to move to the next version once it is released.  Maybe it’s because there are bugs that need to be fix or because there is new functionality, but whatever the reason, I want to know when that version becomes available.  I’ll talk a bit more about the upgrade process itself in the next bullet, but if the company does not have a way to notify you when a new release is available, be wary.
  4. Defined Upgrade Process: Have you ever used a tool that you thought was completely awesome until the first time that an upgrade rolled around?  They tell you copy these files over and it breaks.  Now, run this script and it fails.  You engage support and spend hours on the phone with them and then a week later they offer a webex where a support person will take care of the upgrade for you.  I had to ditch a really interesting tool a while back for this very reason and I’m currently dealing with another one where every upgrade requires a support person to come onsite.  It’s a completely ineffective use of both my time and theirs.  When I designed SimpleRisk, one of the first things I considered was how to make it as simple as possible for a remote person to upgrade the tool without assistance.  I’ve at least got it down to copying some files and running a script which anyone can do.  Even better are the companies where it’s click a button to upgrade.  Better still are the companies that just automatically do the upgrade for you.  In any case, be wary of any upgrade processes that are not well-defined.
  5. Backup Plan: This may not apply to all products or all scenarios, but it’s a good idea when evaluating a product to ask yourself how you will back up the data and recover it if a disaster ever strikes.  If the answer is “We’d just wipe and reinstall”, then cool, but if the answer is “F*ck, I don’t know”, it may be worth having that discussion with the vendor.
  6. Monitoring: Nothing bothers me more than when I’m all excited to use my shiny new toy and when I go to log in it’s down.  In reality, I should know it’s down when it happens because there’s a high likelihood that the tool isn’t doing what it’s supposed to if it’s not running.  Ask your vendor what you should be monitoring in order to ensure that the tool is functioning properly.  If they don’t have a good answer for you, be wary.
  7. Product Roadmap: When you purchase a product, you purchase it not only for what it’s capable of doing for you today, but also for the opportunities that it will provide you with tomorrow.  Ask the vendor about their product roadmap to see if it’s in-line with your vision of how you intend to use the product.  Are there features that you can use down the line?  More importantly, do they have plans to continue to invest in the platform that they are selling you or is it just major bug fixes at this point while they move on to something else.  If the vendor can’t give you a straight answer to this question, then you may have problems.

Don’t get me wrong.  There are plenty of tools out there that fail one or more of these signs and that doesn’t mean that you should completely avoid them, but you shouldn’t expect to pay a premium for them either.  Hopefully the vendor is being honest with themselves and labeling it as “Beta” while they work to iron these things out.  If not, you should be honest with them about your willingness to accept a product that is not “enterprise ready”.  Perhaps you’re willing to accept a little bit of pain for a smaller price tag.  Maybe you want to be able to brag to your peers that you were the first to have that product hotness.  Whatever the reason, just make sure that you are aware of what you’re getting into up front.