We're trying to come to an agreement with a SaaS vendor about performance and availability service level agreements (SLAs). I discussed this topic some in my previous "SaaS Headaches" post. I thought it would be instructive to show people the standard kind of "defense in depth" that suppliers can have to protect against being held responsible for what they host for you.
We've been working on a deal with one specific supplier. As part of it, they'll be hosting some images for our site. There's a business team primarily responsible for evaluating their functionality etc., we're just in the mix as the faithful watchdogs of performance and availability for our site.
Round 1 - "What are these SLAs you speak of?" The vendor offers no SLA. "Unacceptable," we tell the project team. They fret about having to worry about that along with the 100 other details of coming to an agreement with the supplier, but duly go back and squeeze them. It takes a couple squeezes because the supplier likes to forget about this topic - send a list of five questions with one of them being "SLA," you get four answers back, ignoring the SLA question.
Round 2 - "Oh, you said 'SLA'! Oh, sure, we have one of those." We read the SLA and it only commits to their main host being pingable. Our service could be completely down, and it doesn't speak to that. Back to our project team, who now between the business users, procurement agent, and legal guy need more urging to lean on the supplier. The supplier plays dumb for a while, and then...
There's a lot of promise in the new SaaS (software as a service; what used to be called ASPs, or Application Service providers, till Microsoft crapped all over that acronym) and newer PaaS (platform as a service) spaces (and look for a steady stream of new "aaS"es to come). However, there are a lot of gotchas in signing on with a SaaS vendor. You'd like to be able to believe that they have decent performance, uptime, security, etc., especially after the tell you "Oh, all kinds of big companies use us; Dell, IBM..." This is exacerbated by SaaS often being an "end run" around IT in the enterprise, so naive users can get sold a bill of goods without proper technical oversight. SaaS is a big buzzword now, and there are a lot of startups springing up that do not necessarily have experience running large scale sites. Think about how many MMORPG games still get scuttled due to poor operational performance. SaaS is the same.
Here's some things to keep in mind when selecting a SaaS vendor, laced with real life horror stories from our experiences.
1. Performance/Availability. Set a hard performance/availability SLA in the contract. Many vendors won't even have an SLA clause, or they'll have one that says "99.9% uptime!" without any remedy clause for what if they don't hit that. You want a clear SLA with a clear measurement method and clear "money back" if they don't hit it. We use a 2 second global performance SLA as measured by a Keynote Global 35 monitor. But the SLA isn't the whole story - you are counting on these people to accomplish your goals.