A mid-sized accounting firm in Adelaide discovered, eleven months into a three-year managed services contract, that their provider did not actually employ the security analyst they had been told would be monitoring their systems. The person named in the contract had left the company six months earlier. The monitoring dashboard the firm reviewed monthly had been showing green lights generated by automated scripts, not human review. They found out because their systems were compromised in an attack that the monitoring should have caught.
The firm had done what most businesses do when selecting an MSP: they reviewed a proposal document, checked certifications, asked for references, and accepted the lowest competitive price. What they did not do was verify operational claims against operational reality.
Selecting a managed service provider is one of the more consequential technology decisions a business makes, and it is one where the information asymmetry between buyer and seller is acute. This guide is an attempt to close some of that gap.
Start With the Problem, Not the Market
The most common mistake in MSP selection is beginning with vendor evaluation rather than requirements definition. Businesses issue an RFP, receive proposals, and compare them — but if the requirements document is vague, the comparison is meaningless. A proposal promising "comprehensive security monitoring" is indistinguishable from one offering genuinely comprehensive security monitoring until you specify exactly what comprehensive means for your environment.
Before speaking to any providers, work through three questions with the internal stakeholders who understand your operations. First: what are the IT failure modes that would most damage the business, and how quickly would each need to be resolved? Second: what regulatory or compliance obligations attach to your data, and which of them have technology implications? Third: where is the business likely to be in three years, and what technology changes does that trajectory imply?
The answers to these questions become the basis of a requirements document that allows meaningful comparison across providers and, later, forms the substantive core of a service level agreement.
Evaluating Technical Depth
Certifications matter, but they are a floor, not a ceiling. Microsoft Gold Partner status, AWS Advanced Consulting Partner designation, and cybersecurity certifications like ISO 27001 or SOC 2 Type II indicate that an organisation has met a defined standard. They do not indicate how the organisation performs on a Tuesday afternoon when three clients have simultaneous incidents.
Clare Sanderson, a technology procurement specialist at Melbourne consulting firm Meridian Advisory who has overseen more than 40 MSP selection processes for mid-market Australian businesses, recommends what she calls the "depth probe" approach to technical evaluation. "You pick two or three specific scenarios that are relevant to your environment and you ask each provider to walk you through exactly what would happen," she says. "Not in general terms — specifically. Who gets the alert? What system does it come from? What's the escalation path if the first-level person can't resolve it? How long before you call us?"
The answers reveal whether the provider has documented, tested operational processes or is describing an idealised version of how they think things should work. The difference matters enormously when something goes wrong at 11 pm.
Security Capability: Verification Over Claims
Security is where provider claims diverge most dramatically from operational reality. Every MSP in Australia will describe itself as security-focused. The substantive questions are specific: do they operate a security operations centre, and if so, is it staffed 24 hours a day with human analysts, or does out-of-hours monitoring rely on automated alerting to on-call staff? What is the median time from alert generation to analyst review? What security information and event management platform do they use, and can they demonstrate it in a live environment?
Ask for a sample incident report from a recent security event — appropriately anonymised. A provider with mature security operations will have detailed, timestamped documentation of what happened, when, and what was done. A provider whose security capability is primarily marketing will struggle to produce one.
The Adelaide accounting firm mentioned at the start of this article would have avoided its situation by asking one simple question during the sales process: "Can we speak directly with the analyst who will be assigned to monitor our account?" The inability to answer that question would have been informative.
Contract Structure and Exit Terms
The service level agreement is the document that matters most once a relationship begins, and it is the document that receives the least scrutiny during the selection process. Key elements to examine carefully: response time commitments for different priority levels, and how priority is defined — is a server down always P1, or does it depend on the number of affected users? Uptime guarantees and how credits are calculated if they are not met. Data ownership provisions — if you end the relationship, in what format and within what timeframe is your data returned?
Exit terms deserve particular attention. Contracts that impose long notice periods, data extraction fees, or complex off-boarding requirements create lock-in that limits your leverage if service quality deteriorates. A provider confident in their service quality will not need punitive exit terms to retain clients.
The Reference Check That Actually Works
Standard reference checks — will you give us two references we can call? — are close to useless. Providers give references who will speak well of them. The more useful approach is to ask for references from clients who have been with the provider for more than three years (relationships that have lasted through problems, not just honeymoon periods), clients in your industry or of similar size, and — if you can identify them through your own network — former clients who chose not to renew.
Sanderson also recommends asking references a specific operational question rather than a general satisfaction question. "Ask them: describe the last significant incident you had, and walk me through exactly how the provider responded," she says. "That answer tells you more than any satisfaction rating."
The Adelaide firm's compromise was eventually attributed to a configuration gap that a competent security review would have caught. Their new provider, selected after a more rigorous process, identified seven similar gaps in the first assessment. The relationship has now run for two years without a security incident. The difference was not price — the new contract costs 12 percent more than the previous one. The difference was asking harder questions before signing.