Managed Detection and Response Provider Evaluation for Australian Enterprises

Managed Detection and Response (MDR) procurement is an operating model decision, not a feature comparison. The selected provider may monitor sensitive telemetry, investigate incidents and take action inside the organisation’s environment. A credible evaluation must establish what the service will detect, who can respond, how evidence will be produced and what the organisation must still operate itself.

CYBORIUM helps Australian organisations compare MDR providers through a structured, vendor-neutral sourcing process. The selected provider pays CYBORIUM a capped fee. The client contracts directly with that provider, and CYBORIUM does not sell, deliver, operate or invoice the technology.

What an MDR service should provide

An MDR service combines security telemetry, detection engineering, human analysis and an agreed response process. The service may use the organisation’s existing endpoint, identity, cloud and network controls, or include provider-supplied technology. The name alone does not establish the coverage.

Before approaching the market, document the assets and telemetry that matter. This usually includes endpoints, identity systems, cloud workloads, email, network controls and critical applications. Providers should then map their proposed coverage to that environment and identify every dependency or blind spot.

The five decisions that shape an MDR procurement

1. Detection coverage

Ask each provider to map detections to the telemetry sources available in the organisation. The response should distinguish native coverage, coverage that requires an integration, and coverage that is out of scope. A product demonstration is not evidence that the managed service will detect the organisation’s priority attack paths.

2. Response authority

Define what the provider may do without approval, what requires a named client decision owner, and what is prohibited. Test the model against incidents that occur outside business hours. An attractive response-time promise has limited value if every containment action waits for an unavailable approver.

3. Service evidence

Require sample incident reports, detection engineering records, threat-hunting outputs, service review packs and escalation logs. Ask how the provider measures detection quality and false positives. Mean time metrics should state when the clock starts, when it stops and which events are excluded.

4. Operating fit

Clarify the responsibilities that remain with the internal team. These may include maintaining log sources, approving response actions, remediating affected systems and managing communications. The evaluation should also examine analyst location, data handling, subcontractors, service continuity and exit support.

5. Commercial exposure

Compare the complete cost model, including telemetry volume, data retention, endpoint growth, onboarding, integrations, after-hours work and incident response outside the standard service. Test price movement at realistic growth levels. A low entry price can become expensive when data ingestion or retention changes.

Reporting deadlines your MDR contract has to fit

Several Australian obligations run on a clock that starts when your organisation becomes aware of an incident. An MDR provider’s analysts are often the first to see one, so the contract needs to say how quickly you are told.

• APRA CPS 234. An APRA-regulated entity must notify APRA as soon as possible, and no later than 72 hours, after becoming aware of a material information security incident (paragraph 35). A material control weakness the entity cannot fix in good time must be notified within 10 business days (paragraph 36).
• Security of Critical Infrastructure Act. A responsible entity reports a cyber incident with a significant impact on a critical infrastructure asset to ASD’s ACSC within 12 hours. An incident with a relevant impact is reported within 72 hours.

Ask each provider three questions. How long can pass between an analyst confirming an incident and a named person at your organisation being told? Will the provider give you a timeline with timestamps that you can attach to a regulator notification? Who decides that an incident is reportable? The answer to the last one should be you, with the provider supplying the evidence.

Whether either obligation applies to your organisation depends on your sector and your assets. Check with your legal adviser.

A practical MDR evaluation scorecard

A useful scorecard separates mandatory evidence from scored preferences. CYBORIUM commonly structures the decision around the following areas:

• Coverage: required assets, telemetry, use cases and exclusions.
• Detection quality: engineering method, testing, tuning and measurable outcomes.
• Response: authority, escalation, containment actions and decision paths.
• People: analyst capability, local coverage, staffing model and specialist access.
• Assurance: data location, security controls, audit evidence and subcontractor governance.
• Operations: onboarding, integrations, service reviews, reporting and exit support.
• Commercials: full three-year cost, growth assumptions, exclusions and contractual remedies.

Weight the scorecard before proposals are opened. This prevents a polished demonstration or a familiar brand from changing the criteria after the evidence is known.

A worked example with numbers

The figures below are illustrative. They are not drawn from a client engagement. Each area is scored from 0 to 5, multiplied by its weight, and the total is divided by 5 to give a score out of 100.

Provider B scores higher overall, 74 to 71. But B scored 2 on response authority, which means most containment actions would wait for a client approver. If the organisation had set a pass mark of 3 on response authority before opening proposals, B would not reach the final comparison and A would be selected. That is the practical reason to keep mandatory evidence apart from scored preferences. A weighted total can hide a gap you cannot accept.

How to test shortlisted providers

Use the same scenario pack for every shortlisted provider. Include a compromised identity, suspicious endpoint behaviour, cloud control-plane activity and an incident requiring urgent containment. Ask the provider to show how the signal enters the service, how an analyst investigates it, what the client sees and what action follows.

Reference checks should use organisations with a similar environment and service model. Ask about onboarding delays, missed expectations, analyst continuity and the accuracy of monthly reporting. References are more useful when they test the proposed delivery team rather than the provider’s general reputation.

Questions about the first hour of an incident

A scenario test shows how a provider works. These questions tie it to a clock. Put each one to every shortlisted provider and record the answers in the scorecard.

1. At what point does an alert reach a human analyst, and how long passes between the detection firing and that first review?
2. Who at your organisation is called first, by what method, and what happens if they do not answer?
3. Which containment actions can the provider take straight away, such as isolating one endpoint or disabling one account, and which need approval?
4. Where is each step recorded, and can you export that record with timestamps?
5. What does the provider do when your own team changes something during the incident?

A provider that answers in generalities has told you something too.

CYBORIUM’s role in the decision

CYBORIUM defines the requirements, builds the evaluation scorecard, coordinates market engagement, tests evidence, compares commercial structures and supports negotiation. The organisation retains the decision and contracts directly with the selected provider.

The process is designed to produce a clear decision record for security, procurement and executive stakeholders. It also makes the service boundary explicit, which reduces the risk of discovering operational gaps after contract signature.

Questions decision owners ask about MDR

Is MDR the same as a managed SIEM?

No, though they overlap. A managed SIEM collects and monitors logs and raises alerts. MDR adds analysts who investigate and either act or recommend action. Many providers sell both, so check what the contract says the provider will do after an alert. The SIEM vendor evaluation guide covers the platform side.

Can an MDR provider contain an incident without asking us first?

Only if you have agreed that in advance. One workable approach is to pre-approve a short list of low-risk actions, such as isolating a single workstation or disabling a compromised account, and require a named approver for anything that affects servers or business-critical systems. The list belongs in the contract, not in a verbal assurance.

Does MDR replace our internal security team?

No. Your team still maintains log sources, approves response actions, fixes affected systems and handles communications. MDR adds monitoring and investigation capacity. It does not take those responsibilities over.

What should we get back if we leave the provider?

Agree this before signing. Cover your data and logs, the detection rules written for your environment where the contract allows it, runbooks and incident records, and a handover period. If the provider’s detection content is proprietary, find that out during evaluation, not at exit.

Primary references

• NIST Cybersecurity Framework 2.0
• NIST Cybersecurity Framework: Detect
• APRA CPS 234 Information Security
• CISC guidance: Notification of Cyber Security Incidents

Reviewed by Michael Kazantzis for CYBORIUM on 23 October 2026. This guide explains CYBORIUM’s procurement evaluation method. It does not provide legal or regulatory advice.

Also relevant: email as the first ingress path.

Last updated: 23 October 2026