AI contract clauses guide. Last reviewed:
AI contract clauses decide who controls the model, who owns what it learns from your data, and who is responsible when it gets something wrong. Australia has no dedicated AI Act, so these terms live in the supplier agreement or nowhere. The most detailed public starting point is the Australian Government’s own AI contract template, written for Commonwealth agencies. This guide turns it into a plain-English list of eight clauses any Australian organisation can ask for, with the question to put to the supplier and a priority for each. It is a checklist for the conversation, not a contract and not legal advice.
Where do AI contract clauses come from in Australia?
The Digital Transformation Agency (DTA) publishes AI contract material for government agencies: model AI clauses (version 2.0 was published in March 2025) and an AI contract template for AI delivered as a cloud service. The template’s Section 5 is the AI part, and it covers two cases: an off-the-shelf AI system that the customer configures and monitors, and an off-the-shelf or bespoke system where the seller also does the integration, configuration and monitoring.
These documents are written for Commonwealth entities and they say so. They are still the best free statement of what a careful customer asks an AI supplier to agree, and private organisations borrow from them for the same reason they borrow from the Commonwealth Procurement Rules. They sit alongside, not instead of, your own legal advice, the Privacy Act and any regulator that applies to you. For the policy side of the same problem, see our guide to AI procurement guardrails in Australia.
What are the eight AI contract clauses to ask for?
The table groups the template’s AI provisions into eight asks. The priority column uses the MoSCoW method: Must before you sign, Should unless there is a reason not to, Could where risk is higher or the use is more consequential. Adjust it to the risk of your use case, because an AI note-taker and an AI that triages customer complaints do not need the same terms.
| Clause | What it does | Question to ask the supplier | Priority |
|---|---|---|---|
| 1. Which AI, which model | Supplier tells you what AI it uses, the underlying model and where data sits at rest and in processing. Model changes need your approval. | Which model runs my work, where does my data go, and what happens when you change the model? | Must |
| 2. No mining of your data | Supplier may not use your data for its own purposes. The template says a click-through term does not override this and automated collection must be switched off. | Do you use my data, prompts or outputs to train or improve any model, and can that be switched off in writing? | Must |
| 3. Incident notification | Supplier tells you when it knows or suspects an AI incident or an AI hazard, and follows your reasonable directions. | What counts as an AI incident, and how fast do I hear about one? | Must |
| 4. Circuit breaker | The system can be interrupted and stopped immediately on your instruction, through a tool you can reach. | Can I stop this system myself, right now, without raising a ticket? | Must for consequential use, otherwise Should |
| 5. Human oversight and explainability | People can oversee the system and decide not to use, override or reverse an output. The supplier documents capabilities, limits, training and testing data, planned changes and expected life. | Who can override an output, and what documentation do I get? | Should, Must for decisions about people |
| 6. Training data, bias and testing | Supplier identifies and mitigates bias in training data, does not infringe third-party rights, and tests against acceptance criteria you approve. | How did you test for bias, and what does passing look like before I accept it? | Should |
| 7. Handover, destruction and portability | At the end, datasets and your data are handed back and copies destroyed as you direct. | What do I get back when this ends, in what format, and when are copies destroyed? | Must |
| 8. AI management and evidence | Supplier runs an AI management system, for example one aligned to ISO/IEC 42001:2023, and gives evidence within an agreed number of business days. | Can you show me your AI management system and risk records on request? | Could, Should for high-risk use |
What does each clause look like in practice?
1. Which AI and which model
Without this clause you may not know which model produces your results, or when it changes. A model update can alter accuracy, behaviour and risk overnight. Ask for the model name or family, where data is processed and stored, and a right to approve a change in underlying model before it reaches your users. Where a supplier will not commit to approval, ask for advance notice and the right to test before cut-over.
2. No mining of your data
This is the clause that matters most for confidential and personal information. The government template goes further than most standard terms by saying the ban holds even if a user clicks through supplier terms that allow it, and by requiring automated collection to be disabled. A plain starting point for your own wording: the supplier must not use our data, prompts or outputs to train, tune or improve any model, and no click-through or user-level term overrides this. Treat the line as a starter for your lawyer, not final drafting.
3. Incident notification
Standard security clauses cover data breaches. AI adds failure modes that are not breaches: a model that starts producing harmful or biased output, or one that behaves unexpectedly after an update. The template covers an AI incident that has occurred or an AI hazard the supplier suspects, plus a duty to follow your reasonable directions. Ask the supplier to define both terms and give you a notification time you can live with.
4. Circuit breaker
If an AI system acts on its own, or its outputs reach customers or staff without review, you need to be able to stop it. The template requires a circuit breaker that stops the system immediately on the customer’s instruction, through a tool the customer can access. If the only way to stop the system is to email the supplier and wait, you do not have one.
5. Human oversight and explainability
The template requires that humans can oversee the system, that the people responsible understand its capabilities and limits, and that they can decide not to use it or to reverse an output. It also asks the supplier to document how the system works well enough for you to assess it: capabilities, limits, the data used to train and test it, planned changes and expected life. This links directly to your own obligations on automated decisions. See our guide to automated decision-making transparency in vendor contracts.
6 to 8. Testing, exit and evidence
Make the supplier prove the system meets your criteria before you accept it, and agree up front what you get back at the end. Our guide to pricing the exit before you sign applies in full: datasets, logs, configuration and your data should come back in a usable format, and the supplier should destroy copies on your direction. For evidence, a supplier that runs a recognised AI management system, such as one aligned to ISO/IEC 42001:2023, can show you how it manages risk rather than asking you to take it on trust.
What should you watch for in a supplier’s standard terms?
Supplier terms are written for the supplier. When you read them, look for these:
- A right to use your data, prompts or outputs to “improve the service” or train models.
- A right to change the underlying model, features or data location without notice.
- Incident notification that covers security breaches but not AI failures.
- No way for you to stop the system yourself.
- Liability limits or disclaimers that exclude AI outputs altogether.
- Deletion “on request” with no deadline, no confirmation and no coverage of backups or derived datasets.
- Documentation described as “available on request” with nothing specified.
How do privacy law and regulators fit in?
The Privacy Act and the Australian Privacy Principles apply to personal information whatever the technology. Organisations that use AI to make or support decisions that significantly affect individuals face new transparency duties from December 2026, covered in our guide to APP 1.7 and vendor contracts. If you are APRA-regulated, an AI supplier may be a material service provider under CPS 230, which brings audit access, subcontracting notice and exit terms into scope. See our CPS 230 due diligence guide. Your privacy, security and legal teams should review the final terms before signature.
How do you negotiate AI contract clauses?
- Decide the risk of the use case first. A drafting aid and a system that screens applicants are different risks with different terms.
- Rank the clauses with MoSCoW. Settle the Musts before price. Our guide to MoSCoW requirements shows the method.
- Negotiate while suppliers still compete. Terms such as no data mining and exit are far easier to win before one supplier is selected.
- Ask for the evidence, not the promise. Documentation, test results and management system records matter more than a statement of intent.
- Pilot before you commit. Use a testing clause with acceptance criteria you approve in writing.
Ten questions to ask before signing an AI contract
- Which AI model runs my work, and who else supplies parts of the system?
- Where is my data processed and stored?
- Do you use my data, prompts or outputs to train or improve any model?
- What happens when you change the underlying model?
- What counts as an AI incident, and how fast do I hear of one?
- Can I stop the system myself, immediately?
- Who can override an output, and how is that recorded?
- What did you test for bias and accuracy, and what evidence can I see?
- What do I get back when the contract ends, and when are copies destroyed?
- Who is liable if the system causes harm, and is that limit acceptable for this use?
What is CYBORIUM’s role?
CYBORIUM helps Australian organisations compare AI providers against their requirements and prepare the commercial and contract terms before signature. The selected provider pays CYBORIUM a capped fee at an agreed percentage, providers that are shortlisted but not selected pay nothing, and you contract directly with the provider. The decision is always yours. CYBORIUM is not a law firm and does not give legal advice. See our AI vendor evaluation page for how an evaluation runs.
AI contract clauses in short
Ask for eight things: disclosure of the model and where data goes, a ban on mining your data, AI incident notification, a circuit breaker you control, human oversight with real documentation, bias and acceptance testing, handover and destruction at exit, and evidence of AI management. Rank them by the risk of the use, settle the Musts while suppliers still compete, and have your lawyer finalise the wording.
Reviewing an AI supplier agreement? Get the Procurement Readiness Brief or tell us about your project. The client pays CYBORIUM nothing and chooses the provider.
Frequently asked questions
What AI contract clauses should an Australian organisation ask for?
Disclosure of the AI model and where data is processed, a ban on using your data to train models, notification of AI incidents, a circuit breaker you control, human oversight and documentation, bias and acceptance testing, handover and destruction of data at exit, and evidence of an AI management system.
Is the Government’s AI contract template only for government?
It is written for Commonwealth agencies. Private organisations can use it as a checklist, adapted to their own risk, law and regulator, with legal review of the final wording.
Should an AI supplier be allowed to train on my data?
For confidential or personal information, the safer position is no. The government template bans data mining of customer material, even where a user clicks through supplier terms that allow it.
What is a circuit breaker clause?
It requires that the AI system can be interrupted and stopped immediately on your instruction, through a tool you can access, without waiting for the supplier to act.
Do I need ISO/IEC 42001 from an AI supplier?
Not always. It is a useful signal that a supplier manages AI risk systematically. Ask for it, or equivalent evidence, where the use is higher risk.
If the system acts on its own, see AI agent procurement for the agent-specific tests and terms. For the commercial side, see enterprise AI pricing.
Related guidance: AI procurement guardrails in Australia, enterprise AI vendor selection criteria, procurement as a service contract terms.
Sources
- Digital Transformation Agency, AI contract template (cloud version), version 1: Section 5 AI clauses, data mining prohibition and optional ISO/IEC 42001:2023 clauses.
- HLC, Australia’s model clauses provide framework for AI procurement: summary of the DTA AI model clauses, version 2.0, published March 2025.
- ISO/IEC 42001:2023, Artificial intelligence management system: the standard the template references for AI management.
- APRA, Prudential Standard CPS 230 Operational Risk Management: service provider requirements for regulated entities.
