Note: This article provides tips and tricks on contractual mechanisms that can mitigate the risks involved in procuring artificial intelligence (AI). It is not intended to advise on your personal use of AI products and services. You should read it in conjunction with the terms applicable to the specific product that you intend to use.
We have seen a significant increase in procurements for the Commonwealth that involve a form of AI. Most third party services include a form of AI, whether they are public facing generative tools or bespoke enterprise solutions configured to handle Commonwealth data. As artificial intelligence becomes a core feature of new and updated Information and Communication Technology (ICT) technologies, traditional ICT contracts are increasingly being tested by issues unique to AI. While AI can enhance efficiency and functionality, it also introduces new risks and uncertainties that standard contract terms may not adequately address.
Agencies are now subject to, or able to utilise as guidance, a range of AI-related government policies. These policies require vigilance, AI impact checking, accountability, transparency, and ongoing monitoring of AI use. However, these obligations do not extend directly to AI vendors, as government policies are not binding on entities outside government. The vendor community remains highly risk averse in respect of contractual representations about AI. Vendors may provide information about the efficiency, cost-effectiveness, and quality of their AI functionality, but are generally unwilling to accept these commitments contractually.
Traditional ICT contracts are not fit for purpose in dealing with emerging AI issues. In this article, we consider some key legal and contractual considerations for AI and offer some practical tips and tricks to think about when designing approaches to market for services or systems which may utilise AI. In particular, we tackle the question of what contractual protections and technical requirements should be included in your procurements of services and AI solutions to ensure accountability and risk mitigation over the entire AI lifecycle, from design to decommission.
Here is what you can do to achieve a reasonable degree of contractual protection for the procurement of AI tools.
Compliance with Australian Government policies, guidelines and standards
The Commonwealth government has recently released its AI Plan. This document includes a comprehensive guide to government intentions for the use of AI, and of its planned approach. The AI Plan includes an intent to make suitable contract clauses available. These will supplement AI contract clauses already released by the Digital Transformation Agency (DTA).
The AI Plan supplements a number of other policies and guidance documents released to assist agencies in dealing with AI.
The AI Plan is a guidance document, and clauses suggested for use in contracts will not be mandated by legislation. So agencies will retain responsibility for complying with applicable AI policies and for negotiating suitable contract protections.
In our experience, negotiating the inclusion of policies that increase the contractual obligations on the vendor without an express legislative requirement for their inclusion can be challenging. However, it is reasonable to require a vendor under the contract to take steps to assist you, as a Commonwealth entity, to comply with policies, standards and guidelines, especially where your compliance with that policy or guideline will be contingent on the aspects of the AI system over which the vendor has control. Your procurement documents can be designed to include, at a minimum:
Useful contract obligations
Agencies need to work out what AI provisions are appropriate for their contracts, and what AI protections are achievable in the current AI environment (and this principle will need to be regularly checked and reconsidered as the AI environment develops).
Here are some suggestions:
1) Require the use of AI to be approved in the contract (and impose conditions on that use if needed). In our experience, vendors are willing to explain how their products incorporate AI functionality and how that functionality might be used to respond to a customer requirement. The Commonwealth has recently issued a direction that applies to all non-corporate Commonwealth entities and you should also think about any additional specific banned AI use in your own agency, as this may vary.
2) Check and include controls if needed on links to third party products or websites.
3) Build in obligations for human oversight at appropriate points in an AI process. Generally vendors will not take responsibility for the quality of the output of AI functionality, so standard ICT approaches to functional specifications do not work. Include explicit obligations for human review, whether that be at various workflow process points, and/or sample checking that the output is accurate, or where it is a recommendation, is equivalent to one produced by an appropriately qualified human.
4) Include prohibitions on the use of data for learning. Vendors can control how customer data is used and are accepting this sort of restriction (for example, we have seen an approval requirement for this included in vendor AI policy documents).
5) Clarify intellectual property ownership of AI generated outputs, including in respect of First Nations data and artwork. Intellectual Property (IP) provisions often use the concept of contract material, to cover vendor produced deliverables. But AI outputs are customer generated, and may be accessible by the vendor.
6) Strengthen governance and customer control for ongoing review of the use of AI (e.g. to require additional human checking points). Vendors understand the need for reasonable governance relating to the use of AI (for example they are already regularly required to assist customers to comply with applicable government policies).
7) Align the security provisions to manage the way data is inputted to an AI system, and to control any vendor access to that data and output data. Ensure that any vendor rights are not applied by default, but require active customer approval.
Additional AI controls
To manage the specific risks that arise from embedded or vendor-operated AI, agencies can consider incorporating these additional, tailored obligations into their draft contract and requirements. For example, you may wish to include:
Questions to ask in your approach to market documentation
Using your approach to market documentation to find out about how vendors intend to use AI in the fulfilment of contractual obligations is an effective way to mitigate against associated risks. You cannot include contractual protections against a risk you are not aware of – therefore, approach to market documentation should include specific questions asking how AI will be used in the delivery of the services or the design of the solution you are seeking to procure.
In our experience, information about how AI components are used in the provision of services is not always clear in product specifications, terms and conditions or marketing materials. You should ask the vendor specific questions and then draft contractual obligations in your contract based on the vendor’s answers.
For example, you could require vendors to respond to the following questions as part of its response to any approach to market:
By proactively asking these questions, you equip your agency to identify potential AI-related risks early and incorporate tailored contractual safeguards that ensure transparency, compliance and accountability throughout the procurement process.
