Can I use AI to write legal documents or arguments in Australia?

Artificial intelligence tools — particularly large language models (LLMs) like ChatGPT, Claude, and Copilot — are increasingly being used by both lawyers and members of the public to draft legal correspondence, submissions, and arguments. While these tools can be genuinely useful for structuring ideas and identifying general areas of law, their use in legal practice carries a serious and underappreciated risk: AI hallucination.

What Is an AI Hallucination?

An AI hallucination occurs when a large language model generates factually incorrect information with apparent confidence. In a legal context, this typically means the AI invents case names, citation numbers, court names, and section references that sound completely credible but do not exist in any law reporter, database, or official statute.

This happens because LLMs are not search engines. They do not retrieve verified facts — they predict the statistically most likely sequence of words to follow a given prompt. When asked to cite an authority supporting a legal argument, the model generates what looks like a real case citation. But because it is predicting text rather than retrieving facts, it can produce a citation to a case that was never decided, in a court that may not exist, in a year that is incorrect.

LLMs are also specifically programmed to always produce an answer. Unlike a human researcher who would say “I couldn’t find a case on this point,” an LLM will generate something — even if that something is entirely fabricated.

Real-World Consequences: Courts Are Already Sanctioning AI Errors

This is not a theoretical risk. Courts in multiple jurisdictions have already sanctioned lawyers who filed submissions containing AI-generated fictitious authorities:

• United States: In Mata v Avianca, Inc, 22-cv-1461 (PKC) (SDNY, June 2023),¹ two New York attorneys were sanctioned after filing a brief that contained six entirely AI-fabricated case citations. The court found the conduct fell below the professional standard of inquiry.
• Australia: While Australian courts have not yet imposed equivalent sanctions, Australian courts are acutely aware of the risk. Judges have warned publicly that submissions will be scrutinised for AI-generated content.

The Professional Conduct Rules for Queensland Solicitors

A Queensland solicitor is subject to strict duties of candour to the court under the Legal Profession Act 2007 (Qld)² and the Australian Solicitors Conduct Rules 2012 (Qld).³

Key obligations include:

• Rule 19 (frankness in court): A solicitor must not knowingly make a submission to a court that is false, and must not cite a case or legislative provision that the solicitor has not verified is accurate.
• Paramount duty to the court: The duty to the court overrides the duty to the client. A solicitor who files a fictitious AI-generated authority — even inadvertently — risks disciplinary action by the Queensland Law Society or the Legal Services Commission.

A solicitor who files an AI hallucination without verification may face:

• Adverse costs orders against themselves personally
• Referral to the Legal Services Commission
• Conditions placed on their practising certificate
• In egregious cases, suspension or cancellation of their right to practise

Risks for Self-Represented Litigants

Members of the public who represent themselves in court — before QCAT, the Magistrates Court, or any other tribunal — are equally at risk. If you file a document or submission that cites a case that does not exist, the tribunal or court may:

• Disregard the submission entirely
• Raise the matter as a contempt issue in serious cases
• Award costs against you for the inconvenience caused to the other party

Where AI Can and Cannot Help

Task	AI Usefulness	Risk Level
General explanation of a legal concept	High — useful starting point	Low — easy to verify with general sources
Identifying the correct area of law	Moderate — usually points in the right direction	Low to medium
Drafting letters or plain-English summaries	Moderate — good for structure and tone	Low — no citations required
Citing specific cases	Very risky — frequently hallucinated	High — must independently verify every citation
Citing specific legislative sections	Risky — sections are often wrong or outdated	High — check the current Act directly
Checking current legal status	Not reliable — LLMs are trained at a point in time	High — law changes; AI won’t know

How to Use AI Responsibly in a Legal Context

1. Use it to identify issues, not authorities. AI can help you understand what area of law is relevant, but never rely on it for specific citations.
2. Verify every case independently. Check every case name and citation on AustLII , the Queensland Courts website, or a subscription legal database like LexisNexis or Westlaw.
3. Check legislation on the official source. Always verify legislative references directly on the Queensland legislation website or Federal Register of Legislation .
4. Have a qualified lawyer review the document. Before filing any AI-assisted document with a court, tribunal, or government body, have a Queensland solicitor review it.

Related Topics

• What is an AI Hallucination and Why Does It Happen?
• Technology Law Practice Area — Bell & Senior Lawyers
• Legal Matters Ep 17: AI Law, LLM Hallucinations & Medical Privacy

Professional Guidance

Andrew Bell of Bell & Senior Lawyers sits on the Queensland Law Society’s Privacy, Data, Technology and IP Law Committee and the Australian Computer Society’s AI Committee. If you need advice on AI governance, AI-assisted drafting, or technology law compliance, contact us.

📞 (07) 5532 8777 | 🌐 bellsenior.com.au | Contact us

---