Legal Matters Ep 17: Technology Law Part 2 — Artificial Intelligence, Legal Hallucinations & Your Privacy

Andrew Bell returns to 4CRB for the second week of the technology law series, joining Colin Balewski for a wide-ranging discussion on artificial intelligence — what it actually is under the bonnet, why it fabricates plausible but fictitious legal authorities, and what privacy rights apply when a medical practitioner uses AI software to record a consultation. The programme also responds to a wave of listener feedback following last week’s instalment and fields live caller questions on retaining wall responsibility and gifts made during a person’s lifetime.

Key Topics

• Last Week’s Recap: A brief summary of the DMCA safe harbour framework, the CompuServe/Prodigy paradox, and Australia’s narrower safe harbour provisions for carriage service providers.
• Australia–US Free Trade Agreement in the News: The current tariff dispute and its relevance to the AUSFTA’s IP provisions.
• What Is Artificial Intelligence? The government’s difficulty defining AI, the distinction between traditional deterministic programming and statistical machine learning, and Andrew’s background developing credit-scoring and fraud-detection algorithms.
• How Large Language Models (LLMs) Work: LLMs as sophisticated, non-deterministic autocomplete systems — predicting statistically likely word sequences rather than independently verifying facts.
• AI Hallucinations in Legal Practice: Why AI tools confidently invent case names, citation numbers, and legislative sections that do not exist — and the serious professional risks of filing those references in court.
• Caller Robert — Retaining Wall Responsibility: Who bears responsibility for a retaining wall on or near the boundary between two properties under Queensland law?
• AI as “Sophisticated Autocomplete”: The distinction between a Google search (relevance-ranked) and an LLM (statistically generated) — and why asking ChatGPT for an ABN can produce a confidently wrong answer.
• Caller John — Inter Vivos Gifts: Can family members challenge gifts of money made during a person’s lifetime after that person dies?
• Caller Anna — AI in Medical Consultations: A patient was asked whether her specialist could use AI note-taking software in their consultation. What happens to that health data? Is it shared? Can it be used for AI training?
• Closing Announcements: Andrew’s personal tribute to his sister Catherine Bell (1987–2026), who passed away of bowel cancer, with a call for early screening; and congratulations to Ross Lee, awarded the Medal of the Order of Australia in the King’s Birthday Honours for founding My Community Legal.

Listener FAQ Highlighted in This Episode

• Can I use AI to write legal documents or arguments in Australia?
• Who owns the copyright in AI-generated content in Australia?
• What privacy rights apply when AI is used in my medical consultation?
• Who is responsible for a retaining wall in Queensland?
• Can a lifetime gift be challenged as part of a deceased estate?

Listen to the full discussion above.

Recap: Last Week’s Technology Law Discussion

📢 Topic: Summary of Episode 16 — DMCA safe harbour provisions, the CompuServe/Prodigy cases, and Australia’s narrower safe harbour framework.

[Colin] We had a fantastic response to last week’s programme — via email, phone, and text. Some listeners even said it was so interesting that they missed their turn-off! Where would you like to pick things up, Andrew?

[Andrew] For those who weren’t listening last week, technology law is a favourite of mine given my long background designing large, complicated computer systems and my PhD in cybersecurity law. I serve on a number of national-level committees around AI and technology law. What we covered last week were the DMCA safe harbour provisions — the Digital Millennium Copyright Act — and why platforms like Google and YouTube are able to host content that many people find objectionable without being held legally responsible for it. It really comes down to two competing philosophies that emerged in the United States in the 1990s: either platforms needed to curate everything, or large publishers would be treated like a library — not responsible for what is on their shelves. The law came down firmly on the library side, and that framework was then incorporated into Australian law through the Free Trade Agreement that John Howard’s government signed with the United States.

[Colin] The Free Trade Agreement is in the news again right now — America is accusing Australia of allowing slave-traded products and wants to tariff our businesses.

[Andrew] That will be an interesting one. I think that particular dispute will get unwound like most of the other tariffs have been. But it does illustrate how those foundational agreements shape what law applies here decades later.¹

📎 See also: Episode 16: Technology Law Part 1 | Safe Harbour Provisions in Australia

What Is Artificial Intelligence?

📢 Topic: Cutting through the hype — what AI actually is, what LLMs do under the bonnet, and why “thinking machines” is a misleading description.

[Andrew] So today we were going to talk about artificial intelligence law. This is a rapidly developing area. But to start with, what is actually artificial intelligence?

Artificial intelligence is a suite of computer programmes, and everyone has a different definition of what it is. Even the Australian government struggles to define it.² A lot of what we see being called artificial intelligence at the moment I would not say properly meets the definition. But for what we are talking about today, that may not matter.

I have a long background in what is called machine learning and data science across large datasets. That is using statistics to determine what comes next — a field that has been used heavily across industry for the last 15 to 20 years. I have done things like write the machine learning algorithms used to assess credit applications for Coles Myer Mastercard, GE Finance, and 3 Mobile — determining whether applicants were creditworthy and whether they posed a fraud risk. Nowadays we would probably class that as artificial intelligence.

But the most recent AI that everyone is focused on is ChatGPT, Claude, Copilot, and similar tools.

How Large Language Models Work

[Colin] When you ask ChatGPT a question, it searches the entire internet for a consensus answer — is that a fair way of looking at it?

[Andrew] Mostly no, and this is a critical misunderstanding. LLMs are trained at a particular point in time. They are not looking at live data when you ask them a question. They are better understood as a very sophisticated autocomplete.

The way they are built is very different from a traditional computer program. A traditional program is deterministic: you put in some input, it does mathematics, and you get a predictable output. A Large Language Model is non-deterministic, both in the way it is trained and the way it produces output.

Here is a simple example: an LLM gets trained on enormous volumes of text, and it builds a probability model of any combination of words — what are the statistically likely next words in a sequence. If you give it the words “Ron” and “Hermione,” “Harry Potter” is statistically likely to follow. It does not know Harry Potter is a book series. It has simply learned that those words are associated.

[Colin] So it is giving you a statistically likely answer, not an independently verified one?

[Andrew] Exactly. And there is another serious problem: these models are programmed to always give an answer. They are not rewarded for saying “I don’t know.” So they will always try to answer your question, which can be genuinely dangerous — because what they produce is a statistically plausible answer, not a confirmed factual one. Some of the large models have been trained on every book ever written that could be scanned. They work very well in many contexts, but they do not actually know what they are talking about.

[Colin] I tested this myself yesterday. I asked ChatGPT to find the ABN for a company I deal with. It came back with a confident answer — business name, suburb, address, everything looked right. But when I checked the first two digits, something felt wrong. I went to the ABR directly and the ABN was completely incorrect.

[Andrew] And that is the core problem. If the model had been trained on current, complete ABN data, it may have given a good result. But if it was drawing fragments from different websites, the training signal for ABN data would be weak and what it “remembers” may be entirely wrong. It does not know it is wrong. It just gives you the most statistically likely sequence of characters.

📎 See also: AI-Generated Content Copyright in Australia | Technology Law Practice Area — Bell & Senior

AI Hallucinations and the Legal Profession

📢 Topic: Why AI tools fabricate plausible-sounding but non-existent legal authorities — and the professional consequences of filing them. Relevant law: Copyright Act 1968 (Cth) s 32 — human authorship requirement; Legal Profession Act 2007 (Qld) — professional conduct obligations

[Andrew] This brings me to one of the most serious problems in law right now: AI hallucinations in legal practice. A lot of people — both lawyers and non-lawyers — are using ChatGPT and similar tools to write legal arguments. And when you write a legal argument, you say things like “this position is supported by the case of Derry v Peek” or “see section 2 of the Defamation Act.” The problem is that the large language models in common use are predominantly trained on overseas legal materials. They are not reliably trained on Australian law.

What happens is that the model will generate something that looks exactly like a correct Australian case citation — it will have a case name, a year, a court, a citation number. It will look completely credible. But when you go to find it in AustLII or a law reporter, it does not exist. The model invented it because it was the statistically most likely sequence of words to follow a legal argument about that topic.

This is not a minor error. Courts in Australia and overseas have already sanctioned lawyers for filing written submissions containing non-existent authorities.³ In one well-publicised US case, a firm filed a brief citing six entirely fictitious decisions. In Australia, Telstra Corporation Ltd v Phone Directories Company Pty Ltd [2010] FCA 44 has already been cited in discussions about AI and originality, and the courts are acutely aware of the risk.⁴

What the Law Requires

Under the Legal Profession Act 2007 (Qld) , a Queensland solicitor is under a duty of candour to the court — they must not make submissions they know to be false, and they must not cite authorities they have not verified.⁵ Filing a hallucinated case citation — even inadvertently — may constitute a breach of that duty and could result in costs orders, disciplinary referral to the Legal Services Commission, or conditions on a practising certificate.

For self-represented litigants, filing a fabricated authority may be treated as a contempt of court in egregious cases, or may simply result in the argument being dismissed entirely.

The bottom line: AI tools can be a useful starting point for legal research. They can identify the right general area of law and suggest relevant concepts. But every single case name, citation, and section number they produce must be independently verified by a qualified Australian lawyer before it is used in any document filed with a court, tribunal, or government body.

📎 Relevant legislation: Legal Profession Act 2007 (Qld) — solicitor’s duty of candour; Legal Profession Uniform Conduct (Barristers) Rules 2015 See also: AI and Legal Documents FAQ | Technology Law Practice Area

AI and Copyright — Who Owns AI-Generated Content?

📢 Topic: Does copyright protect images, articles, or code produced entirely by an AI system in Australia? Relevant law: Copyright Act 1968 (Cth) s 32 ; IceTV Pty Ltd v Nine Network Australia Pty Ltd [2009] HCA 14

Although this specific question was not raised by a caller this week, it is directly connected to the broader AI discussion Andrew opened, and is one of the most common questions received by the programme following Episode 16.

Under s 32 of the Copyright Act 1968 (Cth) , copyright subsists in original literary, dramatic, musical, and artistic works — but only if there is a human author.⁶ The High Court in IceTV Pty Ltd v Nine Network Australia Pty Ltd [2009] HCA 14⁷ confirmed that Australian copyright requires the exercise of independent intellectual effort by a human being. Purely mechanical or algorithmic outputs — where no meaningful human creative judgment was exercised — are not “original” in the copyright sense.

Applying that principle to generative AI: a raw image produced by Midjourney in response to a one-word prompt, or a document generated by ChatGPT with minimal human direction, very likely does not attract copyright protection in Australia. It falls into the public domain and anyone may copy it freely.

However, the position becomes more nuanced where a human meaningfully contributes through prompt crafting, selection, editing, arrangement, or refinement of the AI output. In those cases, copyright may subsist in the human’s creative choices, even if the underlying output was AI-assisted. This is an area of law that is developing rapidly and formal legislative reform is expected.⁸

Practical implications:

• If your business creates marketing materials using AI image generators, you may not own copyright in those images and competitors may be free to copy them.
• If you are copying AI-generated content from a competitor, you are unlikely to be infringing their copyright — but check whether any accompanying human-created text, layout, or branding is independently protected.
• If you are selling or licencing AI-generated content, be transparent with clients about the copyright status: they may not receive the exclusive rights they expect.

📎 Relevant legislation: Copyright Act 1968 (Cth) s 32 — original works; s 35 (ownership of copyright); s 10 (definitions) Case law: IceTV Pty Ltd v Nine Network Australia Pty Ltd [2009] HCA 14 https://www.hcourt.gov.au/cases-and-judgments/judgments/judgments-1998-current/icetv-pty-limited-v-nine-network-australia-pty-li See also: AI-Generated Content Copyright FAQ | Who Owns Code Generated by AI?

Caller Questions

Caller 1 — Robert | Retaining Wall Responsibility

⚖️ Topic: Robert’s neighbour purchased a vacant lot next to his block and developed it, building up the soil level above Robert’s property. Robert was required to put up a retaining wall as a result. Who bears legal responsibility for that wall? Relevant law: Neighbourhood Disputes (Dividing Fences and Trees) Act 2011 (Qld) ; Gold Coast City Council retaining wall policies

[Andrew] It depends on the height of the retaining wall and whether it sits on a common boundary or not.

[Robert] My neighbour came in later and built up above me. Are we responsible half each?

[Andrew] Is it on the actual boundary between both properties?

[Robert] Yes, on the border.

[Andrew] If it is not on the border, the responsibility rests with the owner on whose property it sits. But where a retaining wall is on or near the common boundary, responsibility generally rests with the owner who directly benefits from it — it is not automatically shared 50/50.

So if the wall was built to create a level yard for the property above yours, and its primary purpose is to prevent their soil from falling down onto you, the responsibility rests with the upper neighbour. If, on the other hand, you had cut into the slope to level your own yard, and the wall is holding back your own excavation, the responsibility rests with you.

There are also different rules depending on whether the wall is over 90 centimetres in height.

The other important point: QCAT does not have direct jurisdiction to hear retaining wall disputes unless there is also a dividing fence on top of the wall, and the fence needs to be removed or replaced as part of the retaining wall works. If there is no fence component, you may need to pursue the matter through QCAT’s minor civil dispute jurisdiction or through the Magistrates Court, depending on the amount in dispute.

Council is also a good avenue — Gold Coast City Council regulates retaining walls and will sometimes issue notices to fix if a wall is in disrepair and poses a risk.

[Robert] So if I had to put it up because they built up above me — I’m responsible?

[Andrew] If you put it up to prevent their soil falling onto you, I suspect the question of who benefits is more nuanced. But practically, if you constructed the wall, the responsibility for maintaining it likely sits with you unless you can reach an agreement with your neighbour or obtain a council order. This is a situation where getting specific legal advice early — before you have spent money — would be worthwhile.

What the Law Says

The Neighbourhood Disputes (Dividing Fences and Trees) Act 2011 (Qld) primarily deals with dividing fences and trees overhanging a boundary.⁹ It does not directly regulate retaining walls. As a result, retaining wall disputes sit in a legal grey area.

The general common law principle is that a retaining wall on or adjoining the boundary is the responsibility of the party who benefits from the wall retaining the soil on their side.¹⁰ If the upper owner elevated their land after the lower owner’s land was already established, the upper owner is generally responsible for containing their own fill. If the lower owner excavated, the lower owner created the condition requiring the wall.

Where both parties have contributed to the conditions requiring the wall, a shared responsibility arrangement may be negotiated. Council approval requirements for walls over 90 cm also introduce regulatory obligations that may include engineering certification.

📎 Relevant legislation: Neighbourhood Disputes (Dividing Fences and Trees) Act 2011 (Qld) See also: Retaining Wall Responsibility FAQ (QLD) | Gold Coast City Council — Retaining Walls

Caller 2 — John | Inter Vivos Gifts — Can Family Challenge Them After Death?

⚖️ Topic: If a person with capacity gives money or assets to their children or other family members during their lifetime, can other family members challenge those gifts after the person dies? Relevant law: Succession Act 1981 (Qld) — family provision; general property law principles

[John] If someone is alive, has capacity, and gives money to their family members — is there any comeback from other family members after they die?

[Andrew] If someone is a competent adult, they can make decisions about their property the way they want to make them. If they give assets or money freely and clearly as part of their adult decision-making process, that is free and clear and there should not ordinarily be any comeback on those gifts.

Now — if they were coerced, if there was undue influence, or if there are other factors that call the capacity or voluntariness into question, then potentially there could be a challenge. But free and voluntary gifts made by a person with capacity? Those should stand.

[John] Do they need paperwork behind it?

[Andrew] I would definitely document it, particularly if it is a significant amount or something you are worried might be questioned later. If it could be queried, I would want a written record of the rationale for the gift — what was given, to whom, why, and the circumstances at the time.

What the Law Says

In law, a gift made during a person’s lifetime is known as an inter vivos gift — “between the living.” The fundamental principle is that a mentally capable adult has an absolute right to dispose of their assets as they choose during their lifetime, and those decisions generally cannot be overridden by disappointed potential heirs after death.

However, there are circumstances in which an inter vivos gift may be vulnerable to challenge:

1. Undue influence: If the donor’s free will was overborne by a person in a position of trust or power, equity may set the gift aside. The High Court in Bridgewater v Leahy (1998) 194 CLR 457¹¹ confirmed this principle in the context of a substantial property transfer by an elderly person to a family member.

2. Lack of capacity: A gift made by a person who lacked the mental capacity to understand its nature and effect may be void or voidable.

3. Notional estate provisions: Queensland’s Succession Act 1981 (Qld) does not currently include the “notional estate” provisions available in New South Wales, which can claw back certain assets disposed of before death for the purposes of a family provision order.¹² In Queensland, a family provision claim is generally limited to assets forming part of the deceased’s estate at the time of death.

Practical advice: For significant gifts — particularly where there are multiple potential beneficiaries who might later dispute them — have a solicitor prepare a simple deed of gift, obtain a contemporaneous medical capacity assessment if there is any risk of later challenge, and keep a clear written record of the donor’s reasons.

📎 Relevant legislation: Succession Act 1981 (Qld) — Pt 4 (family provision claims); Property Law Act 1974 (Qld) See also: Family Provision Claims FAQ (QLD) | Wills & Estates Practice Area

Caller 3 — Anna | AI in Medical Consultations — What Happens to Your Health Data?

⚖️ Topic: Anna attended a medical consultation where her specialist asked permission to use AI note-taking software. She consented, but wants to know: where does that health data go? Is it used to train the AI? Can it be shared? Relevant law: Privacy Act 1988 (Cth) — Australian Privacy Principles (APPs), in particular APP 3 (collection), APP 6 (use and disclosure), APP 8 (cross-border disclosure), APP 11 (security)

[Anna] I had a medical consultation yesterday afternoon with a specialist I hadn’t seen before. He asked whether I’d mind if he had AI turned on — I think it was taking notes. I said that was fine. But I’m wondering: does what was recorded get out to anybody else? Could it be used generally?

[Andrew] It all depends on the plan he has set up. When you use AI note-taking software, your conversation is being processed on someone else’s computer — that is, on the software provider’s servers. There are two critical questions you should ask any health professional who uses AI in a consultation:

1. Is the data stored in Australia, or does it go overseas? My firm uses dedicated legal transcription software, and one of our requirements is that the data is retained within Australia. The reason is practical: if something goes wrong, you can pursue the matter through Australian law. If data is sent overseas, that legal protection becomes much harder to enforce.

2. Is the data used for AI training purposes? As I explained earlier, LLMs are trained by ingesting large amounts of data. If the software provider is using clinical consultation recordings as part of their training dataset, your private health information could effectively be feeding an AI model that other people will use. Most reputable medical AI software providers have contractual restrictions against this — but it is worth asking.

If the data stays in Australia and is not used for training, it is functionally equivalent to a doctor dictating notes into a secure audio recorder that is then transcribed by trusted software. The practical privacy risk is very low. The notes will be used to accurately document your consultation — which actually produces better medical records than scribbled handwriting or post-visit dictation.

I know dentists doing the same thing — making running commentary throughout a procedure with AI recording in the background, so they have a complete record of every step.

[Anna] I think my main concern was just the privacy — I didn’t want it getting out generally.

[Andrew] Absolutely, and that is a very reasonable concern. Asking those two questions — data location and training use — will give you a good picture of the actual risk.

What the Law Requires

Under the Privacy Act 1988 (Cth) , health information is classified as sensitive information and attracts the highest level of privacy protection.¹³ The Australian Privacy Principles (APPs) impose the following obligations on health service providers and their software vendors:

• APP 3 — Health information may only be collected if it is reasonably necessary for the primary purpose of the service (the consultation), and consent must be obtained for collection of sensitive information.
• APP 6 — Health information collected for one purpose (clinical notes) may not be used or disclosed for a different purpose (AI training) without the patient’s consent, unless an exception applies.
• APP 8 — Before disclosing personal information to an overseas recipient (such as a cloud server located outside Australia), the organisation must take reasonable steps to ensure the overseas recipient handles the information in accordance with the APPs.
• APP 11 — The organisation must take reasonable steps to protect the information from misuse, interference, loss, and unauthorised access.

A medical practitioner who uses AI note-taking software without ensuring it complies with the APPs, or who has not disclosed the AI use clearly to the patient, risks a complaint to the Office of the Australian Information Commissioner (OAIC) .

Patients always have the right to ask their doctor to turn off the AI at any point, and to request access to any notes generated from their consultation under APP 12.

📎 Relevant legislation: Privacy Act 1988 (Cth) APPs 3, 6, 8, 11, 12 See also: OAIC — Guide to Health Privacy (May 2025) | AI Medical Consultation Privacy FAQ | Technology Law Practice Area

Closing Announcements

In Memory of Kathryn Bell (1987–2026)

Before closing, Andrew shared some important personal news:

“I have some sad news and some happy news. The sad news is that my sister Kathryn passed away of bowel cancer on Monday evening. She was 38, and only 11 months from diagnosis. Her message was that she wanted to make sure that everyone gets checked early, so they don’t have to go through what she did. She was an incredibly kind person, and what she wanted to be remembered for was being kind. So if everyone can do something kind today without expectation of return, I would really appreciate that as a memory of Kathryn.”

Bowel cancer is the second most common cancer diagnosis in Australia. Early detection dramatically improves outcomes. If you are over 45, or have a family history of bowel cancer, please speak to your GP about a bowel cancer screening test. The National Bowel Cancer Screening Program provides free home testing kits to eligible Australians at cancerscreening.gov.au .

Congratulations to Ross Lee OAM

Andrew also congratulated his friend and colleague Ross Lee of Lee Lawyers , who was awarded the Medal of the Order of Australia (OAM) in the King’s Birthday Honours for founding My Community Legal , a free legal advice service operating out of Robina and Southport at which Andrew volunteers his time every Thursday evening. Congratulations, Ross. A well-deserved recognition.

Key Takeaways

1. AI is sophisticated autocomplete, not intelligence. Large language models predict statistically likely word sequences. They do not independently verify facts. They are programmed to always produce an answer, even when that answer is wrong. Understanding this limitation is essential for anyone using AI tools in a legal, medical, or professional context.
2. AI hallucinations are a serious risk in legal practice. AI tools regularly generate plausible-sounding but entirely fictitious case names, citation numbers, and section references when asked about Australian law. Every AI-generated legal authority must be independently verified before use. Filing unverified AI-generated references in court proceedings carries professional conduct consequences for lawyers and adverse cost risks for all parties.
3. AI-generated content is generally not protected by copyright in Australia. Under s 32 of the Copyright Act 1968 (Cth) and following IceTV [2009] HCA 14, copyright requires human authorship and independent intellectual effort. Purely AI-generated images, articles, or code without meaningful human creative input fall into the public domain and are freely copyable.
4. Retaining wall responsibility follows the benefit, not the boundary. The owner who benefits from the wall retaining soil on their side generally bears responsibility for it in Queensland. QCAT does not have direct jurisdiction over retaining wall disputes unless a dividing fence is also in dispute. Council can be an alternative avenue for enforcement.
5. Inter vivos gifts by a person with capacity are generally final. A competent adult can freely give away their assets during their lifetime, and those gifts cannot ordinarily be reclaimed through the estate after death. Document significant gifts with a deed of gift, a record of reasons, and if appropriate, a capacity assessment.
6. Your health data has strong legal protections when AI is used in a consultation. Under the Privacy Act 1988 (Cth) , health information is sensitive information subject to the Australian Privacy Principles. You have the right to ask your doctor where AI-generated notes are stored, whether your data is used for AI training, and to access any records generated about you.
7. Get bowel cancer screening done early. In memory of Kathryn Bell — please speak to your GP or visit cancerscreening.gov.au .

About This Programme

Legal Matters is broadcast live on 4CRB every Tuesday at 9:00 AM and is hosted by Colin Balewski. Legal insights are provided by Andrew Bell of Bell & Senior Lawyers, Southport, Gold Coast.

A recording of this programme and all resources discussed are available at bellsenior.com.au and 4crb.com .

Bell & Senior Lawyers — (07) 5532 8777 — bellsenior.com.au

Footnotes