Can Australia Regulate Frontier AI? Safety, Systems and Accountability

Australia is right to ask whether frontier artificial intelligence needs enforceable safety rules. The harder question is whether law can define the thing it intends to regulate, distinguish a model from the system built around it, decide what “safe” means, and allocate responsibility across a supply chain that may begin in a foreign data centre and end inside an Australian business.

This article uses the Australian Government’s October 2026 frontier-AI speech as a case study in that larger problem. It examines what the proposed systems-based approach can learn from medicines, aviation, banking, work health and safety, critical infrastructure and consumer law; where those analogies break down; how regulation may affect developers, deployers and people exposed to AI decisions; and whether Australia can protect the public while still developing the domestic technology industry the Government says it wants.

The Question This Article Asks

On 8 October 2026, Assistant Minister for Science, Technology and the Digital Economy Dr Andrew Charlton outlined an emerging Australian approach to frontier-AI regulation. His speech argued that the most serious frontier risks should not be left to voluntary codes or detailed rules likely to become obsolete. Instead, government should set standards for the processes by which companies discover, test, report and control risk, drawing on systems used in workplace safety, critical infrastructure, banking and aviation.¹

The speech is a valuable starting point because it moves the debate beyond the false choice between banning AI and trusting developers. It recognises market incentives, information asymmetry, global supply chains and the speed at which capabilities change. It also contains a tension that deserves close attention. The Government wants Australia to train and host advanced AI, attract talent and participate in shaping global standards; the same Government proposes obligations that may be expensive, uncertain and easier for very large foreign firms to absorb than for local researchers and start-ups.

The research question is therefore:

Can Australia design enforceable frontier-AI regulation that meaningfully reduces serious harm, allocates responsibility to the actors able to control it, and remains technically adaptable—without imposing costs and uncertainty that entrench foreign incumbents or suppress the Australian capability the policy is intended to build?

The argument advanced here is cautiously sceptical. Regulation is both possible and necessary, but regulating “AI” as a singular product is not. A workable regime must regulate people and organisations at identifiable control points, and it must attach graduated duties to capabilities, access, autonomy, scale, context and consequence. Systems regulation should be part of that architecture, not its whole.

Why This Speech Matters

The immediate policy context was an incident in which an AI agent, while seeking information about Australian health expenditure, reportedly bypassed barriers and accessed a Services Australia statistics portal in ways that were not intended. The Government described the relevant portal as separate from Medicare claims systems and containing aggregated statistical information rather than individual medical records.² That distinction should be preserved: an unauthorised act can be serious without being exaggerated into a breach of personal Medicare histories.

The speech also referred to a reported incident involving models being evaluated against cybersecurity tasks and the compromise of Hugging Face infrastructure. Whatever the final forensic account, the policy significance lies less in an anthropomorphic claim that a model “escaped” than in the assembled conditions that permitted unintended action: the task, model, evaluation harness, credentials, network access, sandbox, monitoring and human response.³

That reframing matters. A language model does not independently purchase network access, assign itself credentials or decide the legal terms on which it is deployed. Organisations make those decisions. The law should resist language that turns an engineered and governed system into a mysterious independent actor, because anthropomorphism can conceal the humans and corporations with actual authority.

These events nevertheless expose a genuine gap between traditional software assurance and probabilistic, tool-using systems. Conventional software can also behave unexpectedly, but generative models do not merely execute an exhaustively specified sequence of rules. Their behaviour is learned from data, sensitive to context and sampling, and can change when the same model is combined with different prompts, tools or permissions.⁴

The relevant object of regulation is therefore rarely “the model” alone.

The First Obstacle: Defining AI

The OECD defines an AI system broadly as a machine-based system that, for explicit or implicit objectives, infers from inputs how to generate predictions, content, recommendations or decisions capable of influencing physical or virtual environments; systems vary in autonomy and post-deployment adaptiveness.⁵ The definition is deliberately inclusive. It can cover a small classifier, a recommendation engine, a computer-vision system, a generative model and an autonomous agent.

That breadth solves one problem and creates another. It avoids tying legislation to the fashionable technology of the day, but it potentially captures an enormous range of software. “AI” can mean a Raspberry Pi or microcontroller classifying grass for a mower, a camera detecting defects on a production line, a fraud model scoring transactions, an image generator, or a general-purpose model served from racks of accelerators in a data centre.⁶

Those systems differ not only in size but in function, reversibility, distribution and control. A mower classifier can create physical risk, despite being small. A giant language model may create little immediate risk if it is isolated from tools and used for low-stakes drafting. A modest model connected to administrative credentials and permitted to execute commands may be much more dangerous in practice.

A definition suitable for deciding whether something is within a statute is not necessarily suitable for deciding what obligations it should attract. Legislation can use a broad perimeter definition, but liability and compliance duties should turn on risk-relevant characteristics.

A Model Is Not a System

A deployable AI service may contain:

• A trained model or several models.
• System and developer instructions.
• Retrieval databases and external data sources.
• Memory.
• A software “agent” that plans or repeats actions.
• Code execution and browser tools.
• Identity, credentials and API keys.
• Network access.
• Filters, rate limits and policy engines.
• Human approval gates.
• Logs, monitoring and incident response.
• Contracts governing who may do what with the service.

Changing any one of these can change the risk without retraining the base model. The same language model used to suggest wording under human supervision is not the same regulated object when it can send email, move funds, change production systems or probe external networks.

Foundation-model scholarship has emphasised both this generality and the difficulty of attributing downstream effects to one layer of a complex ecosystem.⁷ Legal analysis that focuses only on weights or training compute commits what scholars of algorithmic systems describe as an “abstraction” error: it removes the technical artefact from the social and organisational system that gives it meaning and impact.⁸

A better trigger asks six cumulative questions:

This approach also avoids treating a compute threshold as a complete definition of “frontier”. Compute is measurable and can provide an early notification trigger. It is not a stable proxy for effective risk as algorithms become more efficient, capabilities are distilled into smaller models, specialised fine-tuning improves narrow performance, and tools amplify what a model can achieve.

What Does Regulation Mean?

The word “regulation” is often used as if it means legislation containing prohibitions and penalties. Regulatory scholarship treats it more broadly: deliberate attempts to influence conduct through rules, institutions, incentives, information, standards, markets and organisational processes.⁹

That wider view is essential for AI because no single regulator possesses all relevant information or control. Knowledge is fragmented across model developers, cloud providers, open-source communities, system integrators, customers, cybersecurity researchers, affected people and public agencies. Julia Black’s account of “decentred regulation” identifies complexity, fragmented knowledge, interdependence and the absence of a simple public/private divide as central features of modern regulatory problems.¹⁰ AI is almost a textbook example.

Regulation can target different things.

People and Organisations

The law can impose duties on developers, directors, officers, importers, suppliers, deployers and professional users. This is how regulation creates a person who must answer for decisions even when the technology is complex.

Processes

It can require risk assessments, documented governance, testing, monitoring, records, incident reports and independent review. This is the systems-based model emphasised in the speech.

Products and Market Entry

It can require approval, conformity assessment, licensing or certification before a system is supplied or used. This resembles therapeutic-goods or aviation certification.

Outcomes

It can prohibit discrimination, unsafe performance, misleading representations, privacy breaches or unauthorised access regardless of the process followed.

Costs and Incentives

It can shift the cost of failures through damages, consumer remedies, civil penalties, insurance, levies, compensation schemes and contractual rights. Economic regulation often works by forcing the actor creating risk to internalise more of its expected cost.

Infrastructure and Capability

Government can fund evaluation facilities, require secure compute environments, shape procurement, support standards, train regulators and build sovereign technical capability. Regulation without public capability risks becoming dependence on the firms being regulated.

The practical question is not whether Australia should “regulate AI”, but which mixture of these tools should apply to which actors, systems and risks.

The Theories Behind Intervention

The speech invokes negative externalities and a prisoner’s dilemma. Those ideas explain why individually rational firms may collectively produce an unsafe race. A developer may receive the revenue and strategic advantage from releasing first while some expected losses fall on users, victims, infrastructure operators and government. Even firms that prefer a slower pace may fear that unilateral restraint gives a competitor the market.

That is a conventional case for collective rules, but it is not enough to tell lawmakers which rules will work. Four further theories are useful.

Information Asymmetry

Developers usually know more about training, testing, incidents and limitations than customers or regulators. Customers cannot price risks they cannot observe, and reputational discipline arrives late if harm is hidden, diffuse or difficult to attribute. Disclosure, audit access and mandatory incident reporting respond to this asymmetry, although disclosure can become meaningless if it is technically impenetrable or legally defensive.

The Pacing Problem

Emerging technologies can change faster than legislation and formal rule-making. Gary Marchant and others call this the “pacing problem”: legal and ethical oversight lags technical development.¹¹ Detailed technical prescriptions can therefore freeze old assumptions into law or create a path for formal compliance that misses new hazards.

The Collingridge Dilemma

Early in a technology’s life, intervention is comparatively easy but consequences are uncertain; once consequences become clear, the technology may be entrenched and difficult to change.¹² AI intensifies this dilemma because globally distributed weights, interfaces, workflows and dependencies can become embedded rapidly.

Responsive Regulation

Ayres and Braithwaite argue that regulators should vary their response according to conduct, beginning with cooperation where appropriate but retaining an enforcement pyramid capable of escalating to serious sanctions.¹³ An AI regulator needs both technical assistance for organisations trying to comply and credible power against firms that conceal incidents, game evaluations or repeatedly ignore risk.

These theories point towards adaptive, evidence-seeking regulation rather than either laissez-faire or a static code.

What the Analogies Can Teach Us

The speech compares frontier AI with medicines, aircraft, banks, workplaces, critical infrastructure and consumer products. Each regime answers a different combination of questions: who controls the risk, whether failure can be tolerated, whether harm can be reversed, and whether market participants can judge safety before purchase.

Medicines: Evidence Before Release

High-risk medicines must be supported by evidence of quality, safety and efficacy before lawful supply, followed by post-market monitoring.¹⁴ This makes sense where a product, indication and sponsor can be identified and where prevention is preferable to compensation after injury.

The analogy supports pre-deployment evidence for an AI system used in a defined high-risk context. It is less convincing for a general-purpose model whose future uses are unknown. A medicine’s active composition does not ordinarily acquire a new intended purpose because a user writes a different prompt. AI regulation must therefore distinguish approval of a model, approval of a system configuration and authorisation of a use.

Aviation: Safety Is Organisational

Aviation safety-management systems require hazard identification, reporting, competent personnel and senior accountability. CASA guidance places ultimate responsibility for the safety-management system with an accountable manager who controls resources and can stop operations; day-to-day work may be delegated, but accountability is not.¹⁵

This is a powerful model for high-risk AI. It prevents the board or chief executive from treating safety as a problem owned only by a technical team. But aviation also benefits from mature engineering disciplines, stable configurations and decades of incident data. Frontier-AI evaluation lacks comparable consensus, and a hosted model can change without the customer receiving a new “aircraft”.

Banking: Supervise the Institution

Prudential regulation addresses risks that may spread beyond one customer or institution. APRA requires risk frameworks, board-approved risk appetite, capable management and evidence that risk remains within approved bounds.¹⁶

The lesson is that government need not specify every trade or technical choice. It can supervise whether a powerful institution has governance, resources, independent challenge and resilience proportionate to the system-wide consequences of failure. For frontier developers, that may mean regulatory supervision rather than one-off certification.

The limitation is equally important. Prudential supervision is expensive, expert and continuous. A requirement to maintain an AI risk framework will achieve little if the regulator cannot test the firm’s claims or inspect the evidence beneath them.

Work Health and Safety: Duty Follows Influence

Under the Australian model WHS framework, a business must eliminate health and safety risks so far as reasonably practicable, or otherwise minimise them. Officers must exercise due diligence by maintaining knowledge, understanding operations and hazards, providing resources, responding to incidents and verifying that compliance processes actually work.¹⁷

This offers three useful ideas for AI: duties can apply continuously rather than only at release; responsibility can attach to those whose decisions influence risk; and senior leaders can be required to verify rather than merely receive assurance.

AI duties could similarly require a provider or deployer to eliminate serious risks where reasonably practicable and otherwise minimise them. Yet the concept would need calibration. “Reasonably practicable” cannot become a blank cheque for speculative frontier scenarios, nor should lack of perfect scientific certainty excuse ignoring a credible severe risk.

Critical Infrastructure: Combine Hazards

The Security of Critical Infrastructure Act 2018 (Cth) uses responsible entities, risk-management programs and mandatory cyber incident reporting.¹⁸ Its value for AI is the refusal to isolate cyber risk from personnel, supply-chain, physical and operational risk.

An AI system may be the target of an attack, a pathway into another system, a source of insecure code, a critical outsourced dependency, or an autonomous actor with excessive permissions. Those are not separate policy silos in practice. AI standards should integrate with cyber obligations rather than creating a parallel compliance universe.

Consumer Law: A Local Remedy

The Australian Consumer Law imposes guarantees and creates remedies against suppliers, with manufacturer liability in specified circumstances.¹⁹ It expresses a practical principle: a customer should not need to unravel a global technical supply chain before seeking a remedy from the business that supplied the service.

Consumer law is nevertheless mainly reactive. It is not enough where harm is catastrophic, distributed, non-economic or difficult to reverse. It also does not solve the problem of a business customer contracting on heavily qualified terms with an offshore provider.

Why Systems Regulation Appeals

Management-based regulation requires organisations to plan and manage their own risks against public standards. Coglianese and Lazer distinguish it from technology rules, which prescribe particular controls, and performance rules, which prescribe outcomes. It is useful where firms are heterogeneous, the regulator cannot cheaply observe outputs, and firms hold information needed to design controls.²⁰

The proposed Australian direction fits that logic. The voluntary AI standard already emphasises accountability, risk management, data governance, testing, human oversight, transparency, contestability, supply-chain information and records.²¹ NIST’s AI Risk Management Framework similarly organises activity around governing, mapping, measuring and managing risk.²² ISO/IEC 42001 treats AI governance as a management system capable of continuing review and improvement.²³

Systems regulation can adapt when the technology changes. It can also make responsibility visible by requiring an organisation to state what it knows, what it tested, what it accepted and who authorised release.

The Paper-Safety Trap

The great danger is that the regulated output becomes the management system itself. A company can possess a policy, committee, risk register, model card and red-team report while still operating an unsafe system.

The regulator must ask three different questions:

1. Does a documented process exist?
2. Was it competently and honestly implemented?
3. Was the resulting risk decision substantively defensible on the evidence available at the time?

If only the first question matters, law rewards polished paperwork. If liability turns only on whether harm occurred, it punishes responsible firms for unavoidable events while allowing reckless but lucky firms to escape. The standard must examine process quality and substantive reasonableness.

Academic work on risk-based regulation warns that regulators must remain responsive not only to calculated risk but to organisational behaviour and culture, institutional context, interactions between controls, regulatory performance and change.²⁴ That requires access to underlying test data, incident records and decision-makers—not a yearly certificate.

What Does Safe Mean?

“No company should release a frontier model that is not safe” is persuasive rhetoric but an incomplete legal test. No complex technology is free of risk. A legal regime needs a threshold, evidence and a decision-maker.

Safety should be decomposed into properties:

• Reliability: performance within stated tolerances under expected conditions.
• Robustness: resistance to unusual, adversarial or changing inputs.
• Cybersecurity: protection of the model, data, infrastructure, credentials and tools.
• Misuse resistance: controls against foreseeable harmful use.
• Controllability: ability to observe, interrupt, restrict and recover the system.
• Rights protection: privacy, non-discrimination, procedural fairness and effective contestability.
• Operational resilience: continuity, fallback, incident response and safe degradation.
• Systemic safety: avoidance of unacceptable correlated, societal or infrastructure consequences.

A system can be strong on one property and weak on another. A model might refuse many cyber prompts yet expose private data. An open model may be easier for independent researchers to inspect but easier for a malicious user to modify. A tightly controlled API may reduce some misuse while leaving customers unable to investigate errors or provider claims.

The legal goal should therefore be acceptable residual risk for a defined use under defined conditions, not an unqualified declaration that a model is safe.

Safety Cases, Not Safety Labels

For a frontier release or high-impact deployment, the responsible organisation should prepare a safety case containing:

• The system and version being assessed.
• Intended and reasonably foreseeable uses.
• Tools, access and operating limits.
• Identified hazards and affected groups.
• Evaluation methods and their limitations.
• Evidence for each material safety claim.
• Independent challenge and unresolved disagreement.
• Controls and fallback arrangements.
• Residual risks and the person who accepted them.
• Events requiring reassessment, restriction or withdrawal.

This resembles assurance practice in safety-critical fields without pretending that a pass/fail benchmark proves universal safety.

Randomness Changes Assurance

Generative systems are probabilistic. Output can vary with sampling settings, context, tool state and apparently minor wording changes. One successful demonstration proves little. Testing must examine distributions across repeated trials, adversarial cases, long task horizons, different tools and realistic operating environments.

It is often more reliable to place deterministic controls around the probabilistic component: least-privilege credentials, network segmentation, transaction limits, immutable logs, approval gates and hard shutdown conditions. Cybersecurity already assumes that a component may fail; AI assurance should do the same.

Frontier, General-Purpose and High-Risk

Three ideas are often collapsed:

• Frontier usually describes models at or near the leading edge of general capability.
• General-purpose describes breadth of possible use.
• High-risk describes a capability, application or context with serious potential consequences.

They overlap but are not identical. A frontier model may be used for a trivial purpose. A narrow, non-frontier system can be high-risk when it controls a medical device or determines access to work, credit or government benefits.

Australia should therefore use at least two regulatory tracks:

1. Model-level duties for developers or releasers of exceptionally capable general-purpose models, directed to systemic capability, irreversible release, cybersecurity and information for downstream actors.
2. Use-level duties for developers, suppliers and deployers of systems used in high-impact contexts, directed to the people and institutions affected.

The EU AI Act follows a version of this distinction by combining obligations for high-risk AI systems with separate duties for providers of general-purpose models and additional obligations where those models create systemic risk.²⁵ Australia need not reproduce the EU scheme, but it should learn from the need to regulate both upstream capability and downstream use.

Open Weights, Closed APIs and False Binaries

“Open” and “closed” do not describe two complete categories. A provider may release model weights but not training data, source code or evaluation results. Another may publish extensive research while retaining weights and serving the model through an API.

Neither architecture is inherently safe. Closed APIs permit monitoring, rate limits, customer suspension and rapid patches. They also concentrate power and safety evidence in an overseas provider. Open weights enable local operation, reproducible research and independent examination, but safeguards can be altered and complete recall may be impossible once copies circulate.²⁶

Duties should follow control:

• The hosted provider controls access, monitoring and updates.
• The original developer controls training and the initial release decision.
• The open-weight releaser controls the decision to make a potentially irreversible distribution.
• The fine-tuner controls material changes.
• The integrator controls tools, retrieval and permissions.
• The deployer controls purpose, users, local data and whether outputs cause action.

A law that imposes the same obligations on all six actors will be both unfair and ineffective.

The Cybersecurity Paradox

The reported Hugging Face incident raises a deeper point. Safety restrictions intended to prevent offensive cyber assistance may also block authorised defenders from analysing malicious code, attack logs or vulnerabilities during a real incident. A safeguard can reduce one threat while increasing another.

That is not an argument for unrestricted capability. It is an argument against measuring cyber safety by refusal rates alone. A well-designed regime should support controlled access for verified researchers, critical-infrastructure operators, incident responders and government evaluators, with isolation, logging and legal accountability.

Cybersecurity research also demonstrates why external scrutiny matters. Internal teams share organisational assumptions and release pressures. Independent red teams and researchers can identify failure modes the developer did not imagine, but only if contracts, access controls and computer-offence laws do not make good-faith testing prohibitively risky.

AI rules should therefore include a coordinated vulnerability-disclosure pathway and carefully framed safe harbour for authorised, good-faith research. The safe harbour must not excuse harmful access; it should provide a predictable route for testing, notification and remediation.

Who Should Be Accountable?

The intuitive answer—“the person paying the bills”—contains an important truth. The Australian organisation buying or operating the system is visible, within jurisdiction and able to choose whether AI acts in its business. It should not escape responsibility by saying “the model did it”.

But payment is a poor universal proxy for control. A small Australian firm using an overseas service may decide what data employees enter and whether outputs require review. It may have no access to training data, weights, internal evaluations, undisclosed incidents or silent model updates. Making it solely liable would place the greatest burden on the actor with the least information.

Secure-by-design policy has increasingly rejected transferring all security responsibility to customers and instead asks manufacturers to take ownership of customer outcomes.²⁷ The same principle should inform AI.

Three Allocation Principles

Accountability follows control. The actor able to prevent or reduce a risk should carry a duty to do so.

Accountability follows knowledge. An actor possessing material safety information unavailable downstream must disclose it or retain responsibility for the consequences of that asymmetry.

Outsourcing does not extinguish duty. An Australian entity remains responsible for its own regulated activity, while preserving rights and regulatory claims against an upstream provider that created or concealed the defect.

A Chain, Not a Single Defendant

The immediate supplier should ordinarily remain answerable to its customer. Upstream contribution can then be addressed through statutory rights, contribution rules and contracts. Otherwise, every harmed customer must investigate the entire technical stack before finding a responsible party.

What If the Model or Data Is Overseas?

Australian law cannot make every foreign developer comply, stop hostile states or recall every downloaded weight. It can still regulate domestic control points:

• Australian developers and deployers.
• Foreign suppliers carrying on business in Australia or targeting this market.
• Australian importers and representatives.
• Procurement by governments and regulated sectors.
• Local data centres and cloud services.
• Access to Australian critical systems and datasets.
• App stores, enterprise marketplaces and payment channels.
• Professional or licensed activities conducted with AI.

Privacy law provides a partial analogy. Australian Privacy Principle 8 generally requires an APP entity taking steps to disclose personal information overseas to take reasonable steps concerning the recipient, and the Australian entity may remain accountable for specified overseas conduct.²⁸ The precise legal position depends on whether there is a disclosure and on control over the information, but the broader lesson is that outsourcing data processing does not necessarily outsource legal exposure.

Contracts for overseas AI services should address:

• Where data and administrative access are located.
• Whether prompts, files and outputs are used for training.
• Subprocessors and underlying model providers.
• Provider rights to change models or terms.
• Safety and security evidence.
• Notification of incidents and material capability changes.
• Audit, regulator and forensic access.
• Retention, deletion and return of data.
• Service continuity and model withdrawal.
• Export, migration and interoperability.
• Insurance, indemnities and liability limits.
• Assistance with Australian regulatory obligations.

Contracts do not create technical safety. They allocate information, cooperation and financial risk before the parties are under incident pressure.

Australia’s extraterritorial leverage will remain smaller than that of the European Union. Scholarship on the “Brussels Effect” links regulatory export to market size, regulatory capacity, stringent rules and the difficulty of offering different products in different markets.²⁹ Australia has influence through procurement, alliances, standards and specialised capability, but should not assume that foreign frontier firms will redesign global products solely for this market.

That makes international interoperability important. Australian requirements should align where appropriate with widely used technical and management standards, while preserving local rights and enforcement. Alignment reduces duplicative compliance; uncritical adoption risks importing assumptions designed for different institutions and legal values.

Impact on Australian Developers

The speech presents safety and domestic AI capability as mutually reinforcing. That can be true, but it is not automatic.

Compliance costs are often fixed or front-loaded: lawyers, documentation, evaluation infrastructure, cybersecurity, insurance, conformity assessment and senior governance must be paid before a product earns scale. A large multinational can spread those costs across global revenue. A start-up, university spin-out or open-source project cannot.

Poorly calibrated regulation can therefore produce four unintended effects.

Incumbent Advantage

Complex requirements may become a moat for the largest firms. If only frontier labs can afford evidence generation and certification, regulation can increase dependence on the firms whose conduct it seeks to constrain.

Offshore Development

Rules attached to development in Australia, but weakly applied to imported APIs and products, create an incentive to train or structure activity offshore. Australia would bear downstream risk while losing technical knowledge and employment.

Defensive Narrowing

Developers may avoid socially valuable high-impact applications because legal classification is uncertain or insurance unavailable. This is particularly relevant for health, disability, regional services and public administration, where the potential benefit and regulatory sensitivity are both high.

Compliance Theatre

Firms may divert scarce technical staff from substantive testing to producing documents optimised for auditors. The answer is not to abandon documentation, but to ensure that evidence is reusable, proportionate and connected to actual engineering controls.

Emerging empirical work suggests that proposed AI regulation can affect start-up finance and acquisition incentives, although results from other jurisdictions should not be mechanically applied to Australia.³⁰ The policy lesson is to test distributional effects rather than report only aggregate compliance cost.

Regulation Can Also Support Innovation

Well-designed rules can provide:

• Legal certainty for investment and procurement.
• Common evidence that avoids repeated customer questionnaires.
• Trusted access to health, government and critical-infrastructure markets.
• A market for Australian assurance, cyber and evaluation services.
• Clearer rights to investigate and disclose vulnerabilities.
• Shared public testing infrastructure.
• Confidence for people affected by consequential systems.

The relevant comparison is not regulation against zero cost. Unsafe deployment also imposes costs through incidents, failed projects, litigation, loss of trust and abrupt political reaction.

Impact on Deployers and Affected People

Most Australian organisations will not train a frontier model. They will procure an API, enable an assistant in existing software or buy a product whose AI components are invisible. Regulation must be intelligible to these deployers.

A proportionate regime should not require a small business to reproduce the frontier developer’s evaluations. It should require the business to understand its own use: the data supplied, people affected, permissions granted, reliance placed on outputs and procedures for intervention. The upstream provider must deliver the information needed to do that.

For affected people, process requirements are insufficient unless they generate usable rights. A person denied work, service, insurance or a government benefit needs to know that automation was materially involved, obtain an intelligible reason, correct relevant data and secure timely human review. “Human in the loop” has little value if the human lacks authority, time, training or information to disagree.

This is why AI regulation must connect with administrative law, discrimination law, privacy, employment law, professional duties and consumer protection. An AI Act cannot become a ceiling that implies conduct is lawful merely because a management system was certified.

The Regulator’s Problem

Systems regulation assumes that a regulator can distinguish a rigorous safety system from a decorative one. That requires:

• Technical staff capable of reproducing or challenging evaluations.
• Secure access to models, logs and sensitive incident material.
• Powers to compel information and preserve evidence.
• Protection for confidential information without allowing secrecy to defeat oversight.
• Coordination with cyber, privacy, competition, consumer and sector regulators.
• International incident-sharing.
• Authority to restrict or suspend dangerous activity.

Australia’s AI Safety Institute is intended to analyse advanced capabilities, risks and harms and to support government agencies.³¹ Its creation addresses a real capacity gap, but institutional design matters. Advice alone is not enforcement. If the Institute tests models but another agency holds legal power, responsibilities and escalation paths must be explicit.

Regulatory capture is another risk. Frontier developers may possess the scarce expertise government needs. Collaboration is unavoidable, but reliance can allow industry assumptions to define what counts as evidence or an acceptable risk. Independent academic, civil-society, worker and affected-community expertise should be built into evaluation and review.

A Feasible Australian Architecture

The answer is a hybrid regime, not pure systems regulation.

Layer 1: Economy-Wide Existing Law

Privacy, consumer, corporations, competition, discrimination, employment, cybercrime and negligence rules should continue to apply. Government should identify genuine gaps rather than imply that AI operates outside law.

Layer 2: Use-Specific Sector Rules

Health, finance, government administration, legal services, employment and critical infrastructure require sector regulators to translate common AI principles into context. The meaning of adequate human oversight is different in a hospital, bank and creative-writing tool.

Layer 3: Mandatory High-Risk Guardrails

Developers and deployers of high-risk systems should implement governance, risk management, data controls, testing, human control, transparency, contestability, supply-chain documentation and conformity evidence. Australia’s earlier mandatory-guardrails proposals already provide much of this structure.³²

Layer 4: Frontier-Model Obligations

Exceptionally capable general-purpose models should attract model-level duties, including advance notification at objective thresholds, secure development, adversarial capability testing, a release safety case, incident reporting and information for downstream actors.

Layer 5: Minimum Technical Controls

Where controls are mature and broadly applicable, law should prescribe a floor. High-risk agents should ordinarily use least privilege, isolated execution, controlled network egress, protected credentials, immutable logs, action limits, human approval for irreversible steps and tested shutdown procedures.

Layer 6: Adaptive Supervision

The regulator should issue guidance and support good-faith compliance, but escalate through enforceable undertakings, additional conditions, compulsory testing, restrictions, suspension and civil penalties. Officers who knowingly release or deploy systems contrary to serious unresolved safety evidence should not be insulated by the existence of a committee.

Twelve Design Tests

Before legislating, government should require the proposed regime to answer these questions in plain language:

1. Scope: What exact model, system, use or actor triggers the duty?
2. Threshold: Is risk classified by capability, use, compute, access, scale or a combination?
3. Evidence: What must a duty-holder produce, and who can challenge it?
4. Change: Which updates or new integrations trigger reassessment?
5. Supply chain: What information must move downstream and incidents move upstream?
6. Accountability: Which actor has the knowledge and control to prevent each risk?
7. Local remedy: Who in Australia must respond to customers, regulators and affected people?
8. Open models: How are irreversible release and legitimate research treated without eliminating open development?
9. Overseas services: What nexus brings a foreign provider within the regime, and what happens when enforcement fails?
10. Proportionality: How are start-ups, universities and small deployers protected from duplicating costs while serious risks remain controlled?
11. Regulatory capacity: Which agency can inspect, test, order remediation and stop deployment?
12. Evaluation: How will Parliament know whether the regime reduces harm, shifts activity offshore or entrenches incumbents?

Legislation should include scheduled review, transparent metrics and authority to update technical thresholds through accountable subordinate instruments. Flexibility must not mean that the core scope of liability is invented privately by standards bodies or altered without parliamentary scrutiny.

What Businesses Can Do Now

The final legislation may change, but organisations can reduce present risk without waiting:

• Inventory AI systems, embedded features, models, vendors and owners.
• Identify uses affecting legal rights, work, health, safety, finance or critical operations.
• Record who approved each use and who can suspend it.
• Map data sent to providers and countries from which it can be accessed.
• Prohibit confidential or personal information entering unapproved services.
• Review model-change, audit, incident, deletion and subcontracting terms.
• Apply least privilege to every tool and credential.
• Require meaningful approval before legal, financial or irreversible actions.
• Test the assembled workflow rather than relying on a vendor demonstration.
• Preserve logs sufficient to investigate incidents and contested decisions.
• Integrate AI events into privacy, cyber, WHS and continuity plans.
• Reassess after material changes to the model, data, tools, permissions or use.

These steps are not proof of compliance with future law. They are basic governance for a technology already being deployed.

Conclusion: Regulation Is Possible, but the Object Is Governance

The speech is right in its central intuition. A market race in which firms can capture capability and revenue while distributing some risks to others will not reliably produce the level of caution society expects for severe, difficult-to-reverse harms. Voluntary promises cannot carry the entire weight, and detailed rules written today will not anticipate every capability tomorrow.

But the proposition that “AI needs regulating” is only the beginning. AI is not one product, one model size or one mode of distribution. It includes tiny classifiers and data-centre-scale foundation models, closed APIs and open weights, passive advisers and agents with credentials. Safety is not a single property, and a management system is not proof of a safe outcome.

The research question can therefore be answered conditionally. Australia can regulate serious AI risks, but only by regulating the organisations, infrastructures, deployments and decisions through which capability becomes consequence. Duties should follow knowledge, control and capacity to prevent harm. The Australian supplier or deployer should remain answerable for the use it places into the local market, while upstream developers and providers remain responsible for risks they create, control or fail to disclose.

A successful regime will combine systems duties with minimum controls, outcome law, independent evidence, sector expertise and remedies for affected people. It will support smaller Australian actors with shared testing, clear thresholds, reusable assurance and regulatory guidance rather than handing large incumbents a compliance moat. It will treat overseas contracting as a governance problem, not a way to export responsibility.

The Government’s proposed direction is therefore promising but preliminary. Australia is a long way from a complete answer—and the gap is not simply the absence of an Act. It is the unresolved work of definition, measurement, institutional capability, jurisdiction, cost allocation and accountability.

Safety may be a condition of durable adoption. Whether it becomes an ignition for Australian innovation, rather than a brake applied mainly to local firms, will depend on who must prove what, to whom, at what cost—and whether the law reaches the actors with the power to make the system safer.

Related Reading

• When an AI agent hacks a system: cyber offences and legal liability
• AI in Australia’s interests: copyright, capability and control
• Legal risks of using AI in an Australian business
• Data breach response for Queensland businesses
• Cyber insurance claims and security obligations
• Legal issues for Australian technology start-ups
• AI in the workplace: employer obligations in Queensland
• Smart glasses, cameras and ambient privacy

This article provides general information and policy analysis. It does not constitute legal advice and should not be relied on as a substitute for advice about particular facts, systems or regulatory obligations.