Why underwriters, not legislators, will set the terms for physical AI

In 2024, Air Canada's customer service chatbot invented a refund policy. A grieving customer asked it about bereavement fares, booked his flight on the strength of what it told him, and submitted his receipts. Air Canada refused to pay. When the customer sued, the airline's defense was that the chatbot was, in its own words, "a separate legal entity that is responsible for its own actions."

The case was decided easily because the damages were small and the chatbot was, in the end, just a webpage that talked back. The liability question collapsed into an ordinary one about misrepresentation, and the tribunal answered it the way tribunals have answered such questions for a century.

But the underlying question — who is responsible when a system decides something a human used to decide? — is going to get much harder, very soon. The next decision-making agents won't live on webpages.

AI will lift pallets, drive trucks, hold scalpels, and walk through warehouses on two legs. Insurance is designed for a world where a human causes the damage — auto policies price the driver, homeowners' policies price the household, workers' comp assumes a person was at the controls. So how do we price the agent?

The major trend will be a shift from pricing, evaluating and managing risk around human operators to system-centric operators or system-centric liabilities. In the future, liability could look materially different.

— Chris Raimondo, EY · Insurance Business America

Most of the discussion about AI's impact on insurance has been centered on operations: internal workflows, claims triage, fraud detection, model-driven pricing. The more consequential story is the one Raimondo is pointing at: a change in what is being insured.

Physical AI — the autonomous machine layer that drives, lifts, decides, and acts in the real world — is becoming its own class of insurable risk and simultaneously transforming all other risk classes. So who builds the framework to govern and price this new status quo?

The model breaks when the operator isn't human

Auto, homeowners, and commercial general liability policies were written for a world where a person caused the damage. Even modern usage-based products are refinements of the same question: how risky is this driver, this household, this operator? Physical AI breaks the frame. When a humanoid drops a pallet, an autonomous vehicle misjudges a pedestrian, or a surgical robot makes an unfavorable movement, the chain of responsibility runs through the operator, the manufacturer, the software developer, and the foundation model provider, often simultaneously.

The contractual situation is worse than most operators realise. Many existing homeowners and general liability policies contain "motorised vehicle" or "aircraft" exclusions that carriers are already applying to mobile robots. Operators of physical AI today frequently carry less coverage than they think. This isn't a gap legislation can fix, because the exclusions are written into existing contracts.

Europe wrote the framework, then withdrew it

Europe saw this coming earlier than most. In a 2020 study for the European Parliament, Artificial Intelligence and Civil Liability, Andrea Bertolini, director of the Jean Monnet European Centre on the Regulation of Robotics and AI, proposed a Risk-Management Approach: for each class of AI application, identify the party best placed to control the risk and hold that party strictly liable as a single entry point for litigation. His four case studies were industrial robots, connected and automated driving, medical robots, and drones. Every one of them is now shipping at scale.

Bertolini's framework underpinned the European Commission's 2022 AI Liability Directive, championed by JURI rapporteur Axel Voss. It was the closest the world came to a legislated answer for who pays when an autonomous system causes harm.

In February 2025, the Commission withdrew it. Six months later, EIOPA's August 2025 Opinion on AI governance explicitly excluded high-risk AI from its scope "to avoid regulatory complexity." Europe, for now, has decided not to legislate.

That's a significant moment, because it leaves physical AI — a category Brussels itself identified as needing a clear liability regime — to be governed by something other than statute. The question is what.

What cybersecurity tells us about the answer

The clearest precedent is cybersecurity, and the comparison is worth taking seriously because it ran the experiment in both directions.

The United States has no comprehensive federal cybersecurity law. What it has instead is SOC 2, ISO 27001, and PCI-DSS — frameworks that emerged because cyber insurers and enterprise procurement teams refused to sign without them. A startup selling to a US bank in 2026 doesn't comply with a statute, it clears an underwriter's questionnaire. The questionnaire updates continuously as threats evolve, and the price of the policy reflects the answers. The market did the regulating.

Europe took the opposite route with NIS2, DORA, and the Cyber Resilience Act — prescriptive directives with timelines, fines, and named accountable parties.

Both approaches produced security improvements. But there's an asymmetry worth noting: companies subject to both regimes will tell you that the underwriter's questionnaire is what changes their engineering practices, while the directive is what changes their compliance documentation.

The insurance-driven layer moves faster, prices risk continuously, and creates financial consequences in real time. Statute does the opposite, almost by design.

Physical AI is now sitting at the same fork — and with the AI Liability Directive withdrawn, the EU has, perhaps unintentionally, ended up on the same path as the US. Insurance will be the de facto governance layer on both sides of the Atlantic.

Why insurance is (better) suited to this particular problem

There are three reasons physical AI may be even better suited to insurance-led governance than cyber was.

1. The failure modes are physical and measurable. A humanoid that drops a pallet leaves evidence. An autonomous vehicle that misjudges a pedestrian generates telemetry. Underwriting can be grounded in observable events rather than disputed definitions of what counts as a breach.
2. The chain of responsibility is genuinely ambiguous in a way no single statute resolves cleanly. When a surgical robot malfunctions, the liable party might be the hospital, the manufacturer, the foundation model provider, or the integrator. A directive picks one. An insurance market prices each contribution to risk separately and lets the allocation emerge from how contracts are negotiated and policies are structured.
3. The unit of risk has changed. There is no human in the loop, which means underwriting an autonomous truck means inspecting the system itself — its safety case, operational design domain, incident logs — not a driver's record. That's actuarial work that has to be built from scratch, and it can only be built by underwriters in close contact with the technology. And because every truck in a fleet runs the same software stack, the underwriter is no longer pricing thousands of individual drivers: they're pricing one system, deployed at scale. That collapses a lot of actuarial machinery that exists today only because human drivers are heterogeneous.

A pattern older than the technology

In 1752, Benjamin Franklin founded the Philadelphia Contributionship, America's first mutual fire insurance company, because eighteenth-century Philadelphia kept burning down and the colonial government wasn't going to fix it. The Contributionship funded fire brigades and wrote the first building safety standards — not because legislators asked, but because the actuarial math required it. The same pattern produced Underwriters Laboratories for electricity, the IIHS for cars, SOC 2 for cloud software, PCI-DSS for online payments.

In a July 2025 essay, AIUC co-founders Rune Kvist, Rajiv Dattani, and Brandon Wang call this the Incentive Flywheel: every transformative technology wave has been governed by the same trio of insurance, standards, and audits. In the UK, techUK's Tess Buckley and Alan Turing Institute professor Lukasz Szpurch put it more directly in September 2025: "Insurance requirements may actually precede regulatory mandates."

The exposure justifies the attention. Goldman Sachs projects the humanoid robot market at $38 billion by 2035. Swiss Re expects global data-centre insurance premiums to roughly double from $10.6 billion today to $24.2 billion by 2030. Commercial drone insurance is already a $1.5 billion market growing more than 20% a year.

The insurance layer for physical AI is being written now, and it will set the operating terms — what gets deployed, where, with what safeguards — long before any legislator catches up.

That's not a failure of regulation. It's how good innovation has always worked.