So when a bridge fails, nobody blames the driver for trusting the bridge. The driver may be the person most exposed to danger, but the responsibility sits with the engineers, builders, inspectors, and agencies that let people cross it.
That is the useful way to read MIT FutureTech's new AI risk report. It is a risk ranking, yes. But the sharper contribution is a responsibility map for AI: who is exposed, who has leverage, and who should be held accountable when the two groups are different.
First up, the TL;DR
MIT FutureTech published a new Delphi study (a structured expert survey where participants can update their answers after seeing anonymized peer feedback) asking 272 AI experts to rank 24 AI risks.
Here's what happened:
- Experts said 18 of 24 AI risk domains have at least a 10% chance of catastrophic outcomes by 2030 under business as usual.
- The top risks were dangerous AI capabilities, competitive dynamics, weapons and cyberattacks, power centralization, and false information.
- Catastrophic meant outcomes like more than 1M deaths, more than $100B in damage, or civilization-scale damage to things like privacy or democracy.
- Even with pragmatic mitigations, five risks stayed above 10%: dangerous capabilities, weapons and cyberattacks, environmental harm, inequality and unemployment, and power centralization.
- The public and AI users were rated most vulnerable, while general-purpose AI developers and governance actors were rated most responsible.
MIT also published an interactive findings page, a responsibility visualization, the full PDF, and a 7-minute video overview.
Why this matters:
The study turns AI risk from a vague doom debate into a practical accountability problem. If the public bears the downside while developers and governments hold the main levers, then asking individual users to "be careful with AI" solves the smallest part of the problem.
Our take:
The scary number needs a careful reading. This is expert judgment, not a calibrated forecast. But even a rough 10% catastrophic-risk estimate is the kind of signal that gets planes grounded, drugs recalled, and bridges inspected before people keep driving over them. The practical ask is boring by design: assign responsibility before the harm arrives.
The key finding
The report's headline number is brutal: under current trajectories, experts judged that 18 of 24 AI risk domains carry at least a 10% chance of catastrophic outcomes between 2025 and 2030.
The study asked experts to assess harm across 10 categories, including physical harm, financial loss, infrastructure damage, civil rights, democratic norms, and privacy. "Catastrophic" meant outcomes such as more than 1M deaths, more than $100B in financial losses, global democratic collapse, or a global loss of privacy.
The five highest-severity risks clustered around a few familiar pressure points:
- Dangerous capabilities: AI systems gaining or being given abilities that could enable deception, cyber-offense, weapons development, manipulation, or self-proliferation.
- Competitive dynamics: companies or countries racing to deploy more powerful AI before safety work catches up.
- Weapons and cyberattacks: humans using AI to build malware, improve weapons, or cause mass harm.
- Power centralization: control of powerful AI concentrating in a small number of companies, governments, or groups.
- False information: AI-generated misinformation undermining people's ability to make good decisions.
That list matters because it pushes the story beyond the usual consumer-AI harms. Bias, scams, and privacy leaks still matter. People are already being hurt by them. But the experts' top severity rankings put the biggest expected harms in systems-level risks: dangerous model capabilities, arms-race incentives, security threats, and power concentration.
In plain English, the most severe risks come from AI becoming powerful enough to matter at institutional scale before society has reliable ways to test it, limit it, and distribute its benefits.
The responsibility gap
The study's cleanest contribution is the split between vulnerability and responsibility.
Experts rated AI users and affected stakeholders, meaning people who are indirectly affected by AI systems, as the most vulnerable actors across nearly all 24 risks. That includes workers subject to AI hiring tools, consumers targeted by AI scams, patients affected by medical AI, voters exposed to synthetic propaganda, and everyone whose personal data gets swept into AI systems.
Then experts assigned the highest responsibility to two upstream groups: general-purpose AI developers, meaning the companies building foundation models, and governance actors, meaning governments, regulators, and standards bodies.
That split is normal in safety-critical industries. Patients do not test the drug supply. Passengers do not certify aircraft. Drivers do not inspect bridge design. In each case, the exposed public depends on a chain of engineers, manufacturers, auditors, regulators, and liability rules.
AI lacks that full chain. The industry has some safety testing, model cards, voluntary commitments, evals, red-team reports, and emerging rules. The report's point is that these mechanisms remain early compared with the scale of the risks experts were asked to evaluate.
That is why the responsibility finding matters more than the raw ranking. A list of risks tells policymakers where to look. A responsibility map tells them who has the duty, capability, and causal influence to do something about it.
The 10% number needs context
The report's counterweight is baked into its own limitations section: these are domain experts, not necessarily calibrated forecasters.
A cybersecurity expert may understand attack paths better than a generalist forecaster, but that does not automatically mean they assign probabilities better. The report also notes that its panel was heavily concentrated in North America and Europe, that self-selected AI risk experts may be more concerned than the broader expert population, and that the five-level severity scale compresses very different kinds of harm into one rubric.
There is also a deeper forecasting debate here. A recent Vox summary of the Forecasting Research Institute's work described how both AI experts and superforecasters underestimated the pace of AI progress from 2022 to 2025. Experts came closer on some benchmarks, but the results did not crown either group as reliable oracles.
That is the fairest skepticism toward this report: the 10% number should guide attention, not settle the future.
The right response is to treat it like a risk signal. When experts in a safety-critical field tell you that multiple failure modes have catastrophic tails, the useful move is neither panic nor dismissal. It is inspection, standards, stress testing, and accountability.
What pragmatic mitigations did, and did not, fix
The study asked experts to compare two scenarios.
In the business-as-usual scenario, organizations and governments continue current practices without extra AI-specific mitigations. In the pragmatic-mitigations scenario, they make cost-effective, realistic efforts to reduce AI risks.
Mitigations helped. Experts lowered expected severity across all 24 risks. The biggest drops showed up in areas such as dangerous capabilities, weapons and cyberattacks, competitive dynamics, and AI misalignment.
Then comes the stubborn part: even with pragmatic mitigations, every risk stayed above a 5% catastrophic probability, and five stayed above 10%.
So some AI risks can be reduced with model-level safety work, better cybersecurity, monitoring, evaluations, and deployment rules. Other risks are structural. Power centralization, labor disruption, environmental pressure, and international competition come from the way AI is financed, deployed, and governed.
A company safety team can block some dangerous outputs. It cannot rewrite the incentives that make every major lab feel pressure to ship faster, raise more money, secure more compute, and turn model access into market power.
That is why the report lands on governance, liability, transparency, standards, and international coordination. The main safety problem is whether the whole AI ecosystem rewards speed and concentration more than caution and accountability.
What this means for organizations
For most readers, the practical takeaway is less "AI could destroy the world" and more "AI risk should be assigned to the level where control actually exists."
That changes how companies should think about their own AI rollouts.
- Information companies should treat AI-generated content, personalization, and synthetic media as trust infrastructure, not content plumbing.
- Finance teams should assume AI-enabled fraud, market manipulation, and model failures will become normal operating risks.
- Healthcare teams should treat privacy, discrimination, and unsafe overreliance as patient-safety issues.
- National security and public-sector teams should prepare for compound risk, where AI is both a tool they use and a weapon others use against them.
- Any company deploying AI should define who owns the risk before the model touches customers, employees, or sensitive data.
The report also gives nontechnical leaders a useful filter for AI governance. Ask four questions: Who is exposed? Who can reduce the risk? Who benefits from moving fast? Who pays when something breaks?
When those answers point to different groups, voluntary caution will probably be weak. That is when rules, audits, liability, and outside enforcement start to matter.
Our take
The report is strongest when it avoids pretending AI risk is one thing.
A deepfake scam, a biased loan model, an AI-enabled cyberattack, a runaway data-center buildout, and a market where three companies control the most powerful models are different problems. They require different tools. The connective tissue is accountability.
The exposed people usually do not control the model architecture, training data, deployment defaults, cloud infrastructure, safety budget, lobbying strategy, or enforcement regime. They are downstream of decisions made elsewhere.
That makes the AI risk debate more concrete. The useful fight is about who has to prove safety, who gets audited, who carries liability, who can slow deployment, and who gets a say when systems affect public life.
The unresolved issue is whether governments can build those enforcement systems quickly enough without turning the largest AI companies into permanent gatekeepers. Strong rules can protect the public. Poorly designed rules can entrench the only companies rich enough to comply.
AI governance now has to solve both problems at once: reduce the risks experts are warning about, and keep the solution from concentrating even more power in the hands of the actors the report says are already responsible.
