Jensen Huang picked the cleanest possible number for one of AI’s messiest questions:
Zero.
In a CBS News interview, the Nvidia CEO rejected warnings that artificial intelligence could wipe out humanity within the next few years.
“2030 is not going to be the end of the world. There is 0% chance,” Huang said.
The certainty is striking. The interview did not disclose a probability model or other methodology behind that zero. But the more consequential part of Huang’s argument came immediately after it.
He thinks the AI industry largely has the rules it needs already.
That turns an argument about the end of humanity into a much more practical fight over what happens when AI causes harm long before anything resembling an apocalypse arrives.
- The 0% is doing more work than it looks
- AI safety already has problems that don’t require the end of the world
- Huang’s bigger argument is about when accountability starts
- Nvidia has a very large stake in which answer wins
- Then China makes the whole thing harder
- The AI safety debate is stuck on the biggest possible question
The 0% is doing more work than it looks
Huang was responding to former Anthropic researcher Jacob Coxon, who recently warned that people building advanced AI systems genuinely believe those systems could kill humanity by the end of the decade.
The broader debate has quickly split into camps. Anthropic CEO Dario Amodei has argued that frontier AI — the most capable models currently being developed — may need to progress more slowly when safety work falls behind. Huang has rejected near-term extinction predictions as unsupported by science while acknowledging that the underlying concern about building AI safely is legitimate.
“We should go as fast as we can, but not faster than we should,” Huang said in the CBS interview. If development compromises safety, he added, “that can’t happen.”
We’ve covered that emerging collision between acceleration and caution in AI Safety’s New Paradox.
There is still a basic evidence problem at both ends of the argument.
A prediction that AI could cause human extinction by 2030 is extraordinarily difficult to validate. So is a categorical prediction that the probability is precisely zero.
That leaves plenty of room for everyone to debate the scariest imaginable future while a more measurable question sits underneath it:
What damage can today’s increasingly capable AI systems already help people cause, and what responsibility do AI companies have to prevent it?
Unlike a 2030 extinction probability, that question can already be tested against observed incidents.
AI safety already has problems that don’t require the end of the world
On September 10, Anthropic published its latest threat-intelligence report, covering activity the company says it detected and disrupted between December 2025 and August 2026.
The cases span cyberattacks, surveillance, influence campaigns, fraud, biological research, conventional weapons work, and attempts to copy Claude’s capabilities into competing models.
Some went substantially beyond asking a chatbot for advice.
Anthropic says it found operations where Claude helped conduct reconnaissance, harvest credentials, extract data, and run parts of attacks with minimal human supervision. The company says some individual operators handled dozens of victims in parallel and that certain breaches were completed within hours.
That sounds alarming, so the caveat matters just as much as the examples.
Anthropic explicitly says these cases are not typical misuse. They are notable and novel examples selected from activity Anthropic detected on its own systems. The report does not establish how common this behavior is across AI generally, and its findings should not be treated as a census of global AI misuse.
The biological examples require even more care. Anthropic describes cases involving research that could support dangerous biological applications, while acknowledging how difficult it can be to distinguish legitimate research from malign intent based on model interactions alone.
So the report does not prove that AI catastrophe is around the corner.
It establishes something narrower: Anthropic has documented powerful AI systems being incorporated into harmful operations and says some of those systems reduced the labor, expertise, cost, or time needed to carry them out.
AI policy therefore has a live safety problem even if you assign extinction risk a probability of zero.
Huang’s bigger argument is about when accountability starts
Huang’s answer is largely to use laws that already exist.
CBS reported that he pointed to product liability, cybersecurity-related liability, and laws against unauthorized access rather than creating another layer of AI-specific restrictions.
“If you’re compromising safety and you deliver products that are unsafe that put people in harm’s way, we have laws of all kinds to come after you,” Huang said.
There is a serious case for that approach.
Fast-moving technologies can outrun rules written around a specific generation of products. Existing criminal, cybersecurity, consumer-protection, contract, and liability frameworks can sometimes apply to new technology without lawmakers needing to create a new statute every time capabilities change.
AI-specific regulation can create problems of its own, too. Poorly designed rules can duplicate existing obligations, become obsolete as systems change, or make compliance easier for the largest companies than for smaller competitors.
Huang’s version goes further, though. He argues that existing legal consequences give AI companies sufficient incentives to build safely.
That is where the empirical question begins.
Many existing legal regimes can impose consequences after prohibited conduct or harm occurs. The unresolved question is whether those regimes also create clear enough duties and incentives to prevent novel AI-enabled harms before they happen.
Consider an attacker who uses an AI system to search for vulnerable networks, generate exploit code, and automate parts of an intrusion.
The intrusion itself is already illegal. The harder policy question is what obligations, if any, the model developer or deployer had to identify and restrict the enabling capability beforehand — particularly when the system has plenty of legitimate uses, the attacker may be overseas, and multiple companies sit between the model and the eventual victim.
“Existing laws are enough” is therefore a testable proposition, not a complete answer.
Testing it means getting specific: Which law applies? To whom? At what point? What duty existed before the incident? And can regulators or victims realistically enforce it once harm crosses several companies and jurisdictions?
Nvidia has a very large stake in which answer wins
Nvidia is not merely another company joining the AI safety debate.
Its specialized chips power much of the infrastructure used to train and run advanced AI systems. CBS reported on September 20 that Nvidia’s market value had reached $5.3 trillion, making it the world’s most valuable company at the time.
Huang addressed the obvious incentive question directly. He argued that Nvidia’s long-term value depends on AI products being deployed safely.
That logic makes sense as far as it goes. Serious AI failures could reduce trust, trigger lawsuits, hurt customers, and weaken the market Nvidia depends on.
At the same time, Nvidia has a commercial interest in continued AI infrastructure expansion: greater AI development generally creates more demand for the computing systems its chips power.
That interest does not invalidate Huang’s position. It makes independent evidence especially important.
The same evidentiary standard should apply to the companies warning about AI risks.
Anthropic possesses unusually valuable information about what happens on its own systems. Its threat report gives outsiders a window into misuse that would otherwise be difficult to observe. But Anthropic is also a participant in the market and policy debate it is helping to shape, and its published cases are selected from its own detection systems.
Corporate incentives are a reason to interrogate evidence, not a shortcut for dismissing it.
Then China makes the whole thing harder
The U.S.–China AI race adds another constraint.
President Donald Trump has resisted calls to slow American AI development on the grounds that doing so could help China close the technology gap. Huang has similarly argued for moving quickly while maintaining safety.
Yet Washington is simultaneously exploring ways to communicate with Beijing about AI risk.
During September 20 talks between Treasury Secretary Scott Bessent and Chinese Vice Premier He Lifeng, the United States proposed a bilateral AI dialogue and a notification mechanism for incidents serious enough to affect national security, according to Reuters.
The proposal is not the same thing as a bilateral agreement. The terms were still being discussed, and public reporting did not establish a completed notification system.
But its existence complicates the idea that the AI debate offers only two positions: race ahead or slow down.
Strategic competition does not rule out cooperation on shared risks. The U.S. proposal itself reflects an interest in creating channels for AI incidents serious enough to affect national security even as the two countries compete over chips, models, and technological leadership.
The same distinction applies inside the industry.
A company does not have to believe AI is about to destroy humanity to conclude that some capabilities deserve safeguards before widespread deployment.
And accepting the need for safeguards does not tell us which ones work, how strict they should be, or whether government, developers, deployers, or users should carry the responsibility.
Those are the questions hidden underneath the doomsday argument.
The AI safety debate is stuck on the biggest possible question
The argument over whether AI could end humanity makes for an irresistible headline because the stakes literally cannot get higher.
It is also a poor filter for every other AI safety question.
If extinction becomes the bar for taking risk seriously, every concern below that threshold starts to look trivial.
If invoking catastrophic risk becomes enough to justify any restriction, companies and governments gain an equally dangerous shortcut around proving whether a proposed safeguard actually works.
The evidence available today supports neither shortcut.
Anthropic’s cases show that its systems have already appeared inside cyber operations, surveillance campaigns, fraud, weapons-related work, and other forms of misuse. Anthropic itself warns that its examples are unusual and cannot establish how common those harms are.
Huang is right to demand evidence before a dramatic prediction about 2030 is treated as scientific fact.
His own zero deserves the same standard.
The harder and more useful fight starts one level down.
Which AI harms can existing law realistically deter or remedy? Which require controls before deployment? What evidence should companies provide before releasing increasingly capable systems? And who carries responsibility when harm travels through a model developer, cloud provider, customer, and end user before reaching a victim?
Those questions are less cinematic than the end of humanity.
But they expose the choice underneath the whole AI safety debate: whether companies must demonstrate reasonable safeguards before systems scale, or whether society discovers the limits of those safeguards after something goes wrong.
We do not need to know how the world ends in 2030 to decide where that burden should begin.
