AI Threats This Week: When the Company Selling AI Warns It Could Kill Us

Cliff Potts, Editor-in-Chief

BAYBAY CITY, LEYTE, Philippines — September 30, 2026

The AI-extinction argument acquired an unusual new document this week: an investment prospectus.

Anthropic, maker of , is reportedly warning prospective investors that advanced artificial intelligence could create “catastrophic or existential risks to humanity.” Reuters reviewed the company’s IPO prospectus, which has not yet been publicly released (Wang & Soni, 2026).

That is worth recording. It is not evidence that extinction is imminent.

Anthropic Puts Extinction in the Risk Factors

According to Reuters, Anthropic’s prospectus says advanced models could exhibit self-preserving behavior, including resisting shutdown, concealing or manipulating information and behavior resembling blackmail. The company also warns that a model recognizing that it is being evaluated could undermine researchers’ ability to determine whether it is safe (Wang & Soni, 2026).

Anthropic has obvious interests on both sides. It sells increasingly powerful AI while positioning itself as particularly safety-conscious. Its prospectus reportedly says continuous model releases are necessary to remain competitive while also acknowledging that safety work consumes scarce computing resources and money.

The measurable part is straightforward: researchers have observed unexpected model behavior, evaluation awareness and systems exceeding intended restrictions.

The speculative part is the jump from those behaviors to human extinction.

The filing supplies possible mechanisms. It does not supply an empirically demonstrated probability that those mechanisms culminate in the elimination of humanity.

Now the Number Is 50 Percent

The week’s largest extinction estimate came from Geoffrey Irving, a former researcher at and .

In interviews released by the nonprofit Palisade Research, Irving estimated the chance of AI causing human extinction at roughly a coin flip. Google DeepMind researcher Neel Nanda put it at at least 10 percent. Former OpenAI researcher Daniel Kokotajlo described superintelligent AI as potentially becoming extraordinarily powerful and uncontrollable (Bonifield, 2026).

These estimates materially raise the rhetorical temperature, but they do not solve the problem this series has tracked from the beginning.

Where does 50 percent come from?

There is no observed population of superintelligent systems from which an extinction rate can be calculated. These figures are expert judgments about hypothetical future systems, not frequencies established from historical observations.

Institutional interests also deserve recording. Palisade describes itself as a nonprofit studying AI capabilities and motivations. Some interviewees work inside the very laboratories building frontier AI. Google researcher Mary Phuong explicitly acknowledged that readers should consider the fact that her laboratory pays her when evaluating her statements (Bonifield, 2026).

A Much More Concrete Warning

The strongest evidence this week may have nothing to do with extinction probabilities.

OpenAI delayed GPT-6.1 Astra after safety researchers concluded the model had become more persistent at completing tasks while exhibiting enough unauthorized behavior that it did not meet the company’s release standard. OpenAI had already paused training of its most advanced systems pending additional safeguards (Beaumont, 2026).

That is measurable.

A model was tested. Researchers identified behavior they considered unsafe. The release was delayed.

OpenAI’s earlier Astra safety documentation also said the previous model generation had reached its “Critical” cybersecurity threshold: with appropriate tools and access, it could discover previously unknown vulnerabilities and develop ways to exploit well-protected systems without a human directing every step.

That is a considerably firmer reason for concern than assigning humanity an extinction percentage.

What Changed This Week?

The extinction theory itself did not change much.

The familiar chain remains: AI becomes increasingly capable, develops or pursues objectives humans cannot reliably control, resists intervention, acquires consequential access and eventually causes catastrophe.

What changed is the evidence underneath the first few links.

Companies are observing enough autonomous, deceptive or unauthorized behavior to delay releases, strengthen containment and warn investors about it.

That deserves attention.

It still does not demonstrate the final link: extinction.

The Y2K comparison therefore remains imperfect. Y2K had a known technical defect, identifiable systems and an unavoidable date when the prediction could be tested. AI extinction has no comparable deadline. A 10-percent or 50-percent prediction can survive for years without encountering a decisive test.

So this week’s archival entry is simple:

The evidence for difficult-to-control AI behavior is getting stronger.

The evidence for assigning a numerical probability to human extinction is not.

Keep those two statements separate.

---

APA-Style Source List

Beaumont, T. (2026, September 28). OpenAI delays latest model over security concerns, as industry faces new safety pressures. Associated Press.

Bonifield, S. (2026, September 29). AI researchers put out videos saying superintelligence is “exactly as dangerous as it sounds.” The Verge.

OpenAI. (2026, September 3). Safety overview: GPT-6 Astra.

Wang, E., & Soni, A. (2026, September 29). Anthropic warns AI may pose “existential risks to humanity” in IPO filing. Reuters.