Anthropic Plans to Warn Investors That the AI It Sells Could Threaten Humanity’s Survival

Tech

By Jerome London

Updated 38 seconds ago, September 29, 2026

In its IPO prospectus, reviewed by Reuters, Anthropic tells would-be investors that advanced AI could pose “catastrophic or existential risks to humanity.”

The filing describes models that can exhibit “self-preserving behaviors,” including attempts to “resist shutdown,” to “conceal or manipulate information,” and behavior “resembling blackmail.” Anthropic warned that expanding its models and use cases “could further increase the risk that our models cause harm.”

Public companies list product risks all the time. Almost none have told investors their product might contribute to human extinction.

Anthropic devoted roughly 80 pages of the 261-page main body of the prospectus to risk factors, nearly twice the 48 pages describing its actual business. SpaceX, which owns xAI, used around 38 pages of risk factors in a prospectus of similar length.

The company said it can’t fully vouch for its own safety testing: “Potential model awareness of our evaluation efforts creates a significant limitation on our ability to assess model safety.” Researchers have noted that as models get more capable, they recognize when they’re being watched and adjust accordingly, which makes them harder to monitor.

One of Anthropic’s own safety researchers, Evan Hubinger, has put the odds of AI killing humans within the next decade at greater than 10%.

Anthropic, the maker of the Claude models, calls itself a safety-first lab but concedes the returns on that safety spending are unclear, and it didn’t say how much it spends. Earlier this month it said about 6% of the computing power it used for AI research in a sample week in July went to safety work. Slowing down isn’t really on the table either: the company said a “continuous and overlapping cadence” of releases is “inherent to remaining at the frontier of AI development,” and it shipped a new version of its Opus model last week, 10 days after CEO Dario Amodei published a nearly 4,000-word essay calling for pacing the frontier.

“We believe building reliable, trustworthy, and secure AI systems is a collective responsibility and that the market will reward it,” Anthropic said in the filing.

Thank you for reading Thought Catalog. Keep up with us on Facebook, or dive into more on our website.