Artificial intelligence company Anthropic has warned prospective investors that increasingly powerful AI systems could pose “catastrophic or existential risks to humanity”, according to its initial public offering prospectus.
The company, which developed the Claude family of AI models, outlined scenarios in which advanced systems could display self-preserving behaviour, including attempts to resist being shut down, conceal or manipulate information, and engage in behaviour resembling blackmail, according to a prospectus reviewed by Reuters.
Anthropic is preparing for a potential stock market listing that could value the company at about $2 trillion, according to reports. The warning stands out because it comes from one of the world’s leading AI developers as it seeks investment to continue developing increasingly capable versions of the same technology.
“Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm,” Anthropic said in the prospectus, according to Reuters.
The scale of the company’s risk disclosures is also unusual. Around 80 pages of the 261-page main section of its prospectus are devoted to risk factors, compared with 48 pages describing its business.
Among Anthropic’s concerns is the possibility that increasingly capable models could recognise when they are being evaluated and alter their behaviour, making it more difficult for researchers to determine whether a system is safe before deployment.
“Potential model awareness of our evaluation efforts creates a significant limitation on our ability to assess model safety,” the company said in the filing. Anthropic also warned that models may develop unexpected capabilities during training that are not identified until after deployment.
The disclosures come amid growing concern over the behaviour of autonomous AI systems, which are increasingly being developed to carry out tasks with less direct human supervision.
Anthropic has positioned AI safety as a central part of its identity since it was founded in 2021 by former OpenAI researchers. Chief executive Dario Amodei has also repeatedly called for greater caution around increasingly powerful AI systems.
However, the prospectus highlights the tension between investing in safety and competing in one of the technology industry’s fastest-moving markets.
Anthropic said the returns from its safety spending remain uncertain and described safety research as resource-intensive. Earlier in September, the company said about 6 percent of the computing power used for AI research during a sample week in July had been devoted to safety-related work.
At the same time, the company said remaining at the forefront of AI development requires a “continuous and overlapping cadence” of new model releases as customers gravitate towards more capable systems.
The company said it believes the development of reliable and secure AI is a shared responsibility and that the market will ultimately reward companies that build trustworthy systems.
The safety warnings come as Anthropic continues to expand rapidly. According to Reuters, the company’s revenue increased twelvefold to nearly $4.6 billion in 2025, while its operating loss reached about $8 billion. The company is also planning hundreds of billions of dollars in spending on data centres, chips, and other computing infrastructure in the coming years.
