Anthropic proves safety of more than 100 public software libraries with Theorem and formal verification

Anthropic said it used Claude in collaboration with Theorem to prove safety properties at the RISC-V assembly code level for more than 100 real-world public software libraries. The company said AI can now produce formal proofs at scale. It also said the proof work uncovered bugs that had gone undetected for years.

If Anthropic’s claims are accurate, AI’s role is moving beyond finding bugs to incorporating proven safety into development processes. For developers focused on software safety and auditing, this could change how formal verification—until now reliant on human labor—is used. However, the work covered more than 100 public libraries, and whether the results can be generalized to all production software is a separate question. Applying the approach to commercial code or frequently updated infrastructure will require assessing how proofs are maintained, the scope of coverage and the computing resources needed.

The key point is Anthropic’s claim that AI has reached the stage of generating proofs at scale, not just reading code. The work covered more than 100 public libraries at the RISC-V assembly code level, shifting the question from whether this is a lab demonstration to whether safety can be mechanically established for low-level implementations. Still, the announcement demonstrates potential applicability; the text does not establish how large a codebase the same approach can handle or how much it reduces the labor and computing resources needed to produce proofs. A related article in Physical AI News, Airbus’ Greg Ombach discusses challenges in scaling Physical AI, examined the computing and data demands involved in scaling VLAs. Anthropic’s announcement concerns software, but it connects to the question of how to incorporate provability and safety guarantees—not just performance—when putting AI to work. In both Physical AI and software development, the idea of adding a verification layer rather than using model output as is appears to be gaining ground. The industry implications include the possibility that formal verification will be built into development workflows from the outset rather than used as a final quality-control step. Competition could then extend beyond model generation performance to include the ability to integrate proof generation, bug detection and checks for compatibility with existing code. For low-level implementations such as those using RISC-V, passing a proof could itself inform adoption decisions, making it important to balance development speed and safety. However, the announcement covered public libraries; its application to commercial software and large legacy systems, the reproducibility of proofs and the cost of maintaining them through updates remain uncertain. See the original source for details.