OpenAI’s release of hundreds of mathematical papers sent shock waves through academia. But an examination of AI-assisted scientific research reveals something more important: which discoveries survived experimental testing, how often the machines succeed, and why AI still struggles with real scientific reasoning.

On October 6, 2026, OpenAI released 722 AI-generated mathematical manuscripts on GitHub, organized into 372 families of related results. The collection included claimed breakthroughs involving the Unique Games Conjecture, the Riemann zeta function, computational complexity, matrix multiplication, and several other difficult mathematical problems. Theoretical computer scientist Scott Aaronson called the event “The Mathocalypse.”

Some mathematicians celebrated the achievement. Others were deeply concerned about the consequences for their careers, the reliability of the results, and the future of human mathematical research. But I think the reaction raises a more important question than whether artificial intelligence can produce mathematical proofs. Has AI actually advanced science, or has it simply become extraordinarily good at producing scientific-looking output?

I’ve spent much of the past year working with AI-assisted research systems to investigate the Hubble tension, cosmological expansion, coupled differential equations, and perturbation theory. Our results have been decidedly mixed. We’ve produced useful mathematical models, explored alternative physical theories, and eliminated several hypotheses. We’ve also encountered impressive-looking equations that failed basic physical consistency tests. This experience made me skeptical of headlines declaring that mathematicians, physicists, and other scientists were about to become obsolete.

To investigate the question more objectively, I examined published AI-assisted discoveries in biology, chemistry, medicine, materials science, computer science, and mathematics. I concentrated on whether the work produced results beyond AI-generated papers: laboratory confirmations, useful algorithms, successful experiments, and advances that other researchers could reproduce.

The Mathocalypse: What OpenAI Actually Accomplished

OpenAI’s October release is undoubtedly an important event in mathematics. According to its published methodology, an unreleased internal model was presented with approximately 4,000 mathematical problems. The company eventually published 372 families of results, comprising 722 manuscripts. After the initial publication, three manuscripts were withdrawn because of a sign error that invalidated a central argument and two dependent results.

Fourteen additional manuscripts required revisions involving mathematical reasoning, assumptions, references, or dependencies. The public repository subsequently listed 719 manuscripts. OpenAI also reported that approximately 42% of the collection had corresponding Lean formalizations. Lean is a formal proof assistant that can mechanically verify mathematical statements and their derivations. This is a substantial advance over simply asking one language model to judge another model’s work.

However, formal verification is not the same as independent human understanding, and the availability of a formalized result does not mean that every assertion in the accompanying manuscript has been independently established. There is also a revealing statistical question. OpenAI began with approximately 4,000 problems and published 372 families of results. That produces a crude ratio of:

372 ÷ 4,000 = 9.3%.

This isn’t a scientifically defensible success rate. The units differ, some prompts may generate multiple related results, and publication selection depends on significance as well as correctness. More importantly, the denominator tells us little about how many unpublished attempts were mathematically plausible, genuinely correct, or later found to be defective. The release is impressive, but the number of manuscripts alone cannot establish the reliability of AI-generated mathematics. Aaronson also made an important observation in his October 9 update: he didn’t know of direct practical applications for the new mathematical results at that time.

AI Has Produced Genuine Scientific Discoveries

The answer to whether AI has made real contributions to science is unambiguously yes. The strongest evidence comes from projects in which computers generated predictions or designs that were subsequently subjected to physical experiments. Several are especially instructive.

AI Discovers New Antibiotic Candidates

In 2020, researchers at MIT and collaborating institutions published a study in Cell describing a deep-learning system trained to identify compounds with antibacterial properties. The system identified a compound that researchers named halicin. Halicin was not a newly invented molecule. It was an existing compound previously studied for another purpose. The important discovery was its antibacterial activity. Laboratory testing confirmed that halicin could kill several difficult bacterial pathogens, including some antibiotic-resistant strains. It also demonstrated efficacy in mouse infection models. The research team subsequently screened more than 107 million molecules computationally and selected 23 predictions for experimental testing. Of those 23 experimentally tested candidates, eight demonstrated antibacterial activity and structural differences from conventional antibiotics. That provides an observable success rate of:

8 successful candidates / 23 tested = 34.8%.

The AI didn’t merely generate a hypothesis that sounded reasonable. It narrowed a vast chemical search space to a small set of candidates, more than one-third of which produced useful laboratory findings. In 2023, another study published in Nature Chemical Biology used deep learning to identify abaucin, a compound with activity against Acinetobacter baumannii, a particularly difficult antibiotic-resistant pathogen. The researchers demonstrated antibacterial activity in laboratory testing and infection control in a mouse wound model. .

Neither compound should be mistaken for an approved clinical antibiotic. Discovering an antibacterial lead is only an early step toward determining whether a drug can safely treat human patients. But, the AI-assisted discoveries were genuine.

An Autonomous Laboratory Makes Real Materials

In 2023, researchers reported an autonomous laboratory known as A-Lab, which combined computational predictions, machine learning, robotic equipment, and experimental characterization to synthesize inorganic materials. The original publication described 41 successful syntheses out of 58 targets, suggesting a success rate of approximately 71%. The laboratory operated for 17 days, performing hundreds of experiments with relatively little human intervention. This sounds like an extraordinary success for autonomous science. But, the paper attracted substantial criticism. Independent researchers questioned whether some of the supposedly new materials were actually new, and whether the experimental characterization sufficiently established their identities. In January 2026, Nature published an author correction. After further review, the corrected study identified 36 successfully synthesized targets out of 57, reducing the supported success rate to approximately 63.2%. Four additional reported successes were judged inconclusive from the available X-ray diffraction evidence. The researchers also clarified that the materials were new to the laboratory’s prediction platform, not necessarily new to science.

The system demonstrated that AI-guided robotics could successfully synthesize many target materials. But it didn’t demonstrate the discovery of 36 entirely new substances unknown to science. A 63% synthesis success rate for a deliberately selected set of targets is still remarkable experimental progress. But it provides an excellent example of why successful execution, genuine novelty, and scientific discovery must be evaluated separately.

AI Designs Proteins That Actually Work

For some reason, AI is really good at protein design, and it provides some of the strongest evidence of AI’s practical scientific value. AlphaFold demonstrated that machine learning could predict protein structures with unprecedented accuracy for many types of proteins. More recently, AI systems have moved beyond predicting existing structures to designing entirely new proteins. Some of these proteins can bind to specific molecular targets, potentially supporting diagnostic tests, biological research, and future therapeutics. A 2026 review by Kosonocky and colleagues examined experimentally validated AI protein-design methods and compared published success rates.

For one AlphaProteo target, BHRF1, researchers reported successful binders for 83 of 94 experimentally tested designs. That’s a success rate of approximately 88.3%. For another target, tumor necrosis factor alpha (TNF-α), the same system produced zero successful binders among 54 tested designs. Dramatically different results from the same general research program. A separate system, BindCraft, demonstrated experimentally validated success rates ranging from 10% to 100% across different protein targets in a 2025 Nature paper. So, there’s is no single meaningful AI protein-design success rate (that I found anyway). Some problems have become relatively tractable, while others remain extremely difficult. Our physics work seems to fall into the latter category.

The 2026 review also warned that success rates across systems cannot always be compared directly because the researchers used different candidate-selection procedures and experimental criteria.

AI Has Even Reached Human Drug Trials

One of the more important developments occurred in 2025 when researchers published results of a randomized Phase 2a clinical trial involving rentosertib, a compound developed using generative AI and designed to inhibit a biological target associated with idiopathic pulmonary fibrosis. The study enrolled 71 patients across four treatment groups, including placebo. The researchers also observed encouraging changes in lung function at the highest tested dose. Mean forced vital capacity increased approximately 98 milliliters in that group, compared with a decline of approximately 20 milliliters in the placebo group. These findings were encouraging but preliminary.

The trial was small, lasted only 12 weeks, and experienced participant withdrawals. Larger trials are necessary to determine whether the drug provides a clinically meaningful, durable benefit. AI contributed to identifying a therapeutic target and designing a drug candidate that progressed into controlled human testing. But it’s not yet proof that AI can reliably discover successful commercial medications. A 2026 review in Nature Reviews Drug Discovery cautioned that AI’s contributions are often difficult to separate from conventional medicinal chemistry, particularly when subsequent optimization steps are incompletely documented.

AI Has Produced Working Computer Algorithms

Not so remarkably, AI is even stronger when we examine computer science. In 2023, Google DeepMind published research describing AlphaDev, a reinforcement-learning system that discovered improved sorting algorithms. The researchers demonstrated correctness and measured performance on actual computer processors. Some of the resulting sorting routines were incorporated into the widely used LLVM C++ standard library. Depending on the input and hardware, the research reported performance improvements as large as 70% for particular small sorting workloads. Improvements for large sequences were much smaller, around 1.7% in the reported examples. .

A 70% improvement in one specialized operation doesn’t mean that all programs become 70% faster, but the system identified a genuine optimization, and the improvement was sufficiently useful to be incorporated into production software. Google’s AlphaEvolve system provides a more recent industrial example. In 2025, Google reported that AlphaEvolve discovered improvements in the scheduling of data-center workloads, recovering an average of approximately 0.7% of its global computing resources. The company also reported improvements to a Gemini training kernel.

These are operational engineering results, although the detailed performance claims come primarily from Google’s own reporting rather than fully independent public audits. This shows that AI-generated algorithms can be tested against clear performance criteria, rejected when they fail, and deployed when they outperform existing implementations.

AI-Generated Scientific Hypotheses Are Beginning to Survive Laboratory Testing

In May 2026, a Google-led research team published Accelerating Scientific Discovery with Co-Scientist in Nature. The system used multiple AI agents to generate, evaluate, criticize, and refine scientific hypotheses. Our own experience has shown this isn’t one of the subjects that AI is good at. The researchers investigated three biomedical applications: drug repurposing for leukemia, biological targets associated with liver fibrosis, and mechanisms of antimicrobial resistance. They reported experimental support for the AI-generated hypotheses. In the antimicrobial-resistance case, the system independently reconstructed an explanation that human researchers had developed but not yet published. That could be a useful demonstration of scientific reasoning, although independently recovering a previously discovered mechanism is not equivalent to establishing an entirely new discovery.

The project also illustrates a limitation in published AI-science evaluations. The authors evaluated hypothesis generation across a much larger group of research goals, but only a selected subset underwent the reported wet-laboratory validation. We therefore can’t divide three successful application areas by the number of general research goals and call the result an overall scientific discovery rate.

So What Percentage of AI Science Projects Actually Succeed?

This was the most difficult question in my analysis. Despite substantial literature on AI-assisted scientific discovery, I couldn’t identify a comprehensive, reliable registry containing all attempted AI science projects and their ultimate outcomes. So, we don’t currently have a defensible industry-wide success percentage. There are several reasons.

Successful projects are more likely to be published and publicized. Failed projects may never leave private laboratories (yeah, I’ve been there). Companies have commercial reasons not to disclose every unsuccessful experiment. Academic projects may stop because of funding or personnel changes rather than mathematical or scientific failure. Even when we say the word success, it doesn’t mean 100% success. A successful simulation doesn’t really count until its value is shown with real field data (in my opinion). A plausible hypothesis, published paper, synthesized compound? No, these are beginning steps.

What the published numbers actually show

Project or evaluation	Measured outcome	Result
MIT antibiotic discovery	Active compounds among 23 selected candidates tested	34.8%
A-Lab, corrected 2026	Successfully synthesized targets	36/57, 63.2%
AlphaProteo: BHRF1	Experimental protein binders	83/94, 88.3%
AlphaProteo: TNF-α	Experimental protein binders	0/54, 0%
ScienceAgentBench, 2025	Scientific programming tasks completed by best agent, within three attempts	32.4%
OpenAI mathematics, 2026	Published result families relative to approximately 4,000 posed problems	About 9.3%, not a validated success rate

These percentages cannot legitimately be averaged, but a consistent pattern emerges. AI performs best when a problem has a well-defined objective, abundant relevant data, and an external procedure that can quickly determine whether a proposed solution works. Problems become substantially more difficult when the goal is to infer a new physical mechanism from incomplete observations and several competing mathematical descriptions. That’s pretty much where our own research has encountered trouble.

An Independent Benchmark Shows How Far Scientific Agents Still Have to Go

In 2025, researchers published ScienceAgentBench, an evaluation designed to measure the ability of language-model agents to carry out authentic scientific computing tasks. The benchmark included 102 tasks extracted from 44 peer-reviewed scientific publications across four disciplines. Instead of asking models general scientific questions, researchers required them to produce working Python programs and evaluated the results which is a lot of what we do. The strongest tested agent, with three opportunities per task, solved 32.4% of the tasks independently. Providing additional expert knowledge increased that figure only modestly, to 34.3%. That’s not an overall scientific discovery rate, and stronger or newer systems may perform differently.

Completing an entire research program requires a lot more than getting individual scripts to execute correctly. Each script has a probability of failing depending on the capability of the agent, and those probabilities compound for large projects. A successful scientific investigation must select meaningful assumptions, understand uncertainty, distinguish competing hypotheses, interpret results, and recognize when its original mathematical model is inadequate. Those requirements help explain the gap between the remarkable performance of specialized AI systems and the uneven reliability of more general scientific research agents.

A Cautionary Example: The Famous 44% Productivity Study Was Withdrawn

During this investigation, I encountered a study that initially appeared to provide exactly the statistical evidence I wanted. A 2024 working paper titled Artificial Intelligence, Scientific Discovery, and Product Innovation claimed to examine AI adoption among 1,018 scientists in a corporate research laboratory. The reported findings were that AI-assisted researchers supposedly discovered 44% more materials, generated 39% more patent filings, and achieved a 17% increase in downstream product innovation. Those numbers circulated widely in discussions of AI’s scientific impact. But there was a big problem. In May 2025, MIT publicly announced that it had no confidence in the provenance, reliability, or validity of the study’s data. The institution requested the paper’s withdrawal, and the preprint was subsequently marked withdrawn.

MIT’s public statement specifically warned that the results should not be relied upon. The 44%, 39%, and 17% figures therefore cannot be used as reliable evidence of AI’s scientific productivity. This is one of the most revealing findings from my literature search. A paper claiming extraordinary productivity improvements from AI became influential before its supporting data could be trusted.

Our Own AI Research Provides an Interesting Comparison

Our Common Deformation Origin cosmology project has investigated whether alternative dynamical descriptions might account for discrepancies in cosmological expansion measurements. We’ve used AI-assisted software development, numerical optimization, differential-equation solvers, observational datasets, and mathematical analysis to construct and test possible models. The project has generated useful mathematical results, but it’s also repeatedly demonstrated that a mathematically attractive answer rarely leads to a valid physical theory once we start digging around using perturbation theory and adding in different datasets.

First: Fitting Data Is Not the Same as Explaining It

We investigated flexible mathematical descriptions capable of fitting reconstructed expansion histories, some produced remarkably small numerical residuals. But improving the fit didn’t reliably produce better predictive behavior on independent observations. Several candidate models developed excessive freedom in their parameters, allowing them to reproduce the calibration data while providing a poor physical explanation. In later tests, hybrid formulations produced better training results but worse full-domain or withheld-data results.

What made the experience particularly interesting was how easily an AI-assisted research system could produce increasingly sophisticated mathematical constructions that reduced a numerical objective without addressing the underlying scientific problem.

Second: Automated Software Tests Can Pass While the Scientific Model Fails

Several phases of our project completed their required programming work and passed substantial test suites. In one phase, 43 automated tests passed, but the scientific evaluation still identified parameter degeneracies, inadequate predictive behavior, and models that failed their acceptance criteria. We’ve increasingly separated implementation acceptance from scientific acceptance so that a passing test suite cannot be mistaken for validation of a physical theory.

Third: The Einstein Constraint Failure

Our recent cosmological perturbation research provides an even stronger example. We developed a coupled scalar perturbation model and investigated its behavior in the early universe. The calculations involved radiation components, scalar-field perturbations, metric perturbations, and Einstein constraint equations. The numerical machinery produced deceptively likable solutions, but independent verification revealed that the proposed primordial initialization was not consistent with the Einstein constraints. A further audit identified problems involving the assumed time or clock mapping and the order of the initial-condition expansion. At the beginning of our test domain, the required constraint relationships did not close consistently. The resulting research verdict was that the Einstein constraints were inconsistent, and we stopped the proposed progression.

Conclusion: Real Progress, Unknown Overall Success Rate

After examining the available evidence, my conclusions are fairly straightforward. AI-assisted research has already produced genuine advances in biology, chemistry, medicine, materials science, and computing. Some advances are experimental discoveries. Others are useful engineering improvements. A smaller number have progressed into human clinical evaluation. The evidence is too strong to dismiss AI science as merely producing sophisticated-looking papers. At the same time, we do not have reliable data to calculate the percentage of all AI science projects that produce meaningful scientific progress. The published success rates describe particular experiments, candidate sets, and benchmark tasks, not the entire population of AI research programs. Anyone claiming that AI scientific research has an overall success rate of 50%, 80%, or 90% should be able to produce the complete project denominator, define success, and account for abandoned and unsuccessful work.

The Mathocalypse is real in one limited sense, the traditional scarcity of sophisticated mathematical problem-solving is disappearing.

Sources and Further Reading

1. OpenAI: Sharing AI Progress in Mathematics, October 2026 and GitHub mathematics release and correction history.
2. Scott Aaronson: The Mathocalypse, October 2026. Firsthand discussion of the release, its implications, and the reactions of mathematical researchers.
3. Stokes et al., A Deep Learning Approach to Antibiotic Discovery, Cell, 2020. Halicin discovery, experimental antibacterial activity, and 8 successful candidates from 23 tested predictions.
4. Liu et al., Deep Learning-Guided Discovery of an Antibiotic Targeting Acinetobacter baumannii, Nature Chemical Biology, 2023. Abaucin discovery and experimental testing.
5. Szymanski et al., An Autonomous Laboratory for the Accelerated Synthesis of Inorganic Materials, Nature, 2023 and 2026 author correction. Corrected synthesis success rate and clarification of claims of novelty.
6. Jumper et al., Highly Accurate Protein Structure Prediction with AlphaFold, Nature, 2021. Experimental evaluation of machine-learning protein structure predictions.
7. Kosonocky et al., Closing the Loop: Experimentally Validated Methods in AI-Driven Protein Design, Current Opinion in Structural Biology, 2026. Comparative review of real experimental protein-design outcomes.
8. Pacesa et al., One-Shot Design of Functional Protein Binders with BindCraft, Nature, 2025. Functional protein-binder generation and laboratory validation.
9. Xu et al., A Generative AI-Discovered TNIK Inhibitor for Idiopathic Pulmonary Fibrosis: A Randomized Phase 2a Trial, Nature Medicine, 2025. Human clinical testing of rentosertib.
10. Mankowitz et al., Faster Sorting Algorithms Discovered Using Deep Reinforcement Learning, Nature, 2023. AlphaDev algorithms and integration into LLVM.
11. Google Cloud: AlphaEvolve on Google Cloud, December 2025. Google’s reported operational performance gains.
12. Gottweis et al., Accelerating Scientific Discovery with Co-Scientist, Nature, 2026. AI-generated hypotheses and experimental validation across three biomedical applications.
13. Chen et al., ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery, ICLR, 2025. Evaluation of 102 authentic scientific programming tasks.
14. MIT Economics: Assuring an Accurate Research Record, May 2025. MIT’s statement rejecting the reliability of the withdrawn AI scientific-productivity study.
15. Artificial Intelligence in Drug Discovery: What It Is, Where We Stand and the Path Forward, Nature Reviews Drug Discovery, 2026. Assessment of AI drug-development progress and limitations in attributing success.

Research completed October 22, 2026. Public results are distinguished from internal, unpublished cosmological research. Reported experimental success percentages are conditional on the tested candidates and are not estimates of the overall success rate of AI-assisted science.