AI for science: telling a new capability from a promising demo

Some of the clearest evidence that AI can change how science is done comes from biology. Predicting a protein's 3D shape from its sequence was a long-standing problem. AI systems were tested on it in a blind, independent competition, and the predictions were then used and checked by labs worldwide. The work was recognised with the 2024 Nobel Prize in Chemistry.

Not every announcement is like that. Here is how to tell the difference.

Signs of a confirmed advance

• Blind or independent testing. The method was evaluated on problems its developers could not see in advance, by people who did not build it.
• Experimental confirmation. A predicted molecule, material, or structure was made or measured in a lab.
• Use by others. Researchers outside the original team adopt the method and report results.
• Peer-reviewed publication with enough detail for others to check.

Signs of a promising demo

• Results are reported only by the developer, often in a blog post or preprint.
• The system "proposes" or "generates" candidates, but few have been tested.
• Performance is shown on a benchmark rather than on an open scientific question.
• Words like "discovered" are used for candidates that have not been verified.

A promising demo can become a confirmed advance. The point is to know which stage a story describes.

Weather and climate

AI weather models trained on decades of historical data have, in peer-reviewed studies, matched or outperformed leading traditional forecasting systems on several measures for medium-range forecasts. National weather services are testing and adopting them alongside physics-based models. Extreme and rare events remain an area to watch closely, because they are scarce in training data.

How we cover science

We say what kind of evidence supports a claim, and whether anyone outside the team has checked it.