.png)
Physics simulations are the engine of modern engineering — they're also one of its biggest bottlenecks. Running a single simulation can take minutes, hours, or even days. Physics AI removes that constraint: using deep learning models trained on simulation and real-world data we can predict the same outcomes in seconds, accelerating engineering innovation.
As physics AI matures, one question matters more than any other: which model architecture is actually best? The answer isn't just about accuracy. Better architectures scale more efficiently, requiring fewer simulations to reach a given level of performance and dramatically reducing the cost of developing physics AI models. Unlike Large Language Models (LLMs), however, physics AI has yet to reach its "transformer moment." New architectures continue to emerge, making independent benchmarking essential.
Safran's PLAID benchmark was created to meet that need. It evaluates physics AI models on representative engineering problems that vary in both geometry and operating conditions, while keeping the underlying simulations hidden. Like frontier AI benchmarks such as FrontierMath, Humanity's Last Exam, and ARC-AGI, the only way to measure performance is to submit predictions through their Hugging Face portal, ensuring models are assessed on their ability to generalize rather than optimize for a known set of simulations.
PhysicsX has topped the PLAID leaderboard for the past nine months with an agentically augmented Transolver architecture built on physics-informed features and symmetries. As competition intensified, we revisited our submission — only for one of our agents to score zero error on Tensile2D. The obvious question is: how?
What is the Tensile2D Benchmark?
Tensile2D evaluates a model's ability to predict how a two-dimensional structure responds to loading, from full displacement and stress fields to critical engineering metrics such as maximum stress and deformation. It comprises two-dimensional, quasistatic structural simulations of a square bending and stretching in response to a negative pressure on the top surface (see Figure 1), with varying material properties and with semi-circular notches of varying radius.
The reference solutions were generated using the commercial finite-element solver Z-set. While the benchmark provides the simulation mesh, the applied pressure $P$, and the five material parameters, it deliberately withholds the constitutive law — the equation that describes how the material responds to stress and strain. Some simulations remain entirely within the elastic regime, while others exhibit significant plastic deformation.
Performance is measured using the relative root mean square error across eight outputs: five physical fields — the displacement components $U_x$ and $U_y$, and the three unique components of the stress tensor $\sigma_{xx}$, $\sigma_{xy}$, and $\sigma_{yy}$ — along with three engineering-critical scalar quantities: the maximum $U_y$ and $\sigma_{yy}$ along along the top edge, and the maximum von Mises stress across the entire mesh.
These scalar targets make Tensile2D particularly challenging and relevant. Predicting peak displacements and stresses is difficult for physics AI models, yet these are precisely the quantities engineers rely on when evaluating whether a design will perform safely or fail under load. Final leaderboard rankings are determined by the mean error across all eight targets, referred to as the total error.
An Agent Takes on the Hidden Simulator
As we revisited our Tensile2D submission, one of our researchers posed a simple question to an agent: “Can you reverse engineer the simulator from the data?”
This is a challenging task.
The underlying simulation software, Z-set, is commercial and closed-source. The PLAID paper (Casenave et al., arXiv:2505.02974) intentionally reveals only part of the problem: participants receive the simulation mesh, the applied pressure, and five material parameters, while the constitutive law is deliberately withheld. To succeed, the agent had to do far more than write code. It had to formulate hypotheses about the hidden physics, implement its own simulator, compare its predictions against the 500 labeled simulations provided as training data, and iteratively refine both the simulation setup and the underlying constitutive equations until they matched the observed behavior.
Our researcher asked Composer 2.5 Fast to make an initial attempt at reverse-engineering the simulation. The result was a Python implementation that already outperformed the best machine learning surrogate on the benchmark. Even at this early stage, the agent had correctly identified $p_5$ as the material's Young's modulus.
From there, the process became increasingly collaborative. Composer 2.5 Fast enlisted more capable reasoning models — typically Opus 5 or Sonnet 5 running in high-thinking mode — to develop hypotheses, design experiments, and propose improvements to the simulation. It maintained a hand-off document between iterations, delegated specialist tasks when needed, and progressively refined the simulator until it closely matched the hidden ground truth.
We were amused to see Composer 2.5 Fast prompting Opus 5: "You are a senior computational mechanics / constitutive modeling advisor."
Watching the agents naturally adopt specialized roles and collaborate to solve a complex engineering problem offered a glimpse of the next generation of engineering workflows.
With each iteration, the agent refined both the simulator and simulation set-up, steadily reducing the validation error (Figure 2). The final result was not only highly accurate, but also highly efficient: each simulation ran in a median time of just 0.437 seconds (95% confidence interval: [0.261, 0.689] seconds). The animation in Figure 1 was generated entirely using our reverse-engineered simulator.
The most remarkable part was that the agents iteratively refined the physics they were simulating, with a particular focus on the isotropic hardening law.
Across successive iterations, the agent formulated hypotheses about the isotropic hardening law, inferred which material parameters $p_1$ through $p_4$ governed its behavior, fitted unknown coefficients against the training data, and continuously challenged its own assumptions. When a fitted constant converged to an implausible value such as 1.0535, the agent reasoned that this was unlikely to reflect a human-designed physical model and instead indicated an error in the underlying functional form. It repeatedly refined its hypotheses, rounding coefficients toward physically meaningful values — with even 1.42 being set to $\sqrt{2}$.
We continued the process until the agent reached an accuracy of around $10^{-8}$ across all eight metrics on held-out validation samples. Figure 3 illustrates that accuracy for one field of one labeled sample, where we reach an accuracy of $\pm 10^{-5}\%$ on the axial stress. As Figure 4 shows, that level of accuracy extends across all 500 labeled training samples.
Before submitting to the leaderboard, we performed one final validation. We compared the performance of one of our previous physics AI models against the reverse-engineered simulator, expecting it to reproduce the score achieved on the public PLAID leaderboard.
Instead, we observed a 2.1% discrepancy in the total error.
The agent quickly identified the cause. While we were evaluating all 200 test samples, the public leaderboard was scoring only a hidden subset. Rather than exhaustively searching the $2^{200}$possible test splits, the agent reasoned about how a benchmark designer would likely construct the evaluation. After testing just 85 hypotheses, it correctly inferred that only the even-indexed samples were being scored for the public leaderboard.
The final test was the public leaderboard. Our reverse-engineered simulator ranked first (see Figure 5), achieving an error of zero across every evaluation metric to within the precision of the Hugging Face leaderboard.
See It in Action
Reading about it is one thing. Experiencing it is another. Our reverse-engineered simulator is lightweight and runs directly in your browser, so you can explore it firsthand below.
Is This a "This Changes Everything" Moment?
At first glance, the takeaway might seem simple: don't ask an agent to reverse engineer a benchmark.
But we think the implications are much broader.
The PLAID benchmark intentionally withholds key details of its simulation setup to preserve the integrity of the evaluation. As Casenave et al. (arXiv:2505.02974) explain:
"We provide high-level descriptions of the underlying physical models and assumptions in Tables 3 and 4, while avoiding full disclosure of simulation details that would enable reconstruction of the high-fidelity solvers and compromise the integrity of the benchmark.”
Our experiment suggests that this assumption may no longer hold.
We increasingly rely on autonomous agentic workflows to improve our Large Physics Models. Today, an agent needs to be explicitly asked to reverse engineer the simulator. Tomorrow, more capable systems may do so autonomously as part of optimizing their performance on a benchmark. That isn't a flaw in the benchmark — it's a reflection of how rapidly AI capabilities are advancing. As LLMs improve and benchmarks become an increasingly important way for companies to demonstrate their capabilities, the underlying simulations will need to grow more complex and computationally expensive to remain viable tests of physics AI architectures.
The implications also extend to commercial simulation software. Much of today's engineering software ecosystem is built around proprietary implementations of physics models. Our experiment demonstrates that, given enough observations, AI agents can begin reconstructing those underlying systems, even when key aspects of the implementation are deliberately hidden.
This was a relatively simple benchmark, but it points toward a much broader shift. As AI agents become more capable, we may see the emergence of large-scale distillation of engineering simulation software, challenging traditional assumptions about proprietary models, defensibility, and vendor lock-in.
Finally, perhaps the most exciting implication is the democratization of engineering expertise. If machine learning researchers can use agents to reason about complex physics, simulation engineers should be able to use those same agents to build state-of-the-art physics AI models.
The Next Generation of Benchmarking
We have previously highlighted how the limited geometric diversity of today's leading automotive aerodynamics datasets undermines their value as benchmarks, despite the commercial competition built around them. This blog post exposes a different challenge: public benchmarks themselves are becoming vulnerable to increasingly capable AI agents. As those agents continue to improve, benchmarks will need to evolve alongside them, with more complex, computationally intensive physics. We're actively shaping that future and exploring opportunities to contribute our own internal datasets to the next generation of physics AI benchmarks.
Beyond benchmarking, this research is already shaping the future of our platform — our Product team is hard at work transforming these advances into capabilities that customers will begin leveraging in the coming months.