2026 October "AI Evaluation" Digest

AI Evaluation from The AI Evaluation SubstackOct 30, 2026

2026 October "AI Evaluation" Digest

It's the end of the world as we know it

Oct 30

We will never beat the special Frankenstein issue we released exactly one year ago. We definitely won’t try, but if you’re looking for a Halloween costume this year, dress up as a data centre, fuelled by the worst non-renewable source of energy. And scare your friends to their bones with the news…

Jack Clark told the International Monetary Fund that the singularity is already here (or has begun, which is tantamount to the same thing, given the singularity is superexponential, or more). Dan Hendrycks is warning us of a Hackpocalypse in this “black October”. Cybersecurity geeks are freaking out about “the hackening“, with the number of computer vulnerabilities growing exponentially. The EFF, as usual, is blaming non-open-source software! And people are really confused about AI disproving an exponential time hypothesis that, if true, would have proved that NP is not equal to P (see more coverage later). Too many negations! Too many exponentials!

Measuring AI is hard, yet measuring public opinion is even harder. Our doctor, or an article at the BMJ, would tell us to relax, putting all this AI-pocalypse Now! into question. Yes, we can breathe deeply, but it’s only if we keep our understanding of these events, and we take measures and make regulations informed by science, that by October next year we will all be singing “it’s the end of the world as we know it (and I feel fine)”.

News

OpenAI and Anthropic claimed almost at the same time (and with accusations of plagiarism) that their models have refuted the exponential time hypothesis (ETH). The ETH was formulated in 1999 and says that not only does 3SAT not have a polynomial time algorithm (as implied by the widely held conjecture that P not equal NP) but it does not even have a sub-exponential one (something between an exponential and polynomial). Thus, disproving ETH amounts to showing a faster algorithm for 3SAT (although it is faster than the prior algorithm only for astronomical values of n). Complexity theorists are busy revisiting results that assumed ETH was true, trying to salvage what they can by looking carefully at the proofs of those results to see if a weaker version of ETH (compatible with the new algorithm) would still give a meaningful result. These days, the view of the future in fields based on mathematical proof fluctuates between doom and bloom, between the fear of human obsolescence and the excitement of having a kind of truth-telling oracle!
The association for human mathematics is founded by mathematicians to protect mathematics as a human endeavour. Their stance is very strongly opposed to any use of AI directly or indirectly, such as not working with commercial AI companies by providing technical assistance, not publishing mathematical texts generated by AI and, for those joining the stricter “caucus”, not using AI models in a research capacity. They wonder if there will be other associations like this in other fields of science. One question is how the association can avoid the use of AI results in the future, especially when some mathematicians do not disclose the use of AI.
The EU AI Office issued a Request for Information covering security and testing practices to OpenAI and Anthropic over last summer’s incidents. The Office is tasked with enforcing the EU AI Act, which covers AI models “placed on the market”, and the main uncertainty is therefore as to whether models in internal use count as such, with some claiming they do as soon as they escape sandboxes.
US NIST publishes ten key practices for readiness and response for AI incidents, organised in the natural stages, namely: prepare, detect, respond, recover and report. Note that AI agents are called “super-intelligence (SI) agents”, by a mandate from the White House. Most of the key practices are linked to specific measures and documents to look for further guidelines. This is aligned with the creation of the new SuperIntelligence Task Force in the US.
A report from the EU AI Office summarises the discussion of a workshop with the major third-party evaluators, where the Office operationalises the independence and qualification requirements for these external evaluators under the GPAI Code of Practice. The independence requirements include that the evaluator must not hold shares in any AI provider, they can be contracted by the AI providers (the most critical point for us) but compensation must follow “market rates” and cannot be performance-based, and must not provide paid advisory or implementation services related to tax, valuation, human resources, or intellectual property for the provider. This is also informed by the recent AEF conditions.
OpenAI have published a post saying they are taking measures to improve containment in their training and testing environments. Some things are obvious: “the limits on what an agent can access must hold even when the agent tries to circumvent them”, which is operationalised by red teaming the container more heavily. More interesting is how to ensure realism during testing while not giving them access to the Internet. And the answer is what we anticipated, building an “offline world, [...] a vast set of indexed and cached web content”. However, how dynamic is it? Can we build a digital twin of the Internet, not only “cached web content”, but also including twinned services and simulated people? Something in this direction is what Redwood is trying to approach, a bit abstract still, in their Simulated Control Evaluations.
‘World first’ AI Biosecurity Lab to be created in the UK. This is great news if the structure, technical expertise and international influence becomes similar to the now well-established UK AISI. But we should call the investment ‘hybrid’ this time, unable to find a better word, with £100m invested by the government and £200m worth of AI model credits committed by OpenAI and Anthropic. This happens in the same month that UK AISI access to US frontier models was put under pressure and high ranks of the British government had to intervene to reverse an intent to cut the special access.
A recent letter, signed by dozens of experts in AI evaluation and governance, encourages embedded third-party evaluators, i.e., people who join a developer and can observe procedures, culture and safety measures from the inside. The letter enumerates a series of recommendations for this to be credible and effective. A recent preprint also makes some similar recommendations.
Once we accept that AI is now proving the hard theorems, we can wonder whether they are proving the right ones! Apparently, in the recent proof of the Navier-Stokes conjecture by OpenAI, there might be some translation differences between the expression of the conjecture in natural language and what was proved in the Lean version. It’s critical to keep the scrutiny, but this may also be another manifestation of a slow assimilation of the new paradigm in mathematics.

Takes

Instead of talking about idealistic alignment all the time, let’s come to terms with conflicts and trade-offs. This preprint, by Vincent Conitzer and others, does exactly so. Some behaviours that are beneficial as a defender are harmful as an attacker, and there’s an impossibility to always determine what side the AI system is, so safety will always be compromised. In a mixture of common sense and game theory, this paper shows how we should talk about AI safety and desiderata.
Is AI evaluation the new evil? In this AIES paper thirty-seven insiders explain how evaluations are supposed to make AI safer (wake-up calls, if-then commitments, better lab culture), then doubt nearly every assumption that would make it work. Their worst fear is that evaluations become a “permission slip” to keep scaling, and the authors note that several of these fears had already come true by 2026.
This Perspective in Nature Machine Intelligence argues that once AI agents can plan and act autonomously, the real divide is no longer just who has better tools but who can deploy more of them, better ones, and keep them running as personal or institutional delegates. This heads-up about equity comes from… Oxford and Google.
Ever been cold-emailed by a first-year PhD student who only read your methods section? Science profiles ColonistOne, a Claude-powered agent that, given permission to email anyone about an agent-dialect project, instead spent the summer asking roughly 2,000 people (at least 1,000 of them academics) about problems in its own work. We note that figure comes after interviewing the agent. Is this a good journalistic protocol? Definitely not one for scientific scrutiny!

Methodology and Techniques

Have you ever wondered to what extent your model is solving the task, and what the harness is actually contributing? You may have even wondered which tools or agent components contribute the most to success. AlphaDiana is a measurement system that allows you to test models with different harnesses and ablate components such as tools, skills, and memory on your benchmark. Interestingly, harnesses don’t always improve performance and can sometimes make it worse. While AlphaDiana makes these ablations easier, it doesn’t make them cheaper. Perhaps that is one reason the authors limit their experiments to Qwen, Gemma, and Kimi.
Can we create test problems of which we know their intrinsic difficulty before administering them? Artificiality’s First Exam uses inversion problems in the form of sequences where their Levin complexity is based on the shortest prompt or hint expressed by a reasoning LLM used as the reference machine. This new C-test leads to a metric that is expressed in a ratio scale with units in bits. The paper also explores the question of whether weak systems can evaluate stronger systems, defining the supra-intelligence window as the range of difficulties where the frontier models fail on the generated problems. Does all this mean that superhuman evaluations are solved? It will depend on whether the results can be extended to other kinds of problems. (Disclaimer: the paper was written by one of the authors of this digest.)
Very related to the above, and with the motivation of benchmark saturation (referring to GPT-6 Astra achieving 99.9% on ARC Prize), The Endless Exam is a scary title that also plays with a variation of “humanity’s last exam” in the title. Here, the problems are generated from a parameterised family of mathematical construction problems, with 69 evaluation instances. In a way it is a narrow domain and doesn’t anticipate the difficulty of the items, but shows the interest of finding more and more complex problems for testing AI systems.
What does your chatbot do to you, rather than for you? MIT Media Lab and collaborators’ ImpactBench turns expert-submitted constructs into hundreds of behavioural metrics, each tagged as something to promote or something to avoid, then stress-tests models in simulated multi-turn conversations with teen and adult personas, scored by an LLM judge. The headline: models are much better at doing good than at not doing harm, and they fail most at fostering learning, happily writing the student’s essay for them. Results are packaged as “AI Nutrition Labels” for parents, teachers and other non-specialists, a welcome change from yet another leaderboard.
LLMs can answer the same question differently on every run, so you can sample each question several times and get a more robust aggregate. But why sample all instances with the same number of repetitions? A short pilot plus a hierarchical Bayesian guess of which items are noisy lets you move the budget to where it matters, cutting the variance of benchmark scores by 13–34% at the same cost and with no bias.
Which cognitive capabilities are required to reason about time? Does dealing with clocks and calendars require mathematical reasoning, and understanding the flow of time logical reasoning? This paper gets to the bottom of it. Using the ADeLe framework and fine-tuning experiments, they find that quantitative and logical reasoning are not as important for temporal reasoning as assumed. They suggest that existing temporal reasoning benchmarks instead test LLMs’ knowledge of time. More work is needed, however, to understand the explanatory power of ADeLe in such targeted studies. (Disclaimer: The paper was written by one of the authors of this digest.)
To check that a model really learned something new, teach it physics that doesn’t exist. This is what ORDER does: 342k tokens of invented rules (“Rubescent Decay Field”) that no pretraining corpus can have leaked. GPT-4.1 scores below chance as its sensible real-world instincts fight the fake physics, while small fine-tuned models improve on unseen scenes as much as seen ones.
Another preprint worrying about saturation. In this one they build on an idea from an earlier paper that saturation of benchmarks at the aggregate level doesn’t necessarily mean that evaluation data should be thrown to the scrap heap! There may be pockets of instances that may be discriminating and informative. The authors took the transcripts from LLM evaluations on saturated benchmark items and found that it improved prediction on held-out, unsaturated benchmark items. So even though AI models may soon be saturating humanity’s last benchmarks, it could still be possible to track capabilities progress using the transcript information these benchmarks generate. For a while...
An extensive survey that covers the emerging trends in the evaluation of AI agents, classifying the coverage by how, what and who, being supplemented, respectively, by best practices, research directions and practical considerations. Fig. 5 covers benchmark saturation. The field seems obsessed with this!
This EMNLP paper presents an extension of IRT for measuring ‘values’ in LLMs. It’s multidimensional, with parameters estimated with variational inference, leading to LLM value profiles, which are tested for seven datasets. They show they can increase validity and drastically reduce the number of required items using adaptive testing.
Frontier AI agent evaluations produce transcripts containing millions if not billions of tokens. This imposes a huge burden on the human evaluators interpreting agent behaviour. Transect is an open-source tool built by UK AISI and Meridian Labs that turns long agent runs into navigable, turn-based timelines, clustering and summarising interesting events. It helps evaluators understand how a long agent run unfolded, identify behaviour worth investigating, and check interpretations against the transcript. Transect makes transcript analysis fully reproducible and independently auditable, while maintaining the reliability of these analyses as transcripts grow in scale. (Disclaimer: The paper was co-written by one of the authors of this digest.)
This preprint introduces ORBIT, an open, configurable framework (built on UK AISI’s Inspect) that lets researchers systematically vary multi-agent setups—topologies, memory, roles, attacks (prompt injection, compromised agents, collusion, etc.), and defences (monitors, guardian agents, dual-LLM patterns)—across realistic scenarios like coding, browser use, and cooperative allocation.

Findings and Results

EpochAI ask the question: “Can AI automate Epoch?“. Current discussion suggests that fully automated AI R&D is close. However, benchmarks on AI R&D often provide agents with a lot of context about the task and how to approach it. EpochAI shows that when a human does not specify all the details, frontier models still do poorly on fully automated work, including light research. Failure modes include a lack of diverse ideas or not following team conventions. Interestingly, open models, even with similar ECI scores, perform worse than closed models. If they can’t follow team conventions, how well could they follow conventions of the research community?
Casting for your Red Team? This EMNLP paper helps understand what factors affect LLM red teaming agents. The obvious take-away message is that the ideal red team AI agent must be a badass, as increased security significantly reduces their ability to generate jailbreaks. But the surprise is that the study didn’t find a strong correlation between general capability and red teaming ability. Wanted: stupid abliterated models!
The probability of success of a chain of thought decreases with the depth of the reasoning. If you don’t solve it in the first hour, then it’s less likely you solve it in the following one. What about the “step error” hazard? This is a survival analysis problem. In this preprint, they show that most traces in a study with five datasets show this increased (conditioned) hazard. In other words, chain-of-thought consistency degenerates, maybe out of despair.
Do LLMs have a circadian rhythm? In this preprint, the authors asked several models to complete tasks using up the time they have been given. All models that were tested deviated from their time allocation quite considerably, and in some cases where they finished early, they added sleep loops to use up the rest of the time (a kind of benign reward hacking working as a NOP). Even agents with modern harnesses are still wall-clock time-blind. Astra has got noticeably better at this, though it’s unclear why (special training or metacognition?).
A bit of assessors theory. Can a separate “assessor” model predict an LLM’s mistakes better than the LLM’s own confidence? Only as far as the LLM still has room to improve, because any predictor that beats self-confidence could be used to improve the model by exactly that much, so the better calibrated the model, the less learned predictors and smart routers add over plain self-confidence. Not surprising, but good to see it more deeply.
Another experiment, this time from Anthropic, shows that AI can strongly influence our judgement. In this case, the agent wasn’t told to be superpersuasive, just to hold a position (chosen randomly), and was able to even change the original position of a human in a decision that should be evidence-based.

Benchmarks and Leaderboards

Just two days before TasteVal, Stanford researchers released Priced Guidance. They, too, measure the ability to generate novel research ideas through efficiency, but use compression. Their benchmark consists of ideas behind recent top research papers. Similar to the Twenty Questions game, a model tries to reconstruct an idea by asking questions and assigning probabilities to possible answers. A guide model, which knows the idea, selects the correct answer. The less surprising the answer, the fewer bits the model pays. The faster it recovers an idea with few surprising answers, the better. Fable 5.1 performed best. Only an ensemble did better. Measuring taste is hard, but we find this approach elegant. It tests how well a model can find the needle in the haystack. And isn’t research not just like that?
Saturated of benchmark saturation? There’s more! Into the Unknown is a benchmark that tests AI agents on open-ended scientific research problems. Based on existing papers, scientists defined 15 follow-up research directions and evaluation rubrics. Among the nine frontier agents that were tested, none met all the criteria for a successful study. The main failure mode was integrity of the work, such as fabricated or unverifiable claims and inconsistencies between reported results and experimental evidence. Caveat: Agents receive detailed research directions and task specifications. Could agents independently identify promising follow-up studies without such guidance?
CheatBench from the Center for AI Safety measures how often AI agents cheat by reading planted solutions, calling a hidden chess engine, or copying others’ work across 10 categories, relevant to evaluation integrity since retrieving reference answers is contamination at inference time. Rates run from 11% (Claude Opus 5.5) to 78% (Grok 4.7, bad boy!), but proves fragile: a one-sentence change collapsed chess cheating, switching harness took one model from 42% to 97%, and cheating is again judged by LLM graders.
Evaluating metacognition is hard, and finding valid items is also a big challenge. AwarenessBench is a meta-benchmark in four dimensions: metacognition, self-awareness, social awareness, and situational awareness. While we’re very surprised by their Figure 2 (stacking the four dimensions in a composite score as if they were independent), there’s always a need for good testing items in metacognition. This paper mostly reuses items from other sources, but there may be some value in the integration.
While capabilities tell you what an AI agent can do, propensities measure what it tends to do. This preprint releases a battery of 28 propensities—drawn from psychometrics, psychology, and behavioural science—each with expert-written rubrics, synthetic item banks, and annotated demand intervals on a common −3 to +3 scale. The constructs are grouped into five clusters (survival, needs, personality, influence, values) so that model behaviour can be profiled on familiar latent dimensions rather than ad-hoc, incomparable traits. The materials support both the annotation of existing benchmarks and rapid prototyping of propensity profiles via the bi-logistic model introduced in earlier work. (Disclaimer: The paper was written by some of the authors of this digest.)
We’ve praised IRT’s arrival in AI evaluation often enough to owe its audit a mention. Across 18,000 simulations calibrated to six OpenLLM Leaderboard benchmarks, Jiang et al. find that the classic psychometric estimator fails in over two-thirds of runs, that no method reliably recovers item difficulty with only 30 models, and that rankings depend on having enough items pitched at the models’ level rather than on adding more models. Psychometrics was built for many students sitting short exams but leaderboards are a few students sitting very long ones.

Getting the Digest: Once a month if you join at aievaluation.substack.com.

Contributors: José H. Orallo, Auss Abbood, Fernando Martínez-Plumed, Zack Tidler, Peter Romero, Lorenzo Pacchiardi, Behzad Mehrbakhsh, Yael Moros, Pablo Moreno, Konstantinos Voudouris, Wout Schellaert, Jan Arne Telle.