Artificial Analysis Intelligence Index v4.2

Anthropic’s Claude Fable 5.1 topped the newly released Artificial Analysis Intelligence Index v4.2, with OpenAI’s GPT‑6 Astra in close pursuit, registering a four‑point advantage over the previous‑generation GPT‑5.6 Sol. The updated index, rolled out as an interim upgrade ahead of the planned v5, introduces more demanding and realistic tasks, expands private held‑out test sets to curb gaming, and upgrades grading infrastructure for greater robustness. New benchmark components include AA‑Briefcase, an in‑house evaluation that subjects models to multi‑week, agentic knowledge‑work projects built by industry experts, and Surge AI’s GDP.pdf, which tests single‑turn professional document reasoning across 100 PDFs and ten domains, requiring synthesis of evidence from 4,592 pages and grading against 1,275 expert‑authored criteria.

The most consequential change in v4.2 is the doubling of private, held‑out test set weighting to 40 percent of the overall score, up from 20 percent in v4.1. This shift incorporates AA‑Briefcase, AA‑Omniscience, and CritPt solutions, markedly reducing the ability of labs to tailor models to public benchmarks. Additional refinements include a new grading system prompt in AA‑LCR v1.1, corrected answer‑key ambiguities, re‑anchored Elo scales for more stable ratings, and hardened grading sandboxes for SciCode to ensure slow but correct code is not penalized. These measures collectively aim to align the index more closely with real‑world use cases and to preserve its relevance amid rapid frontier advances.

Beyond the interim release, the organization signals ongoing work on Index v5, promising further incremental updates. Current leaderboards show Anthropic, OpenAI, Meta and Z.AI occupying the cost‑per‑task Pareto frontier, while GPT‑6 Astra leads the output‑token efficiency curve, outperforming most competitors near the intelligence frontier. In the AA‑Briefcase evaluation, Claude Fable 5.1 and Opus 5 remain ahead, with GPT‑6 Astra and Muse Spark 1.3 trailing. In the GDP.pdf test, GPT‑6 Astra achieved a 33.2 % all‑pass rate, surpassing GPT‑5.6 Sol’s 28.2 % and Claude Fable 5.1’s 26.2 %. The updates position the Index as a more stringent, real‑world‑oriented metric as the AI landscape continues to evolve rapidly.

⚡ Effects Interpreter