Home/Methodology

Methodology & sources

This site exists for one reason: to give "which model is better" an answer you can actually check.

Where the data comes from

Scorescome entirely from published evaluations. We run no tests of our own, accept no vendor-submitted scores and adjust no ranking for commercial relationships. Sources:

Intelligence index
Artificial Analysis Intelligence Index v5 / v4.3.2
Science reasoning
GPQA Diamond (collected by Epoch AI and Metatext)
Terminal agents
Terminal-Bench 4.0(Stanford / Laude Institute)
Document generation
LiveBench Design (the LiveBench project)
Professional tasks
Vals Index (run independently by Vals AI)
Prices
Official vendor pricing pages (OpenAI, Anthropic, Google, DeepSeek and others)
Speed
Artificial Analysis measurements of median throughput and time to first token
Specs
Hugging Face model cards and vendor technical documentation
User preference
LMArena Text Arena Elo (anonymous pairwise votes; a preference-side sanity check only, never part of the ranking)
This is an aggregator, not an evaluation lab. What we can guarantee is accurate citation, traceable sources and no mixing of scoring scales; we cannot guarantee that every upstream evaluation is itself unbiased. After each update the numbers are cross-checked against the original AA leaderboard, the LMArena ranking and vendor price pages, and the result goes into the changelog — most recently on 2026-11-04. For a major purchasing decision, run a small A/B on real samples from your own business.

Three easy mistakes

Comparing across versions

The AA Intelligence Index moved from v4.3.2 to v5 on 2026-10-27. The same model, DeepSeek V4.1 Flash, went from 45.1 to 33.0 — not a regression, just harder questions. The column header carries the version so the two sets are never mixed.

Reading a dash as zero

Missing data does not mean zero capability. New models, open models and models for minority languages are less well covered, so sorting on the composite alone systematically underrates them.

Budgeting on unit price

Two models at $2/$10 can differ 3× in the output tokens one task needs. Opus 5.5 costs twice Sonnet 5.5 per token yet only 10% more per task, because it writes less. Budget on cost per task, not price per token.

Changelog

2026-11-04
Kimi K3.1 Pro released its full weights, so the open leaderboard changed hands and the open-weight v5 ceiling went from 32 to 46. The AA Intelligence Index v5.0.1 re-evaluation refreshed six scores (Carbon 60.0, Opus 59.1, Sonnet 54.5, Sol 54.6, Terra 51.3, Argon 50.4). Gemini 4 Carbon returned to list price at $4/$20, and Mistral Large 4 was added (1T open weights). Later the same day AA's 4 November re-read moved two more (Muse Wonder 44.0, DeepSeek V4.1 Flash 33.0), and the previously missing full-strength Muse Wonder Full was catalogued (v5 50.0 / v4.3.2 56.0, subscription only, not in the price comparison).
2026-11-03
Every figure was checked line by line against the original Artificial Analysis v5 leaderboard, LMArena and each vendor price page. Six Terminal-Bench 4.0 values, five index values and six prices were corrected. The one that changed a ranking: the 65.2% shown for Opus 5.5 is actually Gemini 4 Carbon's score, so it moves to 59.6%, and Kimi K3.1 Pro's 67.2% had been missing — the leader on that evaluation is Kimi, not Opus.
2026-11-03
First release: 28 models and 6 benchmarks, adding GPT-6.1 Terra (10-26), Gemini 4 Carbon (10-28), MiniMax M3.1 Flash (10-26) and Kimi K3.1 Pro (10-23).
2026-10-27
Artificial Analysis enabled Intelligence Index v5; this site switched its primary sort to match, keeping the old scores as a reference column.

Corrections

If a price, a score or a model spec looks wrong, tell us. Once verified, the correction is recorded in the changelog — nothing is changed silently.

Contact: jiaoliudao@qq.com