How the helloranked score works
A plain-language tour of the number on the leaderboards — what goes in, what the ± means, and why we sometimes insist nothing happened.
Step 1: ask real questions, many times
Each industry has a frozen panel of ~50 questions phrased the way people actually talk to assistants, derived from real search demand and reviewed by a human before use. Every week, each question variant goes to each measured model three times — because models answer differently every time, and one sample proves nothing. That's roughly 1,200 answers per industry per week, all stored verbatim.
Step 2: extract who was mentioned, and how
A two-stage pipeline reads every answer: a fast alias scanner finds candidate brand mentions (including tricky ones — "On" the shoe brand, "monday" the software), then an LLM judge confirms each candidate, records the order brands were presented in, whether each was explicitly recommended, and the attributes attached to it. Ambiguous words never count as mentions without confirmation.
Step 3: score per model…
For each brand and model we compute the mention rate (share of answers naming the brand), position score (how early it appears when named), and recommendation rate (explicit endorsements). These blend into a 0–100 per-model score — weights are published on the methodology page, along with a citation component that activates when we add search-mode measurement.
…then weight models by how many people use them
A brand that dominates a niche assistant but is absent from ChatGPT is not very visible in practice. The global score weights each model's score by its assistant's approximate consumer market share. The weights are versioned and shown with their sources — when they're placeholders, the methodology page says so in plain sight.
The ± and the discipline of "stable"
Because answers are sampled, every score carries sampling error. We quantify it by bootstrapping: resample the week's answers 1,000 times, recompute the score each time, and report the middle 95% as the confidence interval — the ± you see everywhere. The same test gates every claim of movement: a brand only appears as a riser or faller when the confidence interval of its week-over-week change excludes zero. If the leaderboard says "stable", it means the movement was smaller than the noise — and we'd rather tell you that than invent a story.
Every methodology change is versioned and annotated on trend charts, and every raw answer is stored so scores can be re-derived. Numbers you can't interrogate are numbers you shouldn't trust.