Which brands do ChatGPT, Gemini and Claude agree on? Cross-model overlap across 4,500 answers

Why it matters

Most AI visibility tools query one model, or show one model at a time. This study puts a number on what that misses: of every brand and category pair that at least one of three model APIs named more than once, 78.2 percent came from one model only, and the three models shared the first-named brand in 12.3 percent of buyer questions. A report that says "you are recommended" after asking one model is reporting that model, not AI search.

Of the brands OpenAI's gpt-4.1-mini named in 2 or more of 50 answers, 45.1 percent were named by neither Gemini nor Claude in the same category. That is why LLM Audit asks three models the same questions and reports the count per model (named in X of N answers), and why the methodology page has no single score: the first study measured that one model does not repeat itself, this one measures that the three do not agree with each other.

The companion study, How repeatable are AI recommendations?, ran the same 4,500 calls and measured repeatability within each model. This page reuses those answers, at zero additional API cost, to measure agreement between models. Both publish the raw answers under CC BY 4.0, so every number here can be recounted.

Read the full reasoning in the methodology, section "Why there is no score", and the companion study, How repeatable are AI recommendations?.

Method

Data: the 4,500 raw answers of the repeatability study (2026-08-23): 30 categories (20 in English, 10 in Spanish about Argentina) x 10 buyer-question templates x 3 model APIs x 5 runs, with the brand lists already extracted by the product's extractor and normalized with its competitor-key rules. No new calls were made for this study.

Per category and model, every brand is counted once per answer (50 answers per model and category: 10 questions x 5 runs). The model's set is the brands named in 2 or more of those answers; its core is the brands named in 25 or more. The 386 answers with no extracted list (41 from OpenAI, 85 from Gemini, 260 from Claude) contribute no brands and are not removed from the denominator; 7 answers named the same brand twice and count it once.

Overlap between models is computed per category on the sets: Jaccard per pair, and for the union of the three sets how many brands are in all three, in exactly two, in one. Group figures sum the counts over the group's categories; the Jaccard reported for a group is the mean of the per-category values. Top-1 is computed per (category, question) cell: each model's most frequent first-named brand across its runs with a list (tie: the one that appeared first). Consensus brands are the brands in the core of all three models for their category. For display, each normalized key is shown as its most frequent raw spelling in the answers.

Models

• OpenAIgpt-4.1-mini

  Temperature 0. 50 answers per category.

• Geminigemini-flash-latest

  Temperature 0. 50 answers per category.

• Claudeclaude-haiku-4-5

  Temperature API default (1.0). 50 answers per category.

Metrics

Set (per model and category)

Brands named in 2 or more of the model's 50 answers in the category. Two is the lowest bar that drops one-off mentions.

Core (per model and category)

Brands named in 25 or more of 50 answers: half the time or more, across the 10 question templates.

In all three / in two / in one

For the union of the three models' sets in a category, how many brands are in all three sets, in exactly two, in exactly one. Summed over categories for the group figures, then divided by the summed union.

Jaccard between models

For two models in a category, the number of brands in both sets divided by the number in either. The group figure is the mean over categories.

Top-1 agreement

Per (category, question): each model's most frequent first-named brand over its 5 runs with a list. All three agree, exactly two agree, or none. A model with no list in any run has no top-1 and cannot agree.

Consensus brand

A brand that is in the core of all three models for its category.

Results

Overall and by language

Sets at 2 or more of 50 answers. Jaccard is the mean over categories; the other columns sum over categories.

Set and core size per model

Three-way membership at four set thresholds

Same count with the set defined at 2, 5, 10 and 25 of 50 answers, summed over the 30 categories.

By category

Set sizes per model, the three-way count on the union of the sets, pairwise Jaccard, and top-1 agreement over the 10 questions.

Consensus brands: core to all three models

Named in 25 or more of 50 answers by each model. Counts are OpenAI / Gemini / Claude. 41 brands in 15 categories; the other 15 categories have none.

The 5 most named brands per model, 10 categories

Count = answers out of 50 that named the brand. The full table for all 30 categories and every brand is in the download.

"AI visibility tools": the three sets

First 12 brands of each model's set, with the count of answers out of 50. Set sizes: OpenAI 28, Gemini 22, Claude 66; 1 brand in all three, 12 in two, 89 in one.

Limitations

• Same dataset as the repeatability study, so the same caveats: budget-tier models (gpt-4.1-mini, gemini-flash-latest, claude-haiku-4-5), not the flagships; one day (2026-08-23); template questions, not real user prompts; 5 runs per question.
• Temperature confound: Claude ran at the API default of 1.0, OpenAI and Gemini at 0. A higher temperature spreads mentions over more brands, which is why Claude's sets are the largest (30.2 brands per category against 16.0 for OpenAI) and its cores the smallest (2.2 against 5.1). That inflates the "one model only" bucket and lowers Claude's pairwise Jaccard. The threshold table shows the effect shrinking as the bar rises.
• The extractor found no list in 386 of 4,500 answers (8.6 percent), 260 of them from Claude. Those answers count in the denominator of 50, so Claude's counts per brand are lower than they would be on listed answers alone; this pushes Claude's brands below the core threshold more often.
• Brand normalization is heuristic. Product models stay apart from their brand ("Brooks Ghost 15" and "Brooks Ghost 16" are two keys), grouped items like "Semrush & Ahrefs" become one key, and some non-brand list items survive the extractor in prose-heavy answers (Claude's running shoes set includes "Try multiple pairs"). All of this adds to the "one model only" bucket. The raw answers are published so anyone can recount with a stricter normalizer.
• The thresholds 2 and 25 of 50 are choices. The results table repeats the three-way count at 2, 5, 10, 25 so the reader can pick the bar.
• Overlap is measured per category, not per question: a brand counts as shared when two models name it anywhere in the category, even to different questions. The top-1 figure is the per-question view.

Download the data

Three files, licensed CC BY 4.0. The per-brand table is new; the raw answers are the ones published with the repeatability study. The script that produced the table ships with the product and its method is on this page; the repository is private, so there is no source link.

• cross-model.csv (518 KB)

  One row per (category, model, brand): how many of the model's 50 answers in that category named the brand, the display label, and whether the brand is in the model's set (2 or more) and core (25 or more). 5,035 rows, every brand named at least once.

• cross-model.json (1.2 MB)

  The same table as JSON, with the definitions and the counts of answers without a list.

• raw.json.gz (2.9 MB)

  The 4,500 raw answers from the repeatability study with the extracted and normalized brand lists, from which this table is computed.

How to cite

LLM Audit, Which brands do ChatGPT, Gemini and Claude agree on? 2026-08-23, https://llmaudit.app/research/which-brands-do-the-models-agree-on

The data is licensed under CC BY 4.0: reuse it with attribution to LLM Audit and a link to this page.

Changelog

• 2026-08-23First publication.

Read next

• How repeatable are AI recommendations? The companion study on the same answers: how much one model repeats itself.
• All research