Measurement rules · updated October 28, 2026
How to read the evidence.
RecoScope measures observed answers to saved shopper questions. Every report should make clear what was collected, which answers were usable, and what the extracted numbers mean.
The distinction that matters
Mentions: how often a brand enters the answer. Ranks: explicitly supported positions in a comparison. A mention can be an alternative or a criticism; it does not automatically receive a rank.
1. Keep the shopper question consistent
Each category uses a saved persona and three standalone prompt types: broad discovery, a budget or use-case constraint, and a comparison asking for ranks. Repeat runs reuse the exact saved text. A changed prompt set establishes a comparison boundary.
Record the collection conditions
Collection uses the live ChatGPT, Claude, Gemini, and Perplexity consumer interfaces. Account tier, model or selected mode, browser surface, native private/temporary settings, collection window, failures and retries are recorded. A setting that cannot be verified is disclosed; it is not silently assumed.
Each prompt starts in a fresh chat. Complete answers and relevant product cards or tables are preserved before extraction. A loading or capture failure may receive one fresh-chat retry, with the original failure retained. Contradictory or undesirable recommendations are not rerun to obtain a preferred result.
Model names, retrieval behavior and commercial features can change. They are collection context, not fixed categories that establish a model’s independence or commercial influence.
2. Measure presence and explicit position separately
| Measure | Rule |
|---|---|
| Mention frequency | One normalized brand per usable model/prompt answer. Alternatives and critical mentions count as visibility. Repeated names within that answer count once. |
| Mention rate | Usable answers mentioning the brand divided by all usable answers in the stated scope, including usable answers that name no brands. |
| Recorded rank | In current collections, a numeric position must be explicitly supported by the comparative answer. Product-card order, appearance order and category-best numbering do not substitute for an overall rank. |
| First and top-three positions | Recorded rank 1, or ranks 1–3, within the stated comparison answers. Unranked responses contribute mentions without invented positions. Ties and empty slots are preserved. |
| Cross-model agreement | Report which services mention or explicitly rank a brand, with the number of usable comparison answers stated. Agreement is an observation about this collection. |
Normalize names, preserve context
Raw brand strings are retained alongside normalized names. A brand is counted once per answer. Retailers, source labels, image-only captions and suggested follow-up questions are not additional product-brand mentions. Grouped ranks, product variants and ambiguous names require explicit interpretation notes.
3. Show coverage and review status
Unavailable, empty or explicitly excluded answers are removed from brand analysis and its denominator. A completed clarification or answer with no named brands remains usable. Reports show usable versus stored answer counts and participating services; a missing numeric ranking does not make an otherwise usable answer unavailable.
Capture, extraction and stored-data checks gate publication. AI-assisted review is distinct from human review. A report says “No human review recorded” unless a human review has been recorded; publication alone does not imply human validation.
Use dates as the source of truth
Collections follow a rolling rotation. The intended cadence is monthly for evergreen categories and weekly during active seasonal periods; actual collection dates and gaps remain visible. “Recent snapshot” means a publication within 45 days. “Historical snapshot” describes an older publication, not an unpublished report.
4. Interpret each snapshot within its limits
- AI outputs vary between sessions. A small set of captured answers does not predict every shopper’s experience.
- Model, tier, prompt, browser, collection window and parsing changes can limit historical comparisons. Older records may use earlier rank conventions; the current rules do not retroactively certify them.
- Trends describe recorded mention positions. They do not establish the cause of a change or a clean growth rate across method changes.
- Product prices, availability, performance, health, safety and certification claims in captured answers are model output unless separately verified.
- Differences between services do not prove bias, advertising influence, or another hidden cause. RecoScope does not measure sales lift.
- Brands do not pay to alter benchmark rankings. A private audit does not change the public benchmark.
Questions about the method
What does a brand mention measure?
A brand counts once per usable answer, including alternatives, comparisons and criticism. Mention frequency measures presence in the captured answers, not endorsement, product quality or sales.
When does a brand receive a numeric rank?
Current collections record a numeric brand rank only when the comparative answer explicitly supports that position. Product-card order, source order and category-best labels do not become inferred overall ranks. Unranked answers remain usable for mentions.
How are collection failures handled?
Unavailable or explicitly excluded answers do not contribute to the usable-answer denominator or brand analysis. Completed answers with no named brands remain usable. Reports disclose coverage, exclusions and retries.
Do differences prove commercial influence?
No. The benchmark observes answers for particular prompts and interfaces. It does not establish the hidden cause of a recommendation, advertising influence, model bias or sales impact.