Project Case Study · Built and Operated by Robert Hu
Which brands do AI assistants recommend, for which shopper needs, and how does that change? I built RecoScope to investigate those questions with repeated prompts, structured data, and public reports that people can inspect.
Operating scale
4
AI systems evaluated
14
Categories tracked
119
Published benchmark runs
489
Brands measured
6,669
Brand mentions
March 2026
Operating since
Figures come from published, public benchmark records. They measure the research corpus, not users, customers, or commercial impact.
How the system works
Evaluate
The same commercial-intent prompts are tested across ChatGPT, Claude, Gemini, and Perplexity in the same time window, with the same persona and prompt text reused for repeat runs. Tier and model changes are recorded as comparison boundaries.
Capture
Responses are collected from the live consumer interfaces, with tier, model, mode, and surface evidence recorded. Unavailable cells and retries remain explicit.
Normalize
Brand references are cleaned and standardized, connected to categories, prompts, agents, and time periods, so the same brand is counted as one across models and runs.
Analyze
Mention frequency and explicit recommendation rank are evaluated separately. Changes over time are interpreted with collection gaps and instrument changes in view.
Validate
Capture and ingestion checks gate publication. Published reports retain their quality and human-review status; publishing a run is not the same as human review.
Publish
Tracker reports, research findings, prompt-level pages, and private analysis outputs are all generated from the same structured dataset.
Does a brand appear for the shopper's needs, or only when the shopper names it?
Is the brand explicitly recommended, or simply included among the alternatives?
Do different assistants select the same brands, and does that pattern persist in repeat runs?
These observations can help scope a brand or retail pilot and define its research questions. They do not establish why a model selected a brand or whether changing product content will improve sales.
Inspect a category report →The data model
Every public output is derived from one normalized model. Each entity connects to the next, so a single benchmark run is fully traceable from raw response to published finding.
Categories
The product categories under evaluation (office chairs, running shoes, protein powder, and more), each with a benchmark cadence.
↓
Runs
A single benchmark of a category at a point in time. Runs carry a status and are only public once published.
↓
Agent responses
The raw output from each model for each prompt in a run, preserved separately from the structured data derived from it.
↓
Brand mentions
Brand references retain their raw string and normalized name. Rank is recorded only when explicitly supported; unranked mentions remain unranked.
↓
Run insights
The interpreted findings for a run: cross-model differences, common traits, gaps, and the key takeaway.
Decisions behind the platform
The system reflects a series of product and operating decisions, each made to keep the evidence comparable, auditable, and honest.
Separate visibility from selection
A brand can be named as an alternative or a poor fit. Mention counts measure presence; explicit ranks measure recommendation position. Carousel placement is never substituted for rank.
Keep the shopper question constant
Repeat runs reuse the category's persona and exact prompts. Changing the question and the model at once would make a movement difficult to interpret.
Record the instrument, not just the answer
The live tier, model, mode, and response surface are recorded. A model or tier change is a boundary in the series, rather than an unexplained change in brand performance.
Keep failures visible
Unavailable responses, retries, and partial runs retain their status. Missing cells are not invented recommendations, and a rejected capture is not presented as a clean result.
Preserve the path back to evidence
Raw responses remain separate from normalized brand records. That makes it possible to check an interpretation against the response instead of treating the extracted table as the only evidence.
Use explicit publication and access gates
Public queries filter for published, public runs. Quality and human-review status remain separate; private analyses have a different access path.
What the system produces
Public research and an example brand analysis, supported by the same measurement system.
Tracker →
Live category benchmarks with cross-model rankings, trend data, and movement over time.
Research →
Findings drawn from the dataset: cross-model patterns and how recommendations shift.
Methodology →
How prompts are run, parsed, normalized, and scored, and how models are classified.
Private analysis →
A per-brand visibility report, scored from the same data. See an example output.
The public application uses Next.js, React, and TypeScript, with a Neon PostgreSQL database and Vercel deployment. Collection uses the live ChatGPT, Claude, Gemini, and Perplexity interfaces; those observations are distinct from an API-based model evaluation.
I use AI-assisted tools to develop and operate the system. My work centers on the commercial question, the measurement rules, the data and report design, and the decisions about what the evidence supports.
Inspect the source code →What RecoScope does not claim
RecoScope does not reproduce model training data, explain every causal factor behind a recommendation, or provide universal rankings. It measures observed model outputs for the tested prompts and interfaces. It does not establish sales lift, explain model internals, or guarantee that another session will return the same answer. Collection can require manual intervention; this is an operating research project, not a claim of fully unattended reliability.
RecoScope was designed, built, and is operated by Robert Hu to make AI product discovery measurable.
About Robert & career focus