01
Begin with the Configuration Lock
Before reading a headline metric, confirm what was measured: channel, system, exact API model or visitor surface, web-search status, language, region, prompt families, language scenarios, repeats and date. These fields define the result.
If a later cycle changes those inputs, it is a new baseline version. A before/after chart is meaningful only when the relevant method stays fixed and the changed variables are disclosed.
02
Check the denominator, not only the percentage
A mention rate of 40% means different things at 2 of 5 answers and 80 of 200 answers. Look for planned answers, valid answers, technical invalids, retries and missing rows. One unexpected extra row should stop the cycle rather than disappear inside an average.
Retries should belong to one specific technical invalid. Re-running a whole prompt family or system can create hidden fan-out and bias the sample.
03
Trace every claim back to evidence
For an AI-answer finding, open the prompt, raw answer snapshot, system, model or surface, timestamp, citation URLs and validation state. For a website-readiness finding, open the page URL, text or HTML fragment, selector, rule ID and rule version.
A citation URL proves that the answer referenced that URL in the captured run. It does not automatically prove that every sentence in the answer is correct or that the cited page supports the exact claim. That needs semantic citation review.
04
Separate observation, interpretation and action
Observation states what the captured evidence contains. Interpretation explains why it may matter. Recommendation proposes a change. A strong report keeps those layers visible instead of presenting an inferred cause as a measured fact.
Automated recommendations should be labelled as automated. An Expert Verified status requires an actual QC record covering semantic mentions, citations, factual errors and the approved priority actions.
05
Use the report as a controlled loop
Choose the highest-priority evidence-backed fix, preview it, implement it through an owner-controlled process and verify the exact page. Keep readiness before/after separate from AI visibility before/after.
When the change is ready for a real re-measurement, use the same Configuration Lock. Report the observed delta with its denominator and date, then monitor only while the baseline remains comparable.