Investigate a hallucination
A hallucination flag means an engine asserted something that contradicts your brand profile. Work it in four steps: confirm the claim is actually wrong, trace it to a cited source, fix the underlying information, then watch whether it recurs. The goal is not to argue with the engine. It is to fix the web pages the engine read.
Before you start
Open the flagged run from the project. AppearIn shows you the exact wording the engine used, which engine produced it, and the sources that answer cited. You will need all three. Read hallucination flags for what the flag does and does not mean.A worked example
Acme sells a project management tool. On a scheduled run, ChatGPT answers the prompt "does Acme have a built-in time tracker?" with: "Yes, Acme includes a native time tracker with billable-hours reports." Acme has no such feature. AppearIn raises a hallucination flag because the answer asserts a feature that is not in Acme's brand profile. The walkthrough below uses this flag.
Work the flag
Confirm the claim is actually wrong
Read the saved answer in full, not just the flagged sentence. Sometimes the engine is right and your brand profile is stale. You may have shipped the feature, changed the price, or renamed a plan and never updated the profile. If that is the case, fix the profile and the flag clears on the next run.
For Acme, the time tracker genuinely does not exist, so this is a real hallucination. Move on.
Trace it to a cited source
A hallucination almost always traces to one page the engine read. Open the sourcesAppearIn captured for that answer and look for where the false claim could have come from: an outdated comparison article, a competitor's feature matrix that lists Acme in the wrong column, a stale review, or sometimes your own page with ambiguous wording.
In the example, the top cited source is a third-party "best PM tools" roundup from 2023 that wrongly grouped Acme under "tools with time tracking." That one page is the root cause.
Fix the underlying information
Engines re-read the web, so the fix is to correct the information at the source, not to flag the engine. If the page is yours, edit it for clarity. If it is a third party, request a correction, since most roundup and review sites will fix a factual error when you point to your own documentation.
For Acme, you email the roundup's editor with a link to your features page and ask them to move Acme out of the time-tracking column. You also add an explicit line to your own features page: "Acme integrates with time trackers; it does not include one natively." That removes the ambiguity an engine could repeat.
Track recurrence across scheduled runs
A fix is not done when you make the edit. It is done when the flag stops coming back. Keep the prompt in your prompt set and watch the next few scheduled runs. Engines pick up corrected pages on their own crawl schedule, so expect days to weeks, not minutes.
For Acme, the flag clears on ChatGPT after about two weeks once the corrected roundup is re-crawled. You confirm it stays clear across three consecutive runs before treating it as resolved.
What good looks like
A well-investigated flag has:
- A clear verdict on whether the claim is wrong or your profile was stale.
- A named root-cause source, not a vague "the engine made it up."
- A concrete fix, an edited page or a sent correction request, with a date.
- Two or three clean scheduled runs after the fix before you call it resolved.
Common mistakes
Trusting the flag before reading the answer. A stale brand profile produces false hallucination flags. Always confirm the claim against reality first.
Fixing the engine instead of the source. You cannot edit what ChatGPT says. You can edit the page it read. Spend your effort on the source.
Declaring victory on one clean run. Data is probabilistic. One run without the flag is noise; a run of clean runs is a trend.
What this cannot do
AppearIn gives you early warning and evidence. It cannot guarantee an engine never hallucinates. Hallucination is inherent to large language models: the same prompt can produce a false claim on one run and a correct answer on the next, and a corrected source can still be ignored or re-misread. What you get is the exact wording, the engine, the cited sources, and a recurrence trend, which is enough to fix the root cause and prove the fix held. See methodology for how flags are detected and why single runs are not conclusive.