The reason ChatGPT states wrong facts in the same confident tone as right ones is that the model has no external signal at generation time. To add that signal, run the answer through a three-step grounding loop: extract the atomic factual claims, search the live web for each one, and label it supported, unsupported, or contradicted with the sources attached. This is exactly what tools like TrustLens and browser fact-checkers do under the hood, and you can build it in about 40 lines. The search step is the part people get stuck on, because you need clean, structured results, not a page of HTML to parse. This tutorial uses Scavio's Google endpoint (POST https://api.scavio.dev/api/v1/google, Authorization: Bearer) so each claim comes back as titled snippets you can score directly. Call the whole thing the Scavio grounding checklist: extract, retrieve, compare, score, cite.
Prerequisites
- A Scavio API key (50 free credits, no card) from scavio.dev
- Python 3.10+ with the requests package
- An LLM you can call for the claim-extraction and comparison steps (any provider)
- An answer you want to check (paste any ChatGPT/Claude/Gemini response)
Walkthrough
Step 1: Extract atomic claims from the answer
Split the LLM answer into standalone factual claims, one verifiable statement each. 'The Eiffel Tower is 330m tall and was completed in 1889' becomes two claims. Opinions and hedged statements ('probably', 'many people think') are skipped, since they aren't checkable. Ask your model to return a JSON list of claim strings so the next step can loop over them.
CLAIM_PROMPT = '''Extract every standalone, checkable factual claim from the\ntext below. Ignore opinions, predictions, and hedged statements.\nReturn a JSON array of short claim strings.\n\nTEXT:\n{answer}'''Step 2: Search the live web for each claim
For each claim, run a search that returns real evidence. Use the claim itself (or a keyword-reduced version) as the query. Scavio's Google endpoint returns organic results as structured JSON; setting light_request to false also pulls the knowledge graph, which often answers factual claims (dates, heights, populations) directly. Take the top 5 snippets as the evidence set for that claim.
import os, requests
H = {"Authorization": f"Bearer {os.environ['SCAVIO_API_KEY']}",
"Content-Type": "application/json"}
def evidence_for(claim: str) -> dict:
r = requests.post("https://api.scavio.dev/api/v1/google", headers=H,
json={"query": claim, "light_request": False}, timeout=30)
r.raise_for_status()
d = r.json()
return {
"snippets": [{"title": o["title"], "link": o["link"],
"text": o.get("snippet", "")}
for o in d.get("organic", [])[:5]],
"knowledge_graph": d.get("knowledge_graph"),
}Step 3: Compare the claim to the evidence
Send the claim plus its evidence snippets back to the LLM and ask for a verdict: supported, unsupported, or contradicted, with the specific source link that decides it. Keep the model's job narrow. It isn't answering the question, it's only judging whether the evidence backs the claim. That constraint is what makes the second LLM pass reliable where the first pass hallucinated.
VERDICT_PROMPT = '''Claim: {claim}\n\nEvidence (search snippets):\n{evidence}\n\nReply as JSON: {{"verdict": "supported|unsupported|contradicted",\n"source": "<the link that decides it>", "why": "<one sentence>"}}'''Step 4: Roll verdicts into a trust score
Turn the per-claim verdicts into one number so a human knows whether to act on the answer. A simple, defensible scoring: supported = 1.0, unsupported = 0.4, contradicted = 0.0, averaged across claims, times 100. Anything with a contradicted claim drops fast, which is the behavior you want. Attach the source links so the score is auditable, not just a vibe.
Step 5: Handle the edge cases before you trust the score
Two failure modes to guard: a claim with zero relevant snippets should be 'unsupported', never 'supported by absence', and a claim that search confirms but with a stale source (old pricing, superseded version) needs a freshness check on the snippet date. Grounding reduces hallucinations, it doesn't eliminate them, so surface the sources and let the human make the final call on high-stakes claims.
Python Example
import os, json, requests
H = {"Authorization": f"Bearer {os.environ['SCAVIO_API_KEY']}",
"Content-Type": "application/json"}
SCORE = {"supported": 1.0, "unsupported": 0.4, "contradicted": 0.0}
def search(claim):
r = requests.post("https://api.scavio.dev/api/v1/google", headers=H,
json={"query": claim, "light_request": False}, timeout=30).json()
return [f"{o['title']}: {o.get('snippet','')} ({o['link']})"
for o in r.get("organic", [])[:5]]
def fact_check(claims, verdict_fn):
# verdict_fn(claim, evidence) -> {'verdict','source','why'} via your LLM
results = []
for c in claims:
ev = search(c)
v = verdict_fn(c, ev)
results.append({**v, "claim": c})
trust = round(100 * sum(SCORE[r["verdict"]] for r in results) / len(results))
return {"trust_score": trust, "claims": results}
if __name__ == "__main__":
claims = ["Scavio's free tier gives 50 credits",
"Scavio returns Google AI Overviews"]
# plug in your own verdict_fn; second claim should come back contradicted
print(json.dumps(fact_check(claims, verdict_fn), indent=2))JavaScript Example
const H = { Authorization: `Bearer ${process.env.SCAVIO_API_KEY}`,
"Content-Type": "application/json" };
const SCORE = { supported: 1.0, unsupported: 0.4, contradicted: 0.0 };
async function search(claim) {
const r = await fetch("https://api.scavio.dev/api/v1/google", {
method: "POST", headers: H,
body: JSON.stringify({ query: claim, light_request: false }),
}).then((r) => r.json());
return (r.organic || []).slice(0, 5).map(
(o) => `${o.title}: ${o.snippet ?? ""} (${o.link})`);
}
async function factCheck(claims, verdictFn) {
const results = [];
for (const c of claims) results.push({ ...(await verdictFn(c, await search(c))), claim: c });
const trust = Math.round(100 * results.reduce((s, r) => s + SCORE[r.verdict], 0) / results.length);
return { trustScore: trust, claims: results };
}Expected Output
A JSON report per answer: an overall trust_score (0-100) plus a list of claims, each with its verdict, the source link that decided it, and a one-line reason. A clean answer scores near 100; an answer with one contradicted claim drops below 70 and flags for review. Cost is one Google call per claim; at light_request: false that's 2 credits ($0.01) each, so a 6-claim answer costs about $0.06 to verify against live sources. The 50 free credits cover roughly four full answers end to end.