Overview
AI agents generate text that may contain outdated or fabricated claims. This workflow takes agent output, extracts verifiable claims, searches for each claim via Scavio, and flags statements that contradict live search results. Catches hallucinations before they reach users.
Trigger
After every agent response that contains factual claims.
Schedule
Event-driven
Workflow Steps
Extract Claims from Agent Output
Parse the agent's response to identify verifiable factual claims: prices, dates, company names, product features.
Search for Each Claim
For each claim, construct a search query and call Scavio API to find current data.
Compare Claim vs Search Data
Check if the search results support, contradict, or are silent on the claim.
Flag Contradictions
Mark claims that contradict search data as potential hallucinations. Include the contradicting source.
Return Verified Response
Return the agent output with inline annotations showing which claims are verified, unverified, or contradicted.
Python Implementation
import requests, os, json, re
API_KEY = os.environ["SCAVIO_API_KEY"]
H = {"x-api-key": API_KEY, "Content-Type": "application/json"}
def search_claim(claim: str) -> dict:
resp = requests.post(
"https://api.scavio.dev/api/v1/search",
headers=H,
json={"query": claim, "country_code": "us"},
timeout=10,
)
data = resp.json()
return {
"results": len(data.get("organic_results", [])),
"ai_overview": data.get("ai_overview", {}).get("text", ""),
"top_snippet": data.get("organic_results", [{}])[0].get("snippet", "") if data.get("organic_results") else "",
}
def verify_agent_output(output: str, claims: list) -> list:
"""Verify extracted claims against live search data."""
verifications = []
for claim in claims:
search_data = search_claim(claim)
status = "unverified"
if search_data["results"] == 0:
status = "no_data"
elif claim.lower() in search_data["ai_overview"].lower() or claim.lower() in search_data["top_snippet"].lower():
status = "supported"
else:
status = "check_manually"
verifications.append({
"claim": claim,
"status": status,
"source_snippet": search_data["top_snippet"][:150],
})
return verifications
# Example: verify agent claims
agent_output = "Scavio offers 250 free credits per month and charges $0.005 per credit."
claims = ["Scavio 250 free credits per month", "Scavio $0.005 per credit"]
results = verify_agent_output(agent_output, claims)
for r in results:
print(f"[{r['status'].upper()}] {r['claim']}")JavaScript Implementation
const H = {'x-api-key': process.env.SCAVIO_API_KEY, 'Content-Type': 'application/json'};
async function searchClaim(claim) {
const r = await fetch('https://api.scavio.dev/api/v1/search', {method:'POST', headers:H, body:JSON.stringify({query:claim, country_code:'us'})});
const d = await r.json();
return {results:(d.organic_results||[]).length, aiOverview:d.ai_overview?.text||'', topSnippet:(d.organic_results||[])[0]?.snippet||''};
}
async function verifyAgentOutput(claims) {
const verifications = [];
for (const claim of claims) {
const s = await searchClaim(claim);
let status = 'unverified';
if (s.results === 0) status = 'no_data';
else if (s.aiOverview.toLowerCase().includes(claim.toLowerCase()) || s.topSnippet.toLowerCase().includes(claim.toLowerCase())) status = 'supported';
else status = 'check_manually';
verifications.push({claim, status, sourceSnippet:s.topSnippet.slice(0,150)});
}
return verifications;
}
const results = await verifyAgentOutput(['Scavio 250 free credits per month', 'Scavio $0.005 per credit']);
for (const r of results) console.log('['+r.status.toUpperCase()+'] '+r.claim);Platforms Used
Web search with knowledge graph, PAA, and AI overviews