The Problem
Most AI agent failures are retrieval failures: the LLM never saw the right data, so it could not reason correctly. Teams invest in better models, longer contexts, and fancier prompts, but the bottleneck is upstream -- the agent's retrieval layer determines answer quality more than the reasoning model. A GPT-4 class model with bad retrieval produces worse answers than a smaller model with excellent retrieval.
The Scavio Solution
Design a retrieval layer that routes queries to the right data sources, fetches data in parallel, and merges results into a unified context. Use Scavio's multi-platform API as the search backbone: route product queries to Amazon/Walmart, opinion queries to Reddit, tutorial queries to YouTube, and general queries to Google. One API key, one integration, six platforms of retrieval surface area.
Before
Agent has one retrieval path: Google search via SerpAPI. User asks a product question -- agent gets blog posts, not pricing. User asks for opinions -- agent gets SEO content, not Reddit threads. User asks about a trend -- agent misses TikTok entirely. Retrieval quality: 40% relevant results.
After
Agent routes by intent: product -> Amazon+Walmart, opinion -> Reddit, tutorial -> YouTube, trend -> TikTok, general -> Google. Same query, 3x more relevant context. Retrieval quality: 85%+ relevant results. Four parallel API calls complete in under 3 seconds.
Who It Is For
AI agent architects designing the retrieval subsystem of their agent and looking for a pattern that routes queries to the most relevant data sources.
Key Benefits
- Intent-based routing puts the right data source behind each query type
- Parallel retrieval across platforms keeps latency under 3 seconds
- One API integration covers six platforms of retrieval surface
- Retrieval quality improvements compound across all downstream tasks
- Each retrieval call costs $0.005 -- even 4-platform queries cost $0.02
Python Example
import requests, os
from concurrent.futures import ThreadPoolExecutor
API_KEY = os.environ["SCAVIO_API_KEY"]
H = {"x-api-key": API_KEY, "Content-Type": "application/json"}
INTENT_ROUTES = {
"product": ["amazon", "walmart"],
"opinion": ["reddit", "google"],
"tutorial": ["youtube", "google"],
"trend": ["tiktok", "youtube"],
"news": ["google"],
"local": ["google_maps"],
}
def classify_intent(query: str) -> str:
"""Simple keyword-based intent classifier. Replace with LLM classifier for production."""
q = query.lower()
if any(w in q for w in ["price", "buy", "cost", "product"]):
return "product"
if any(w in q for w in ["opinion", "think", "review", "recommend"]):
return "opinion"
if any(w in q for w in ["how to", "tutorial", "guide", "setup"]):
return "tutorial"
if any(w in q for w in ["trending", "viral", "popular"]):
return "trend"
return "news"
def fetch_platform(query: str, platform: str) -> dict:
resp = requests.post(
"https://api.scavio.dev/api/v1/search",
headers=H,
json={"query": query, "platform": platform, "country_code": "us"},
timeout=10,
)
results = resp.json().get("organic_results", [])[:5]
return {
"platform": platform,
"results": [{"title": r["title"], "snippet": r.get("snippet", ""), "url": r["link"]} for r in results],
}
def retrieval_layer(query: str) -> list[dict]:
intent = classify_intent(query)
platforms = INTENT_ROUTES.get(intent, ["google"])
with ThreadPoolExecutor(max_workers=4) as pool:
futures = [pool.submit(fetch_platform, query, p) for p in platforms]
return [f.result() for f in futures]
# Agent retrieval: right platforms for the query
context = retrieval_layer("what do people think about the new MacBook Pro")
for source in context:
print(f"--- {source['platform']} ({len(source['results'])} results) ---")
for r in source["results"][:2]:
print(f" {r['title']}")JavaScript Example
const API_KEY = process.env.SCAVIO_API_KEY;
const H = {"x-api-key": API_KEY, "Content-Type": "application/json"};
const INTENT_ROUTES = {
product: ["amazon", "walmart"],
opinion: ["reddit", "google"],
tutorial: ["youtube", "google"],
trend: ["tiktok", "youtube"],
news: ["google"],
local: ["google_maps"],
};
function classifyIntent(query) {
const q = query.toLowerCase();
if (["price", "buy", "cost", "product"].some(w => q.includes(w))) return "product";
if (["opinion", "think", "review", "recommend"].some(w => q.includes(w))) return "opinion";
if (["how to", "tutorial", "guide", "setup"].some(w => q.includes(w))) return "tutorial";
if (["trending", "viral", "popular"].some(w => q.includes(w))) return "trend";
return "news";
}
async function fetchPlatform(query, platform) {
const res = await fetch("https://api.scavio.dev/api/v1/search", {
method: "POST",
headers: H,
body: JSON.stringify({ query, platform, country_code: "us" }),
});
const results = (await res.json()).organic_results || [];
return {
platform,
results: results.slice(0, 5).map(r => ({ title: r.title, snippet: r.snippet || "", url: r.link })),
};
}
async function retrievalLayer(query) {
const intent = classifyIntent(query);
const platforms = INTENT_ROUTES[intent] || ["google"];
return Promise.all(platforms.map(p => fetchPlatform(query, p)));
}
const context = await retrievalLayer("what do people think about the new MacBook Pro");
for (const source of context) {
console.log(`--- ${source.platform} (${source.results.length} results) ---`);
source.results.slice(0, 2).forEach(r => console.log(` ${r.title}`));
}Platforms Used
Web search with knowledge graph, PAA, and AI overviews
Amazon
Product search with prices, ratings, and reviews
YouTube
Video search with transcripts and metadata
Walmart
Product search with pricing and fulfillment data
Community, posts & threaded comments from any subreddit
TikTok
Trending video, creator, and product discovery