The Problem
Local LLMs (Llama, Mistral, Qwen running via Ollama, vLLM, or llama.cpp) have no built-in web access. They hallucinate freely on any question about current events, prices, or recent releases. Adding web grounding requires a search API that returns structured data a local model can consume. Most search APIs target cloud-hosted models and assume you are running OpenAI-compatible tool calling, which local models often do not support reliably.
The Scavio Solution
Build a simple retrieval layer that queries Scavio before each LLM call, formats the results as context, and prepends them to the prompt. No tool calling required. The pattern works with any local model because it is just text-in, text-out. Scavio returns structured JSON that you format into a context block. The local model sees grounding data in its prompt window and answers with facts instead of hallucinations.
Before
Before web grounding, the local Llama model confidently stated outdated pricing, invented product features, and fabricated URLs. Users learned to distrust the model for anything time-sensitive, limiting it to creative and coding tasks only.
After
After adding Scavio grounding, the model answers time-sensitive questions with cited data. Users trust it for price checks, news summaries, and product research. Hallucination rate on factual questions dropped from roughly 40% to under 5%.
Who It Is For
Developers running local LLMs via Ollama, vLLM, or llama.cpp who need web grounding without cloud dependencies. Privacy-conscious users who want search-grounded answers without sending data to OpenAI.
Key Benefits
- Works with any local model: no tool calling or function calling required
- Simple context prepend pattern compatible with Ollama, vLLM, llama.cpp
- Hallucination rate on factual questions drops from ~40% to under 5%
- Free 250 queries/month covers personal local LLM use
- AI Overview text provides pre-summarized context that fits small context windows
Python Example
import requests
import json
API_KEY = "your_scavio_api_key"
OLLAMA_URL = "http://localhost:11434/api/generate"
def grounded_query(question: str, model: str = "llama3.2") -> str:
# Get web context from Scavio
search_res = requests.post(
"https://api.scavio.dev/api/v1/search",
headers={"x-api-key": API_KEY},
json={"platform": "google", "query": question, "ai_overview": True},
timeout=15,
)
search_res.raise_for_status()
data = search_res.json()
# Build context block
context_parts = []
if data.get("ai_overview"):
context_parts.append(f"AI Overview: {data['ai_overview']['text']}")
for r in data.get("organic", [])[:3]:
context_parts.append(f"- {r.get('title', '')}: {r.get('snippet', '')}")
context = "\n".join(context_parts)
# Query local LLM with context
prompt = f"Use the following web search results to answer the question.\n\nSearch Results:\n{context}\n\nQuestion: {question}\nAnswer:"
llm_res = requests.post(OLLAMA_URL, json={"model": model, "prompt": prompt, "stream": False}, timeout=60)
return llm_res.json().get("response", "")
answer = grounded_query("What is the latest Ollama version in 2026?")
print(answer)JavaScript Example
const API_KEY = "your_scavio_api_key";
async function groundedQuery(question, model = "llama3.2") {
const searchRes = await fetch("https://api.scavio.dev/api/v1/search", {
method: "POST",
headers: { "x-api-key": API_KEY, "content-type": "application/json" },
body: JSON.stringify({ platform: "google", query: question, ai_overview: true }),
});
const data = await searchRes.json();
const parts = [];
if (data.ai_overview) parts.push(`AI Overview: ${data.ai_overview.text}`);
for (const r of (data.organic ?? []).slice(0, 3)) parts.push(`- ${r.title}: ${r.snippet}`);
const prompt = `Use the following web search results to answer the question.\n\nSearch Results:\n${parts.join("\n")}\n\nQuestion: ${question}\nAnswer:`;
const llmRes = await fetch("http://localhost:11434/api/generate", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ model, prompt, stream: false }),
});
return (await llmRes.json()).response ?? "";
}
console.log(await groundedQuery("What is the latest Ollama version in 2026?"));Platforms Used
Web search with knowledge graph, PAA, and AI overviews