Cache Search Results to Cut Agent Token Costs

The Problem

A support or research agent answers the same questions over and over. "What are your hours?", "how do I reset my password?", "is X compatible with Y?" come in dozens of times a day, and every single one fires a fresh Google SERP call plus a fresh summarization pass through the model. You pay twice: once in search credits for a result you already fetched an hour ago, and again in input and output tokens to re-read and re-summarize the exact same page snippets. With Scavio's Google SERP endpoint at 1 credit per call (2 credits when light_request is false for knowledge graph, people also ask, and related searches), a chatty agent burns through 7,000 monthly credits faster than it should, and the model bill grows in lockstep because each repeated query re-summarizes thousands of tokens of retrieved context.

The Scavio Solution

Put a keyed cache in front of the search call so repeated queries skip both the live API request and the re-summarization tokens. Normalize the user's intent into a stable cache key (lowercase, trimmed, maybe a hash of the canonical question), then look it up. On a hit you return the stored answer directly: zero Scavio credits, zero model tokens for that turn. On a miss you call POST https://api.scavio.dev/api/v1/google with Bearer auth, summarize once, and store the finished answer under the key with a TTL. The honest tradeoff: caching is wrong for anything that must be real-time, like breaking news, live prices, or stock and sports scores. For those intents set a very short TTL or bypass the cache entirely so you never serve a stale number. For evergreen FAQ-style questions, a TTL of hours or days is safe and is where the savings come from.

Who It Is For

Teams running support, FAQ, or research agents that field the same intents repeatedly and watch both their search-API credits and their model token bill climb. If your agent re-answers "how do I..." questions all day, a cache pays for itself.

Key Benefits

• A cache hit costs zero Scavio credits and zero model tokens for that turn
• Repeated FAQ-style queries stop double-paying for search and re-summarization
• Per-intent TTL keeps real-time queries fresh while evergreen answers stay cached
• Bypass flag lets breaking-news and live-price intents skip the cache cleanly
• Your 7,000 monthly Project credits cover far more unique questions

Python Example

import time, hashlib, requests

API_KEY = "your_scavio_api_key"
URL = "https://api.scavio.dev/api/v1/google"
_cache = {}  # swap for Redis in production

def _key(query: str) -> str:
    norm = query.strip().lower()
    return hashlib.sha256(norm.encode()).hexdigest()

def cached_search(query: str, ttl: int = 3600, bypass: bool = False) -> dict:
    k = _key(query)
    now = time.time()
    if not bypass and k in _cache:
        value, expires = _cache[k]
        if now < expires:
            return value  # hit: no Scavio call, no model tokens
    resp = requests.post(
        URL,
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={"query": query, "light_request": False},
        timeout=15,
    )
    resp.raise_for_status()
    data = resp.json()
    _cache[k] = (data, now + ttl)
    return data

# Evergreen FAQ: cache for an hour. Live prices: ttl=0 or bypass=True
print(cached_search("how to reset password", ttl=3600))
print(cached_search("btc price right now", ttl=0, bypass=True))

JavaScript Example

import crypto from "node:crypto";

const API_KEY = "your_scavio_api_key";
const URL = "https://api.scavio.dev/api/v1/google";
const cache = new Map(); // swap for Redis in production

const keyFor = (query) =>
  crypto.createHash("sha256").update(query.trim().toLowerCase()).digest("hex");

async function cachedSearch(query, { ttl = 3600, bypass = false } = {}) {
  const k = keyFor(query);
  const now = Date.now();
  if (!bypass && cache.has(k)) {
    const { value, expires } = cache.get(k);
    if (now < expires) return value; // hit: no Scavio call, no model tokens
  }
  const res = await fetch(URL, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${API_KEY}`,
      "content-type": "application/json",
    },
    body: JSON.stringify({ query, light_request: false }),
  });
  if (!res.ok) throw new Error(`scavio ${res.status}`);
  const data = await res.json();
  cache.set(k, { value: data, expires: now + ttl * 1000 });
  return data;
}

// Evergreen FAQ: cache for an hour. Live prices: ttl 0 or bypass true
console.log(await cachedSearch("how to reset password", { ttl: 3600 }));
console.log(await cachedSearch("btc price right now", { ttl: 0, bypass: true }));

Platforms Used