An r/n8n thread mentioned the OP was using 'Google Custom Search plus manual scraping' and wanted a single API. This tutorial walks the replacement path. Search moves to POST /api/v2/google, and the manual scraping half moves to Scavio's Extract endpoint (POST /api/v1/extract), which returns any page as Markdown, HTML or text under the same key.
API Docs: Extract API, Crawl API
Prerequisites
- Python 3.10+
- Scavio API key
Walkthrough
Step 1: Identify Google Custom Search calls
Usually with a CSE key + cx ID.
Python
# Before:
# r = requests.get('https://www.googleapis.com/customsearch/v1', params={'key': KEY, 'cx': CX, 'q': q})Step 2: Replace with Scavio
No CX needed; Scavio searches the open web.
Python
# After:
r = requests.post('https://api.scavio.dev/api/v2/google',
headers={'Authorization': f'Bearer {SCAVIO_API_KEY}'},
json={'query': q}).json()Step 3: Map response
items[] becomes organic_results[].
Python
# Google CSE: r['items'][i]['link']
# Scavio: r['organic_results'][i]['link']Step 4: Add extract endpoint for content
Replaces the 'manual scraping' half of the OP's flow.
Python
# Scavio Extract reads one URL and returns the page as markdown, html or text.
# mode: "normal" (1 credit), "advanced" renders JavaScript (1 credit),
# "ultra" for the hardest pages (2 credits). Only a successful call is billed.
# Docs: https://scavio.dev/docs/extract
def scavio_extract(url, headers, mode="normal", fmt="markdown"):
r = requests.post("https://api.scavio.dev/api/v1/extract", headers=headers,
json={"url": url, "format": fmt, "mode": mode}, timeout=90)
r.raise_for_status()
return r.json()["data"]["content"]
def fetch(url):
return scavio_extract(url, {'Authorization': f'Bearer {SCAVIO_API_KEY}'})Step 5: Compare quotas
Google CSE caps at 100/day on free, $5/1K above. Scavio free is 50 credits on signup + $30/mo for 7K.
Text
// Daily research agent making 50 queries: Google CSE = $7.50/mo above quota; Scavio = $30/mo flat once the 50 free signup credits are used.Python Example
Python
# Migration takes ~20 minutes for a typical agent.JavaScript Example
JavaScript
// Same in TS.Expected Output
JSON
Same query intent, structured JSON, plus extract endpoint that replaces 'manual scraping' under the same key. No more two-vendor split.