The Problem
Playwright and Puppeteer scripts that scrape search engines and marketplaces break regularly when sites change their HTML structure. A Google SERP layout change breaks selectors across your entire pipeline. Amazon product page redesigns require emergency script updates. Each break causes data gaps, on-call alerts, and 2-8 hours of engineering time to fix. The team spends more time maintaining scrapers than building product features.
The Scavio Solution
Replace browser automation scripts with Scavio API calls for all search engine and marketplace data. The API returns structured JSON that never changes format due to site redesigns. Your pipeline sends POST requests instead of launching browsers. The response schema is documented and versioned. Keep Playwright only for sites that require browser interaction (login, checkout, form submission).
Before
Before migration, the pipeline ran 12 Playwright scripts across Google, Amazon, and Walmart. Average time between selector breaks: 2-3 weeks. Monthly maintenance: 10 hours. The on-call rotation included 'scraper babysitting' as a standing responsibility.
After
After migration, 8 of 12 scripts were replaced with Scavio API calls. Selector breaks dropped from biweekly to zero for migrated scripts. Monthly maintenance dropped from 10 hours to 2 hours (remaining 4 Playwright scripts for non-search sites). On-call scraper pages disappeared.
Who It Is For
Engineering teams maintaining Playwright/Puppeteer scrapers for search engines and marketplaces. Anyone whose on-call rotation includes scraper maintenance as a regular responsibility.
Key Benefits
- Zero selector maintenance for search and marketplace data
- Documented, versioned JSON schema instead of fragile DOM selectors
- Faster execution: 1s API call vs 3-10s browser page load
- No browser instances to manage, no memory leaks to debug
- Keep Playwright only for sites that truly need browser interaction
Python Example
import requests
API_KEY = "your_scavio_api_key"
# Before: Playwright script (breaks every 2-3 weeks)
# async with async_playwright() as p:
# browser = await p.chromium.launch()
# page = await browser.new_page()
# await page.goto(f"https://www.google.com/search?q={query}")
# results = await page.query_selector_all("div.g") # breaks on layout changes
# for r in results:
# title = await r.query_selector("h3") # breaks on heading changes
# link = await r.query_selector("a") # breaks on link structure changes
# After: Scavio API (never breaks from site changes)
def search(query: str, platform: str = "google") -> list[dict]:
res = requests.post(
"https://api.scavio.dev/api/v1/search",
headers={"x-api-key": API_KEY},
json={"platform": platform, "query": query, "ai_overview": True},
timeout=15,
)
res.raise_for_status()
data = res.json()
return [{
"title": r.get("title", ""),
"link": r.get("link", ""),
"snippet": r.get("snippet", ""),
"position": r.get("position", i + 1),
} for i, r in enumerate(data.get("organic", []))]
results = search("best search api 2026")
for r in results[:5]:
print(f"#{r['position']} {r['title']}")
print(f" {r['link']}")JavaScript Example
const API_KEY = "your_scavio_api_key";
// Before: Playwright (breaks biweekly)
// const page = await browser.newPage();
// await page.goto(`https://www.google.com/search?q=${query}`);
// const results = await page.$$eval("div.g", ...) // fragile selectors
// After: Scavio API (stable JSON schema)
async function search(query, platform = "google") {
const res = await fetch("https://api.scavio.dev/api/v1/search", {
method: "POST",
headers: { "x-api-key": API_KEY, "content-type": "application/json" },
body: JSON.stringify({ platform, query, ai_overview: true }),
});
if (!res.ok) throw new Error(`scavio ${res.status}`);
const data = await res.json();
return (data.organic ?? []).map((r, i) => ({ title: r.title ?? "", link: r.link ?? "", snippet: r.snippet ?? "", position: r.position ?? i + 1 }));
}
const results = await search("best search api 2026");
for (const r of results.slice(0, 5)) console.log(`#${r.position} ${r.title}\n ${r.link}`);Platforms Used
Web search with knowledge graph, PAA, and AI overviews
Amazon
Product search with prices, ratings, and reviews
Walmart
Product search with pricing and fulfillment data
YouTube
Video search with transcripts and metadata