Tutorial

How to Replace a Headless Browser with a Search API

When the target is publicly indexed, structured search beats Playwright. Practical examples: Amazon, Reddit, YouTube, Google Shopping.

An r/webdev thread asked what browser automations work in 2026. The often-overlooked answer is 'skip the browser when the target is indexed.' This tutorial walks four common cases where Scavio replaces a Playwright script entirely.

Prerequisites

  • Python 3.10+
  • Scavio API key

Walkthrough

Step 1: Case 1: Amazon product listing

Replace 200 lines of Playwright + proxy with one POST.

Python
import requests, os
API_KEY = os.environ['SCAVIO_API_KEY']

def amazon(asin):
    return requests.post('https://api.scavio.dev/api/v1/amazon/product',
        headers={'x-api-key': API_KEY}, json={'asin': asin}).json()

Step 2: Case 2: Reddit thread + comments

Threads with full comment trees as JSON.

Python
def reddit_thread(url):
    return requests.post('https://api.scavio.dev/api/v1/reddit/thread',
        headers={'x-api-key': API_KEY}, json={'url': url}).json()

Step 3: Case 3: YouTube video metadata

No need to render the page.

Python
def yt_video(url):
    return requests.post('https://api.scavio.dev/api/v1/youtube/video',
        headers={'x-api-key': API_KEY}, json={'url': url}).json()

Step 4: Case 4: Google Shopping prices

Browser would render JS for prices; SERP returns them structured.

Python
def shopping(q):
    return requests.post('https://api.scavio.dev/api/v1/search',
        headers={'x-api-key': API_KEY}, json={'query': q, 'search_type': 'shopping'}).json()

Step 5: When to keep the browser

Auth-gated dashboards, JS-only SPAs, pixel-perfect screenshots — these still need Playwright/Browserbase/Stagehand.

Text
// Decision rule: if a curl + JSON parse works on the public URL, skip the browser. If not, use a cloud Playwright vendor.

Python Example

Python
# Use the cases above as drop-in replacements for the corresponding Playwright scripts.

JavaScript Example

JavaScript
// Same pattern in TS.

Expected Output

JSON
Scrapers that took 30-60s per run with proxies and retries return in 1-3s with no rotation infrastructure. Cost drops from $0.05+ per run (proxy + browser) to $0.0043.

Related Tutorials

Frequently Asked Questions

Most developers complete this tutorial in 15 to 30 minutes. You will need a Scavio API key (free tier works) and a working Python or JavaScript environment.

Python 3.10+. Scavio API key. A Scavio API key gives you 500 free credits per month.

Yes. The free tier includes 500 credits per month, which is more than enough to complete this tutorial and prototype a working solution.

Scavio has a native LangChain package (langchain-scavio), an MCP server, and a plain REST API that works with any HTTP client. This tutorial uses the raw REST API, but you can adapt to your framework of choice.

Start Building

When the target is publicly indexed, structured search beats Playwright. Practical examples: Amazon, Reddit, YouTube, Google Shopping.