Anti-Bot Search MCP vs Paid SERP API: Build vs Buy

An r/mcp post launched a free anti-bot Google search MCP after benchmarking 6 existing free Google search MCPs and finding all of them broken. The build is impressive — warm Chrome with stealth profile, 4-worker pool, Mozilla Readability + Turndown, no API key, MIT licensed. The decision rule for whether to use it vs a paid SERP API is one question: who pays if it breaks at 2am?

Why free Google search MCPs keep breaking

Google's anti-bot detection is an arms race. Every few weeks they update fingerprint detection, challenge mechanisms, and rate limits in ways that invalidate stealth-Chrome setups. Small OSS maintainers can't track the changes at the cadence Google ships them. The 6 broken MCPs the OP benchmarked are a feature of the ecosystem, not a bug.

The OP's build is well-engineered

Persistent --user-data-dir for the warm profile (fresh Chrome launches detect; warm sessions don't as easily). 4-worker pool with auto-rebuild on detected blocks. Mozilla Readability for content extraction (better than raw HTML for LLM input). Turndown for HTML-to-markdown. Headless by default, visible only for demos. This is the right shape if you're going to build.

// Build path: puppeteer-extra + stealth, persistent profile
import puppeteer from 'puppeteer-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';
puppeteer.use(StealthPlugin());

const browser = await puppeteer.launch({
  headless: 'new',
  userDataDir: './chrome-profile',  // persist across runs
  args: ['--no-sandbox'],
});

The decision rule

• Hobby project, learning, side-project bot — build path is fine. Free, occasionally breaks, you maintain it.
• Production agent, paid customers, ops-critical — buy a hosted SERP API. Vendor maintains the parsers and absorbs the arms race.

The buy path

# Hosted, multi-platform, free 500 calls/mo
claude mcp add scavio https://mcp.scavio.dev/mcp \
  --header 'x-api-key: $SCAVIO_API_KEY'

Free tier (500 calls/mo, no card) overlaps the hobby/learning use cases where the build path also wins. Paid tier ($30/mo for 7K credits) covers production volumes. Either way, no anti-bot maintenance burden.

What multi-platform buys you that the build path doesn't

The OP's build covers Google. Production agents almost always end up needing Reddit signal, YouTube comments, Amazon product data, or Walmart inventory within a quarter. Building each of those as a separate stealth-browser MCP is a 4-month engineering project. Hosted SERP APIs that cover multiple platforms (Scavio: Google + Reddit + YouTube + Amazon + Walmart + extract) buy you time on the wrong problem.

What the build path teaches

For developers who want to understand anti-bot, the build path is genuinely educational. Read the OP's repo. Understand how stealth fingerprinting works, why warm profiles matter, why a worker pool is better than serial calls. That knowledge transfers to every web-data project. Then ship production on a paid API; you understand why you're paying.

Cost math at common volumes

• Personal project, 500 queries/mo: free either path.
• Side project, 5K queries/mo: $30 Scavio, or 30-50 hours engineering tax on the build path.
• Production agent, 50K queries/mo: ~$215 Scavio, or 50-100 hours engineering tax + reactive maintenance.
• SaaS with paying customers: paid API, no question. Maintenance burden compounds against the customer SLA.

Honest credit to the OP's project

Anti-bot stealth MCPs are useful for personal Claude Code agents that need "just enough" Google search and where 2am downtime doesn't cost anything. The OP filled that slot. Use it. Just don't put a paid customer behind it. The free plan from a hosted vendor is the better default for anything that smells like production.

The boring lesson

Build for learning, buy for production. The ecosystem has clear products on both sides; pick by the failure cost, not the per-month subscription line. A $30/mo subscription that doesn't break the morning of a customer launch is cheaper than the engineer-hours of patching a stealth-Chrome setup under deadline pressure.