Tutorial

How to Search Multiple Platforms in Parallel

Learn how to search Google, Reddit, YouTube, Amazon, and Walmart simultaneously using async parallel requests for faster data retrieval.

Searching multiple platforms sequentially takes 5-10 seconds as each API call waits for the previous one to complete. Searching in parallel cuts this to the latency of the slowest single call (typically 1-3 seconds). This tutorial implements parallel multi-platform search using Python asyncio and JavaScript Promise.all, both calling Scavio's API with the same key for all platforms.

Prerequisites

  • Python 3.8+ with aiohttp, or Node.js 18+
  • A Scavio API key from scavio.dev

Walkthrough

Step 1: Set up async search in Python

Install aiohttp and create an async search function.

Bash
pip install aiohttp

Step 2: Build parallel search function (Python)

Search all platforms simultaneously using asyncio.gather.

Python
import asyncio, aiohttp, os

API_KEY = os.environ['SCAVIO_API_KEY']
URL = 'https://api.scavio.dev/api/v1/search'

async def search_platform(session, platform: str, query: str) -> dict:
    async with session.post(URL, headers={'x-api-key': API_KEY, 'Content-Type': 'application/json'},
                           json={'platform': platform, 'query': query}) as resp:
        data = await resp.json()
        return {'platform': platform, 'results': data.get('organic', [])}

async def parallel_search(query: str, platforms: list = None) -> list:
    if platforms is None:
        platforms = ['google', 'reddit', 'youtube', 'amazon', 'walmart']
    async with aiohttp.ClientSession() as session:
        tasks = [search_platform(session, p, query) for p in platforms]
        return await asyncio.gather(*tasks)

# Usage:
results = asyncio.run(parallel_search('wireless earbuds'))
for r in results:
    print(f"{r['platform']}: {len(r['results'])} results")

Step 3: Build parallel search function (JavaScript)

Search all platforms simultaneously using Promise.all.

JavaScript
const URL = 'https://api.scavio.dev/api/v1/search';
const H = {'x-api-key': process.env.SCAVIO_API_KEY, 'Content-Type': 'application/json'};

async function parallelSearch(query, platforms = ['google', 'reddit', 'youtube', 'amazon', 'walmart']) {
  const results = await Promise.all(
    platforms.map(async platform => {
      const resp = await fetch(URL, {
        method: 'POST', headers: H,
        body: JSON.stringify({platform, query})
      });
      const data = await resp.json();
      return {platform, results: data.organic || []};
    })
  );
  return results;
}

const results = await parallelSearch('wireless earbuds');
results.forEach(r => console.log(`${r.platform}: ${r.results.length} results`));

Step 4: Merge and deduplicate results

Combine results from all platforms into a single ranked list.

Python
def merge_results(parallel_results: list) -> list:
    merged = []
    seen_urls = set()
    for pr in parallel_results:
        for r in pr['results']:
            url = r.get('link', r.get('url', ''))
            if url not in seen_urls:
                seen_urls.add(url)
                merged.append({**r, 'source_platform': pr['platform']})
    return merged

results = asyncio.run(parallel_search('wireless earbuds'))
merged = merge_results(results)
print(f'Total unique results: {len(merged)}')

Python Example

Python
import asyncio, aiohttp, os

async def parallel_search(query, platforms=['google', 'reddit', 'youtube']):
    async with aiohttp.ClientSession() as session:
        async def search(p):
            async with session.post('https://api.scavio.dev/api/v1/search',
                headers={'x-api-key': os.environ['SCAVIO_API_KEY'], 'Content-Type': 'application/json'},
                json={'platform': p, 'query': query}) as r:
                return {'platform': p, 'results': (await r.json()).get('organic', [])}
        return await asyncio.gather(*[search(p) for p in platforms])

results = asyncio.run(parallel_search('wireless earbuds'))

JavaScript Example

JavaScript
async function parallelSearch(query, platforms = ['google', 'reddit', 'youtube']) {
  return Promise.all(platforms.map(async p => {
    const r = await fetch('https://api.scavio.dev/api/v1/search', {
      method: 'POST', headers: {'x-api-key': process.env.SCAVIO_API_KEY, 'Content-Type': 'application/json'},
      body: JSON.stringify({platform: p, query})
    });
    return {platform: p, results: (await r.json()).organic || []};
  }));
}

Expected Output

JSON
Multi-platform search results returned in parallel, reducing total latency from sequential 5-10s to parallel 1-3s.

Related Tutorials

Frequently Asked Questions

Most developers complete this tutorial in 15 to 30 minutes. You will need a Scavio API key (free tier works) and a working Python or JavaScript environment.

Python 3.8+ with aiohttp, or Node.js 18+. A Scavio API key from scavio.dev. A Scavio API key gives you 500 free credits per month.

Yes. The free tier includes 500 credits per month, which is more than enough to complete this tutorial and prototype a working solution.

Scavio has a native LangChain package (langchain-scavio), an MCP server, and a plain REST API that works with any HTTP client. This tutorial uses the raw REST API, but you can adapt to your framework of choice.

Start Building

Learn how to search Google, Reddit, YouTube, Amazon, and Walmart simultaneously using async parallel requests for faster data retrieval.