Tutorial

How to Build a GSC + MCP Analytics Pipeline

Combine Google Search Console data with live SERP checks to find ranking mismatches caused by AI Overviews and SERP features.

Google Search Console tells you where Google thinks you rank. Live SERP checks tell you where you actually rank right now, including SERP features GSC ignores. Combining both reveals keywords where GSC shows position 3 but live results show position 8 due to AI Overviews pushing organic results down. This tutorial validates GSC data with live Scavio checks and flags mismatches.

Prerequisites

  • Python 3.10+
  • google-auth and google-api-python-client
  • A Scavio API key from scavio.dev
  • GSC API access for your site

Walkthrough

Step 1: Pull top queries from GSC

Authenticate and pull top queries with reported positions.

Python
import os, requests

SK = os.environ['SCAVIO_API_KEY']
SH = {'x-api-key': SK, 'Content-Type': 'application/json'}

# Simulated GSC data (replace with real GSC API call)
gsc_data = [
    {'query': 'serp api python', 'gsc_position': 3.2, 'clicks': 120},
    {'query': 'web scraping api', 'gsc_position': 5.8, 'clicks': 85},
    {'query': 'tiktok data api', 'gsc_position': 2.1, 'clicks': 200},
]

Step 2: Validate with live SERP checks

For each GSC query, run a live check and compare positions.

Python
def live_rank(query, domain):
    data = requests.post('https://api.scavio.dev/api/v1/search',
        headers=SH, json={'query': query, 'country_code': 'us'}).json()
    for r in data.get('organic_results', []):
        if domain in r.get('link', ''): return r['position']
    return None

def validate(gsc_data, domain='mysite.com'):
    for row in gsc_data:
        live = live_rank(row['query'], domain)
        diff = live - row['gsc_position'] if live else None
        flag = f' MISMATCH ({diff:+.0f})' if diff and abs(diff) >= 3 else ''
        print(f"  {row['query']:30} GSC: {row['gsc_position']:.1f} Live: {live or 'N/A'}{flag}")
    print(f'Cost: ${len(gsc_data) * 0.005:.3f}')

validate(gsc_data)

Step 3: Check SERP features causing mismatches

Detect AI Overviews, PAA boxes, and snippets pushing organic down.

Python
def check_features(query):
    data = requests.post('https://api.scavio.dev/api/v1/search',
        headers=SH, json={'query': query, 'country_code': 'us'}).json()
    features = []
    if data.get('answer_box'): features.append('answer_box')
    if data.get('ai_overview'): features.append('ai_overview')
    if data.get('related_questions'): features.append(f"paa({len(data['related_questions'])})")
    return features

for row in gsc_data:
    features = check_features(row['query'])
    if features: print(f"  {row['query']}: {', '.join(features)}")

Step 4: Run full pipeline

Combine GSC pull, validation, and feature analysis.

Python
def run_pipeline():
    print('Validating GSC data...')
    validate(gsc_data)
    print('\nSERP features affecting rankings:')
    for row in gsc_data:
        f = check_features(row['query'])
        if f: print(f"  {row['query']}: {', '.join(f)}")
    print(f'\nTotal cost: ${len(gsc_data) * 2 * 0.005:.3f}')

run_pipeline()

Python Example

Python
import os, requests
SK = os.environ['SCAVIO_API_KEY']
SH = {'x-api-key': SK, 'Content-Type': 'application/json'}

def check(query, domain):
    data = requests.post('https://api.scavio.dev/api/v1/search',
        headers=SH, json={'query': query, 'country_code': 'us'}).json()
    pos = next((r['position'] for r in data.get('organic_results', []) if domain in r.get('link', '')), None)
    has_ao = bool(data.get('ai_overview'))
    print(f'{query}: pos={pos}, ai_overview={has_ao}')

check('serp api python', 'mysite.com')

JavaScript Example

JavaScript
const SK = process.env.SCAVIO_API_KEY;
const SH = { 'x-api-key': SK, 'Content-Type': 'application/json' };
async function check(query, domain) {
  const data = await fetch('https://api.scavio.dev/api/v1/search', {
    method: 'POST', headers: SH, body: JSON.stringify({ query, country_code: 'us' })
  }).then(r => r.json());
  const match = (data.organic_results || []).find(r => r.link.includes(domain));
  console.log(`${query}: pos=${match?.position || 'N/A'}, ai=${!!data.ai_overview}`);
}
check('serp api python', 'mysite.com').catch(console.error);

Expected Output

JSON
  serp api python                  GSC: 3.2 Live: 4
  web scraping api                  GSC: 5.8 Live: 9 MISMATCH (+3)
  tiktok data api                   GSC: 2.1 Live: 2
Cost: $0.015

  web scraping api: ai_overview, paa(4)

Related Tutorials

Frequently Asked Questions

Most developers complete this tutorial in 15 to 30 minutes. You will need a Scavio API key (free tier works) and a working Python or JavaScript environment.

Python 3.10+. google-auth and google-api-python-client. A Scavio API key from scavio.dev. GSC API access for your site. A Scavio API key gives you 250 free credits per month.

Yes. The free tier includes 250 credits per month, which is more than enough to complete this tutorial and prototype a working solution.

Scavio has a native LangChain package (langchain-scavio), an MCP server, and a plain REST API that works with any HTTP client. This tutorial uses the raw REST API, but you can adapt to your framework of choice.

Start Building

Combine Google Search Console data with live SERP checks to find ranking mismatches caused by AI Overviews and SERP features.