Agentic traffic (Claude, ChatGPT, Perplexity, agent frameworks) will hit your site more than human browsers by end of 2026. Standard analytics misses most of it. This tutorial adds a real-time agentic traffic classifier to any Node or edge runtime, with reverse lookup via Scavio to classify unknown user agents.
Prerequisites
- Node 20+ or edge runtime (Cloudflare Workers, Vercel)
- A Scavio API key
- Access to request headers/logs
Walkthrough
Step 1: Capture request signals
User-agent, IP, Accept headers are the baseline.
JavaScript
function captureSignals(req) {
return {
ua: req.headers['user-agent'] || '',
ip: req.headers['x-forwarded-for'] || req.ip,
accept: req.headers['accept'] || ''
};
}Step 2: Match known agent UAs
Start with a hard-coded list.
JavaScript
const KNOWN = [
/ChatGPT-User/, /GPTBot/, /ClaudeBot/, /PerplexityBot/, /OAI-SearchBot/,
/anthropic/i, /Claude-Web/, /Agent\//i
];
function isKnownAgent(ua) { return KNOWN.some(r => r.test(ua)); }Step 3: Reverse lookup unknown UAs
For unknown suspicious UAs, Scavio search confirms if it is a documented bot.
JavaScript
async function classifyUnknown(ua) {
const r = await fetch('https://api.scavio.dev/api/v1/search', {
method: 'POST',
headers: { 'x-api-key': process.env.SCAVIO_API_KEY, 'Content-Type': 'application/json' },
body: JSON.stringify({ query: `\"${ua}\" bot OR crawler OR agent\` })
});
const data = await r.json();
return data.organic_results?.some(o => /bot|crawler|agent/i.test(o.snippet));
}Step 4: Emit a structured event
Push classified hits to your log pipeline.
JavaScript
function emit(signals, type) {
console.log(JSON.stringify({ ts: Date.now(), type, ...signals }));
}Step 5: Build a live dashboard
Stream events into Grafana or a simple SQLite counter for a live count.
Bash
# SQLite counter example
sqlite3 agentic.db 'SELECT type, COUNT(*) FROM hits GROUP BY type;'Python Example
Python
import os, requests, re
API_KEY = os.environ['SCAVIO_API_KEY']
KNOWN = [r'ChatGPT-User', r'GPTBot', r'ClaudeBot', r'PerplexityBot']
def classify(ua):
if any(re.search(p, ua) for p in KNOWN):
return 'known_agent'
r = requests.post('https://api.scavio.dev/api/v1/search',
headers={'x-api-key': API_KEY},
json={'query': f'"{ua}" bot OR crawler'})
hits = r.json().get('organic_results', [])
return 'probable_agent' if hits else 'human'
print(classify('ChatGPT-User/1.0'))JavaScript Example
JavaScript
const API_KEY = process.env.SCAVIO_API_KEY;
const KNOWN = [/ChatGPT-User/, /GPTBot/, /ClaudeBot/, /PerplexityBot/];
export async function classify(ua) {
if (KNOWN.some(r => r.test(ua))) return 'known_agent';
const r = await fetch('https://api.scavio.dev/api/v1/search', {
method: 'POST',
headers: { 'x-api-key': API_KEY, 'Content-Type': 'application/json' },
body: JSON.stringify({ query: `"${ua}" bot OR crawler` })
});
const data = await r.json();
return data.organic_results?.length ? 'probable_agent' : 'human';
}Expected Output
JSON
Real-time stream of classified hits: known_agent, probable_agent, human. Typical finding: 30-45% of 2026 traffic is agentic on B2B SaaS landing pages.