Resumen
Diario ejecutar: per gov-doc topic, dork-search via Scavio for indexed paginas; route auth-gated objetivos to Playwright. Extraer structured registros.
Desencadenador
Diario cron 7am
Programación
Diario 7am
Pasos del flujo de trabajo
1
Cargar objetivo lista (dominio + topic)
From un YAML config o DB tabla.
2
Per objetivo: clasificar indexed vs auth-gated
Use un per-target marcar establecer durante onboarding.
3
Indexed: Scavio dorked search a traves de 4 plantillas
sitio:, filetype:, intitle:, inurl: variations.
4
Dedupe URLs a traves de plantillas
Same URL a traves de dorks = one fuente.
5
Scavio /extraer for top-N URLs
Markdown listo for LLM extraccion.
6
Auth-gated: Playwright/Stagehand obtener
Only el small subset ese requiere login.
7
LLM structured extraccion
Per markdown blob, return JSON {titulo, date, resumen, entities}.
8
Agregar to registros DB
Postgres / Sheets / etc.
Implementacion en Python
Python
import requests, os
H = {'x-api-key': os.environ['SCAVIO_API_KEY']}
DORKS = ['site:{d} filetype:pdf {t}', 'site:{d} intitle:{t}', 'site:{d} inurl:reports {t}']
def search_first(domain, topic):
urls = []
for tpl in DORKS:
q = tpl.format(d=domain, t=topic)
r = requests.post('https://api.scavio.dev/api/v1/search', headers=H, json={'query': q}).json()
urls.extend(o['link'] for o in r.get('organic_results', [])[:5])
return list(set(urls))Implementacion en JavaScript
JavaScript
// Same in TS.Plataformas utilizadas
Búsqueda web con grafo de conocimiento, PAA y resúmenes de IA