r/framer is full of threads about 'LLMs can't read Framer sites' because Framer ships JavaScript-heavy pages that LLM crawlers cannot render. Same problem affects single-page React apps. This tutorial walks through auditing your site to flag LLM readability issues before they hurt your ChatGPT and Perplexity visibility.
Prerequisites
- Python 3.8+
- A Scavio API key
- A site URL to audit
Walkthrough
Step 1: Fetch the rendered HTML
Use Scavio to fetch the fully rendered page (post-JS).
Python
import requests, os
def fetch_rendered(url):
r = requests.post('https://api.scavio.dev/api/v1/extract',
headers={'x-api-key': os.environ['SCAVIO_API_KEY']},
json={'url': url, 'render_js': True})
return r.json()Step 2: Fetch the raw HTML (pre-JS)
Fetch without JS rendering to see what LLM crawlers actually see.
Python
def fetch_raw(url):
r = requests.post('https://api.scavio.dev/api/v1/extract',
headers={'x-api-key': os.environ['SCAVIO_API_KEY']},
json={'url': url, 'render_js': False})
return r.json()Step 3: Compare text extraction
Count words in each. If raw is a fraction of rendered, you have a problem.
Python
from bs4 import BeautifulSoup
def count_words(html):
return len(BeautifulSoup(html, 'html.parser').get_text().split())
def llm_readable_ratio(url):
raw = fetch_raw(url)
rendered = fetch_rendered(url)
rw = count_words(raw.get('html', ''))
rnw = count_words(rendered.get('html', ''))
return rw / rnw if rnw else 0Step 4: Check meta and structured data
LLMs love title, description, and JSON-LD. Make sure they are in the raw HTML.
Python
def check_metadata(raw_html):
soup = BeautifulSoup(raw_html, 'html.parser')
return {
'title': bool(soup.title),
'description': bool(soup.find('meta', {'name': 'description'})),
'og_tags': bool(soup.find('meta', {'property': 'og:title'})),
'json_ld': bool(soup.find('script', {'type': 'application/ld+json'}))
}Step 5: Report the audit
Print a summary with readability ratio and metadata presence.
Python
def audit(url):
raw = fetch_raw(url)
ratio = llm_readable_ratio(url)
meta = check_metadata(raw.get('html', ''))
print(f'URL: {url}')
print(f'LLM-readable ratio: {ratio:.0%}')
print(f'Metadata: {meta}')
if ratio < 0.5:
print('WARNING: Most content is hidden behind JavaScript. LLMs will miss it.')Python Example
Python
import os, requests
from bs4 import BeautifulSoup
API_KEY = os.environ['SCAVIO_API_KEY']
def fetch(url, render):
r = requests.post('https://api.scavio.dev/api/v1/extract',
headers={'x-api-key': API_KEY},
json={'url': url, 'render_js': render})
return r.json().get('html', '')
def audit(url):
raw_words = len(BeautifulSoup(fetch(url, False), 'html.parser').get_text().split())
rendered_words = len(BeautifulSoup(fetch(url, True), 'html.parser').get_text().split())
ratio = raw_words / rendered_words if rendered_words else 0
print(f'{url}: {ratio:.0%} LLM-readable')
if ratio < 0.5:
print(' Problem: LLMs only see half of the content.')
audit('https://example.framer.com')JavaScript Example
JavaScript
async function fetchHtml(url, renderJs) {
const r = await fetch('https://api.scavio.dev/api/v1/extract', {
method: 'POST',
headers: { 'x-api-key': process.env.SCAVIO_API_KEY, 'Content-Type': 'application/json' },
body: JSON.stringify({ url, render_js: renderJs })
});
return (await r.json()).html;
}
async function audit(url) {
const raw = await fetchHtml(url, false);
const rendered = await fetchHtml(url, true);
const rw = raw.replace(/<[^>]+>/g, ' ').split(/\s+/).length;
const rnw = rendered.replace(/<[^>]+>/g, ' ').split(/\s+/).length;
console.log(`${url}: ${(rw / rnw * 100).toFixed(0)}% LLM-readable`);
}
audit('https://example.framer.com');Expected Output
JSON
https://example.framer.com: 12% LLM-readable
Problem: LLMs only see half of the content. Consider SSR, Next.js app router, or a static hero fallback.