Why Do TikTok Profile Scrapers Keep Breaking? It Is the secUid, Not the Proxy

If your TikTok profile pipeline breaks every few weeks, the identifier layer is almost certainly the cause, not the proxy pool. Resolve each username to its secUid once through a call that handles the token layer for you, cache that secUid as the stable key, and gate every downstream read on the presence of the payload rather than the HTTP status. Scavio's /api/v1/tiktok/profile returns sec_uid from a plain username for 1 credit, and /api/v1/tiktok/user/posts takes that sec_uid directly, which removes both the page render and the token you were previously scraping out of it.

The reason this is worth spelling out: teams reach for a bigger proxy pool when the symptom appears, and the symptom is not a network symptom. A blocked request looks like a block. What actually happens here is quieter and more expensive.

secUid and msToken Are Not the Same Thing

Most TikTok pipelines carry two opaque strings, and they get treated as one category. They behave nothing alike.

secUid is an account identifier. It looks like this, captured live from the NASA account:

It always starts MS4wLjABAAAA, it is per-account, and it is stable. It is not a credential and it does not expire on a timer. It is simply which account you mean, in the form the profile-scoped endpoints want.

msToken is the opposite: a short-lived anti-bot token bound to a browser session. It rotates, it expires, and it is the thing that dies mid-run.

If you built a scraper that renders a profile page and regexes both values out of the HTML, you have coupled a stable identifier to an unstable one. Every time the msToken expires, you re-render the page to get a new one, and you re-extract the secUid you already had. Then TikTok changes the markup, the regex misses, and both go at once. That single design choice accounts for most of the breakage people attribute to detection.

The fix is unglamorous: get the secUid from something that is not a page parse, and store it.

Resolving a Username to a secUid

One call, no browser:

import os
import requests

API_KEY = os.environ["SCAVIO_API_KEY"]
BASE = "https://api.scavio.dev"
HEADERS = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}

r = requests.post(
    f"{BASE}/api/v1/tiktok/profile",
    headers=HEADERS,
    json={"username": "nasa"},
)
payload = r.json()
user = payload["data"]["user"]

print(user["sec_uid"])
print(user["uid"], user["unique_id"], user["follower_count"], user["aweme_count"])

Captured output, run against the live endpoint:

That call billed 1 credit and returned in 3.1 seconds. The user object carries 63 fields; sec_uid, uid, unique_id, follower_count, following_count, aweme_count, total_favorited and signature are the ones most pipelines actually consume.

Worth noting honestly: uid and sec_uid are different identifiers for the same account, and the posts endpoint wants the second one. Storing only the numeric uid is a common and annoying mistake, because it looks like the more canonical key and is not the one that works.

The Posts Call Takes secUid, Not a Username

SEC_UID = "MS4wLjABAAAAU9BRVzC8oCaegVnia8IbqWhPb_-dbU7s00Y3wS1_Nx8g5RUaYvyXrpejgjdxTwd6"

r = requests.post(
    f"{BASE}/api/v1/tiktok/user/posts",
    headers=HEADERS,
    json={"sec_user_id": SEC_UID, "count": 5},
)
data = r.json()["data"]

for v in data["aweme_list"]:
    s = v["statistics"]
    print(v["aweme_id"], s["play_count"], s["digg_count"], s["comment_count"], v["desc"][:40])

print("cursor:", data["max_cursor"], "has_more:", data["has_more"])

Captured output:

1 credit, 1.9 seconds, 155 fields per video. Two API-side details that will bite you if you assume otherwise:

• has_more is the integer 1, not a boolean. if data["has_more"] is True silently ends your pagination after one page.
• create_time is a Unix integer, not an ISO string.

Pass a username where sec_user_id belongs and you get a clean rejection, which is the good case:

{
  "error": "Invalid request: sec_user_id: must be a sec_user_id",
  "details": [{ "path": "sec_user_id", "message": "must be a sec_user_id" }]
}

That one costs nothing and tells you exactly what is wrong. The other failure modes do not.

The Failures Arrive as HTTP 200

This is the part that makes profile pipelines rot quietly, and it is worth being blunt about our own behaviour here rather than pretending the surface is cleaner than it is.

A truncated or stale secUid does not error. Send one and the response is HTTP 200:

{
  "data": {
    "status_code": 0,
    "status_msg": "No more videos",
    "version": "v2"
  },
  "credits_used": 1
}

There is no aweme_list key at all. status_code is 0, which normally means success. A naive reader does data.get("aweme_list", []), gets an empty list, and records "this creator posted nothing this week." Do that across a few thousand accounts and you have a dataset that is confidently wrong rather than obviously broken. And it bills.

A username that does not exist behaves the same way. /api/v1/tiktok/profile on a nonsense handle returns HTTP 200 with no user object:

{
  "data": { "statusCode": 10221, "statusMsg": "", "needFix": false },
  "credits_used": 1
}

10221 is TikTok's "user not found", surfaced as a 200 and charged as a successful call. We pass the upstream shape through here; the caller has to know the code. That is a wart, not a feature, and the guard below exists because of it.

The rule that follows from both: never branch on response.status_code for these endpoints. Branch on whether the payload you asked for is present.

class TikTokMiss(Exception):
    pass


def get_profile(username: str) -> dict:
    r = requests.post(
        f"{BASE}/api/v1/tiktok/profile", headers=HEADERS, json={"username": username}
    )
    r.raise_for_status()
    user = r.json().get("data", {}).get("user")
    if not user or not user.get("sec_uid"):
        raise TikTokMiss(f"no profile for {username}")
    return user


def get_posts(sec_uid: str, count: int = 20, cursor: str = "0") -> tuple[list, str, bool]:
    r = requests.post(
        f"{BASE}/api/v1/tiktok/user/posts",
        headers=HEADERS,
        json={"sec_user_id": sec_uid, "count": count, "cursor": cursor},
    )
    r.raise_for_status()
    data = r.json().get("data", {})
    if "aweme_list" not in data:
        raise TikTokMiss(f"bad or stale sec_uid: {sec_uid[:24]}...")
    return data["aweme_list"], str(data.get("max_cursor", "0")), data.get("has_more") == 1

"aweme_list" not in data is the load-bearing line. An account with genuinely zero videos returns the key with an empty list; a bad secUid omits the key entirely. That distinction is the difference between "nothing new" and "your identifier is dead", and it is invisible from the status code.

Cache the secUid, Not the Page

Once resolution is a real API call rather than a page parse, the economics change and the caching strategy becomes obvious.

Resolve a creator once, store sec_uid alongside the username, and every subsequent run costs 1 credit per page of posts instead of 2 calls per creator per run. For a roster of 500 creators polled daily, that is 500 profile calls in total rather than 500 every single day.

def sync_creator(conn, username: str) -> list:
    row = conn.execute(
        "select sec_uid from creators where username = ?", (username,)
    ).fetchone()

    if row is None:
        user = get_profile(username)
        conn.execute(
            "insert into creators (username, sec_uid, uid) values (?, ?, ?)",
            (username, user["sec_uid"], user["uid"]),
        )
        sec_uid = user["sec_uid"]
    else:
        sec_uid = row[0]

    try:
        posts, _, _ = get_posts(sec_uid)
    except TikTokMiss:
        # Only re-resolve on an actual miss. This is the rare path, not the hot one.
        user = get_profile(username)
        sec_uid = user["sec_uid"]
        conn.execute(
            "update creators set sec_uid = ? where username = ?", (sec_uid, username)
        )
        posts, _, _ = get_posts(sec_uid)

    return posts

The re-resolve path matters because secUid is stable, not immortal: an account that is deleted, renamed or made private will invalidate it. Handling that as an exception rather than as the default flow is what keeps the credit cost flat.

Both endpoints bill 1 credit per call, measured live rather than quoted from a table: credits_used: 1 on the profile call and credits_used: 1 on a 5-video posts page.

What This Does Not Fix

Being straight about the boundaries:

• This removes the token layer from your code. It does not remove rate limits. A tight loop over a large roster still needs concurrency control on your side.
• Follower and following reads are also secUid-scoped and paginate by cursor, so the same guard applies, but they are heavier calls than a posts page.
• Private accounts resolve to a profile and then return nothing useful downstream. That is TikTok's boundary, not something an API tier gets around.
• The response bodies are the raw upstream shape, verbose and nested. 63 fields on a profile, 155 per video. If you want a narrow row, project it yourself at the edge.

For what the identifier itself is and where else it turns up, the TikTok secUid glossary entry is the short version. The TikTok API overview lists the profile-scoped endpoints and their per-call cost.

The summary is one sentence: treat secUid as a cached database column and msToken as somebody else's problem, and the pipeline stops rotting on a schedule.