Our crawler

WhiteTieCheck crawler

November 2026

When someone runs a check on a website, we make a small number of requests to that site to read publicly available information. Requests identify themselves as:

WhiteTieCheck/1.0 (+https://whitetie.ai/bot)

What it fetches

  • Only the website submitted for the check: the homepage, an About page, robots.txt and llms.txt.
  • It never fetches competitors' sites, never runs JavaScript and never tries to get around a block, challenge or rate limit.
  • It respects robots.txt rules for WhiteTieCheck.

Blocking it

Add this to your robots.txt:

User-agent: WhiteTieCheck
Disallow: /

If your site blocks our request, the check reports that as a finding, because AI assistants may be blocked too.

Questions: hello@whitetieseo.com.