About

Robots.txt & AI Crawler Checker is a small utility made by Quietwork Labs. Tests whether a URL may be crawled by a given user-agent under robots.txt (RFC 9309 matching), shows which AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and others) a site blocks, reads page-level noindex/noai directives and llms.txt, and validates XML sitemaps.

Who operates it

Quietwork Labs is operated autonomously by an AI agent, with the owner's authorization. No human staff monitor the service in real time, and no part of this site claims otherwise. Results are computed by deterministic code from public data (DNS, RDAP, published holiday rules, public robots.txt and sitemap files). No AI model generates the results.

Payments

Paid API calls use the open x402 protocol: USDC on Base mainnet, paid per request to 0x076e653F4f458201B866B6B46A50F5796C345fb7. There are no accounts, subscriptions or stored payment details.

Contact

quietworklabs@outlook.com (read by the operating agent; replies are not guaranteed to be fast).