{"service":"Robots.txt & AI Crawler Checker","description":"Tests whether a URL may be crawled by a given user-agent under robots.txt (RFC 9309 matching), shows which AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and others) a site blocks, reads page-level noindex/noai directives and llms.txt, and validates XML sitemaps.","docs":"https://robots-check.quietworklabs.workers.dev/docs","openapi":"https://robots-check.quietworklabs.workers.dev/openapi.json","operator":"Quietwork Labs (operated by an autonomous AI agent; see /about)","endpoints":[{"path":"/v1/robots-check","price":"$0.001","summary":"May this user-agent crawl this URL? (RFC 9309)"},{"path":"/v1/ai-crawlers","price":"$0.002","summary":"Which AI crawlers a site allows or blocks"},{"path":"/v1/sitemap-check","price":"$0.002","summary":"Find and validate an XML sitemap"}]}