# doesitrun is a free public tool: search engines and AI assistants are welcome on all public pages. # Crawlers that only collect training or resale data are excluded; this is a reservation under Art. 4(3) EU Directive 2019/790 / ยง 44b(3) UrhG. User-agent: GPTBot Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: OAI-SearchBot Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: ChatGPT-User Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: ClaudeBot Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: Claude-SearchBot Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: Claude-User Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: anthropic-ai Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: PerplexityBot Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: Perplexity-User Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: Google-Extended Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: Applebot Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: Applebot-Extended Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: Amazonbot Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: Meta-ExternalAgent Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: YouBot Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: CCBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Diffbot Disallow: / User-agent: cohere-ai Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: serpstatbot Disallow: / User-agent: AhrefsBot Crawl-delay: 5 Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: SemrushBot Crawl-delay: 5 Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: SiteCheckerBotCrawler Crawl-delay: 5 Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: Barkrowler Crawl-delay: 5 Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: rogerbot Crawl-delay: 5 Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: PetalBot Crawl-delay: 5 Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: SeekportBot Crawl-delay: 5 Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline User-agent: * Allow: / Disallow: /api/ Disallow: /admin Disallow: /go Disallow: /auth/ Disallow: /f/ Disallow: /s/ Disallow: /offline Sitemap: https://doesitrun.com/sitemap.xml