Check whether your site blocks or allows every major AI crawler — training (GPTBot, ClaudeBot), search/citation (OAI-SearchBot, PerplexityBot), and live user fetches (ChatGPT-User) — in one read of your robots.txt.
Enter a domain above, or call the API directly: GET https://can-ai-read.lastminutedeals.workers.dev/check?url=example.com
Read your site's /robots.txt and check the rules for each AI user-agent (GPTBot, ClaudeBot, OAI-SearchBot, PerplexityBot, Google-Extended, ChatGPT-User, and others). A Disallow rule matching '/' for a given agent blocks it. Because the agents do different jobs — training, search indexing, or live user fetches — you must check each one; blocking one does not block the others. This tool fetches your live robots.txt and reports the status of every major AI crawler at once.
No — and this is the most common mistake. GPTBot is OpenAI's TRAINING crawler. ChatGPT's SEARCH results are gathered by a different agent, OAI-SearchBot, and live user requests by ChatGPT-User. If you block only GPTBot, ChatGPT can still cite your pages in search. But if you block OAI-SearchBot (or blanket-block all OpenAI agents), you remove yourself from ChatGPT search answers and lose that referral traffic. Keep training and search crawlers separate in robots.txt.
To stay visible and citable in AI answer engines while controlling training use: ALLOW the search/citation crawlers (OAI-SearchBot for ChatGPT, PerplexityBot for Perplexity, Claude-SearchBot for Claude) and the user-action fetchers (ChatGPT-User, Perplexity-User, Claude-User). If you want to keep your content out of model training, you can separately DISALLOW the training crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot) without harming your AI-search presence.
Deterministic — reads your live robots.txt, no AI model involved, nothing stored. Operated by Last Minute Deals HQ LLC.
Building agents that need to reach sites like this? See answers for AI-agent operators.