Pricing
The page title and description describe the checker as free. No paid plans are listed on the page.
Learn moreAI Ranked
Paste a robots.txt and see which of 21 AI crawlers it blocks
Intermediate
What is AI Ranked?
AI Ranked is an analytics tool. A free browser-based checker that tests a pasted robots.txt against 21 AI crawlers, shows which rule allows or blocks each one, and generates a recommended config for search and training policies.
AI Ranked's robots.txt checker answers a narrow but important question: which AI crawlers can actually read a given site. A user pastes the contents of their robots.txt, picks a path to test, and receives a table covering 21 AI user agents from OpenAI, Anthropic, Perplexity, Google, Apple, Microsoft, Meta, Common Crawl, ByteDance, Amazon, DuckDuckGo and Mistral AI. Each row shows the bot's purpose (training, AI search or user fetch) and whether it is allowed or blocked, along with the exact line that decided the outcome. The file has to be pasted because browsers block cross-site reads (CORS), so the page cannot fetch a robots.txt on its own. The page states that nothing pasted leaves the browser. Matching follows RFC 9309: consecutive User-agent lines form a group, a bot obeys only the group that names it, the longest matching path wins, and Allow beats Disallow on a tie. Wildcard and end-anchor characters are supported. Beyond the verdict table, the tool generates a recommended configuration with two policies: allow AI search, user fetches and training, or allow search and fetches while blocking training. The snippet copies existing wildcard rules into the named groups so a bot with its own group does not gain access to private paths. Below the tool sits editorial material: a worked example comparing The Guardian and The New York Times, a priority guide for which blocked rows to fix first, a decision rule on blocking GPTBot, and four common accidental-blocking mistakes, including a leftover staging rule and CDN-level bot blocking. The tool suits SEO professionals, site owners and B2B software marketers who want to stay citable in AI answers while controlling model training. It is narrower than a full AI visibility platform, since it audits crawler access only, but it links to a citability grader and a comparison of free visibility checkers for the next steps. Among alternatives, it fits as a quick, free, no-signup diagnostic.
How do you use AI Ranked?
1Open your robots.txt
Visit yourdomain.com/robots.txt in a new browser tab and copy the whole file. It always sits at the root of the domain.2Paste the file into the checker
Paste the contents into the Your robots.txt box. Nothing pasted leaves the browser.
3Set the path to test
Enter a path such as the pricing page, a top blog post or a docs page rather than only the root.4Read the status table
Review which of the 21 agents are allowed or blocked, and check the rule shown for each. Fix blocked AI search agents first.5Apply the recommended config
Choose a policy, delete old groups for the same bots, then copy the generated block into robots.txt.
Pros and cons
Pros
- Tests 21 AI user agents at once and labels each as training, AI search or user fetchAI
- Shows the exact robots.txt line that decided every allowed or blocked resultAI
- Follows RFC 9309 matching rules, including longest-match and wildcard handlingAI
- Generates a copy-ready config that preserves existing wildcard rules in named groupsAI
- Pasted content stays in the browser, and the page is free to useAI
Cons
- The file must be copied and pasted by hand, since the page cannot fetch robots.txt itselfAI
- Checks robots.txt rules only, so CDN or firewall bot blocking goes undetectedAI
- Tests one path at a time, so several key pages require repeated runsAI
- Does not audit AI visibility or citation results, only crawler accessAI
- The recommended snippet uses a placeholder sitemap URL that must be edited before useAI
How much does AI Ranked cost?
Features
Checks a pasted robots.txt against 21 AI user agents, shows the deciding rule per bot, tests any path, flags training versus search versus user-fetch crawlers, and generates a recommended config for two policies.
Frequently asked questions about AI Ranked
How much does AI Ranked cost?
The page title and description describe the checker as free. No paid plans are listed on the page.How do you use AI Ranked?
The walkthrough on this page covers 5 steps: 1. Open your robots.txt 2. Paste the file into the checker 3. Set the path to test 4. Read the status table 5. Apply the recommended config.What platforms does AI Ranked support?
AI Ranked is available on Web App.What are the limitations of AI Ranked?
The file must be copied and pasted by hand, since the page cannot fetch robots.txt itself. Checks robots.txt rules only, so CDN or firewall bot blocking goes undetected. Tests one path at a time, so several key pages require repeated runs.
