
AI Crawler Audit — Are You Blocking ChatGPT?
Check whether your site is visible to AI search or accidentally blocking it. Audits robots.txt for GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended and 7 more, detects llms.txt, and returns an AI-readiness score with the exact fixes. No key, no login.
AI Crawler Audit — Are You Blocking ChatGPT? returns 14 documented output fields, including domain, aiReadinessScore, aiSearchReadiness, blocksAiSearch and blockedSearchBots. Every field carries a declared type, a written description and an example value. 45 runs are recorded on Apify.
- 14
- Output fields
- 5
- Inputs
- 45
- Runs
- 2026-10-15
- Last run
01 — output
What it returns
Every field the extractor writes to its dataset, with its type, a written description and an example value.
| Field | Type | What it holds | Example |
|---|---|---|---|
| domain | stringnull | Domain name, without the scheme. | example.com |
| aiReadinessScore | integernull | How ready this site is for AI crawlers, from 0 to 100. | 0 |
| aiSearchReadiness | stringnull | Readiness for AI search in words. | <aiSearchReadiness> |
| blocksAiSearch | booleannull | Whether the site blocks AI search crawlers. | true |
| blockedSearchBots | stringnull | Search crawlers the site blocks. | <blockedSearchBots> |
| blockedBots | stringnull | User agents the site blocks. | <blockedBots> |
| hasLlmsTxt | booleannull | Whether the site publishes an llms.txt file. | true |
| llmsTitle | stringnull | Title the site publishes in its llms.txt. | <llmsTitle> |
| allowedBotCount | integernull | How many of the checked bots are allowed. | 0 |
| totalBotsChecked | integernull | How many bots were checked. | 0 |
| sitemapCount | integernull | How many sitemaps were found. | 0 |
| robotsStatus | integernull | Outcome of reading robots.txt. | 0 |
| recommendations | stringnull | The recommended actions for this row. | ["Declare a canonical URL","Publish a sitemap"] |
| status | stringnull | Outcome recorded for this row. | ok |
02 — sample
One record
Assembled from the example values declared on the schema above, not invented at render time.
{
"domain": "example.com",
"aiReadinessScore": 0,
"aiSearchReadiness": "<aiSearchReadiness>",
"blocksAiSearch": true,
"blockedSearchBots": "<blockedSearchBots>",
"blockedBots": "<blockedBots>",
"hasLlmsTxt": true,
"llmsTitle": "<llmsTitle>"
}03 — input
What you give it
| Input | Type | Description |
|---|---|---|
| domains | arrayrequired | Domains or URLs to audit. Scheme, www and path are stripped automatically. |
| onlyBlockedSites | booleanoptional | Filter the output down to the sites with a real problem — useful when auditing a long client list. |
| includeBotDetail | booleanoptional | Adds a botDetail array with the verdict and matched robots.txt rule for every AI crawler checked. |
| useProxy | booleanoptional | Route requests through Apify Proxy. Rarely needed — robots.txt is public. |
| maxItems | integeroptional | Hard limit, strictly enforced. You are never charged beyond this. |
04 — run
Run AI Crawler Audit on your own input
It runs in the cloud on Apify. Nothing to install, and you are billed per result returned rather than per attempt.
05 — questions
Asked before running it
14 of 14 fields described
- What does AI Crawler Audit — Are You Blocking ChatGPT? return?
- It returns 14 documented output fields: domain, aiReadinessScore, aiSearchReadiness, blocksAiSearch, blockedSearchBots, blockedBots, hasLlmsTxt, llmsTitle, allowedBotCount, totalBotsChecked, sitemapCount and robotsStatus. Each field declares its type, a description of what it holds, and an example value. The remaining 2 are listed in the output schema below.
- What input does AI Crawler Audit — Are You Blocking ChatGPT? require?
- domains is required. A further 4 inputs are optional. The full input schema, with types and defaults, is below.
- Does AI Crawler Audit — Are You Blocking ChatGPT? need an API key or login?
- No. None of its inputs is a credential, and it reads only public data. Nothing to sign up for before the first run.
- How do I run AI Crawler Audit — Are You Blocking ChatGPT??
- It runs on Apify. Start it from its listing with the inputs above, or call the Apify API with the Actor's ID. Results land in a dataset you can export as JSON, CSV or Excel, or read through the Apify API into your own pipeline.