9 in 10 websites keep AI engines out.
Stradiji just audited how sites treat AI crawlers. Three in four give them no explicit access in robots.txt at all. Their founder put it bluntly: companies are paying to show up in AI answers while their own robots.txt tells those systems to stay out.
It's the simplest possible GEO failure. Before you worry about what the assistants say about you, check whether you've even let them in the door.
We run a free AI Readiness Score that checks exactly this, across ChatGPT, Gemini, Claude, and Perplexity: where you show up, where competitors do, and what to fix first.
When did you last read your own robots.txt?
ngmi
npub14p…jdzry
2026-10-14 07:07:30 UTC
Author Public Key
npub14p7fhlz9wrlptt7a5mqqjkk79460k98r999l7dafv4fznqaa565qhjdzryPublished at
2026-10-14 07:07:30 UTCEvent JSON
{
"id": "a515fa9b04ac4c1f43584d699d36a54aeb8771c2300301f02ee3c6989f9728aa",
"pubkey": "a87c9bfc4570fe15afdda6c0095ade2d74fb14e3294bff37a965522983bda6a8",
"created_at": 1791961650,
"kind": 1,
"tags": [
[
"client",
"Ditto",
"31990:781a1527055f74c1f70230f10384609b34548f8ab6a0a6caa74025827f9fdae5:ditto"
]
],
"content": "9 in 10 websites keep AI engines out.\n\nStradiji just audited how sites treat AI crawlers. Three in four give them no explicit access in robots.txt at all. Their founder put it bluntly: companies are paying to show up in AI answers while their own robots.txt tells those systems to stay out.\n\nIt's the simplest possible GEO failure. Before you worry about what the assistants say about you, check whether you've even let them in the door.\n\nWe run a free AI Readiness Score that checks exactly this, across ChatGPT, Gemini, Claude, and Perplexity: where you show up, where competitors do, and what to fix first.\n\nWhen did you last read your own robots.txt?",
"sig": "061b347e7480e40cede39625c6a8757084c4c3b345552b24bcae8038beb5f3a7137c0eb0e0a2097a14601977acc2c1a6bb1a9a32978844171d5feb25e5b8b679"
}