# A crawler policy is published

How the A crawler policy is published check works, what a failure means, and how to fix it in the AI Context Files dimension.

- Dimension: AI Context Files (5% of the total score)
- This check: 1 point
- Check id: `ai-context-robots`

## What does this check look at?

robots.txt is readable, so a crawler policy is published. Readable means a body with non-whitespace content: an HTTP 200 with an empty or whitespace-only body publishes no policy and fails this check, exactly as it fails robots-present.

## How do I fix it?

- Publish robots.txt stating which crawlers are welcome.

## Why is it worth 1 points?

Deliberately low. The check is worth 4 of this dimension's 6 points and the dimension carries 5% of the total, so the file is capped at 3.3 of the 100. It is scored because the convention is cheap to follow, not because it moves a ranking.

## What else is measured in this dimension?

- [/llms.txt is present and structured](https://llmention-geo.com/checks/llms-txt/) — 4 points
- [A markdown alternate is declared](https://llmention-geo.com/checks/markdown-alternate/) — 1 point

## How do I see my own result?

Run a free scan on your own domain at https://llmention-geo.com/ - the report lists every failed check with the evidence that produced it.

The full method is published at https://llmention-geo.com/methodology/
