A small sample is useful when it is described as a small sample and dangerous when it is described as a benchmark. The difference is not pedantry: a benchmark implies a fixed, representative reference you can measure yourself against; a sample of thirty hand-picked homepages scanned once describes thirty rows on one date. Both can be honest. Only one can support "the industry average is 62".
This article is about how to read the 30-site study correctly — what it supports, what it cannot, and how to quote it without turning it into something it is not. It is also a general template for reading any small-sample SEO or GEO study, including the ones that are less careful than this one.
What the study actually is
- 30 homepages, chosen by hand. The page says so in its own comments and limits: a convenience sample, not a random one, and not a ranking of the largest sites on the internet.
- One scan, on 3 October 2026. One run of the same public endpoint the product ships, no hand-adjusted results.
- Homepages, not articles. Homepages are short, navigational and carry little of the evidence an answer needs; a low homepage score does not mean a site is invisible in AI answers.
- Readiness checks, not citations. The score describes whether a page is in a state that makes citation possible; it does not measure whether an engine cites the site.
What it can support
| Claim | Supported? | Why |
|---|---|---|
| "9 of these 30 homepages did not admit an AI crawler on 3 October 2026." | Yes | It is a count over the listed rows on the stated date. |
| "Of those 9, 7 said so in robots.txt and 2 refused the request." | Yes | The data separates the mechanisms. |
| "Every news publisher in the sample blocks at least one AI crawler." | Yes, for the sample | It is a statement about the publishers included, not the industry. |
| "The average homepage scores 62." | For these rows | It is a mean of the sample, not an industry average. |
| "A 0-of-30 A rate shows nobody is ready." | No | It shows none of these homepages scored an A under this method on this date. |
What it cannot support
- Prevalence. Thirty hand-picked sites cannot estimate what fraction of the web blocks AI crawlers. The selection is not random, and the sample is small.
- A ranking of the largest sites. The list is a convenience sample across infrastructure, developer platforms, news and German industry; it is not "the 30 largest".
- Prediction. A site can score 12 and still be quoted tomorrow. The study measures readiness, not citation.
- Causation. "News publishers block because AI reduces referrals" is a hypothesis the data cannot test; the data shows the block, not the motive.
- Fine-grained comparisons. 62 versus 60 is one point on a 30-row sample; treating it as a meaningful difference is reading noise as signal.
- Generalisation across page types. These are homepages. A product page, an article and a documentation page are different measurements.
The common misreadings
| Misreading | What is actually true |
|---|---|
| "30 of the largest sites" | 30 hand-picked homepages across a few categories. |
| "9 of 30 sites block AI" → "30% of the web blocks AI" | 9 of 30 in this sample; no estimate about the web. |
| "Low score means invisible in AI answers" | The score is readiness; the articles may be in better shape than the homepage. |
| "Blocking causes invisibility" | Blocking prevents crawling; whether it prevents citation depends on the engine and other sources. |
| "My score beats Salesforce" | Different scan, different date, different page type. |
How to quote it honestly
The template is one sentence with three parts: the scope, the count, and the date. For example: "In a hand-picked sample of 30 homepages scanned on 3 October 2026, 9 did not admit at least one AI crawler — 7 by a robots.txt rule and 2 by a server refusal." That sentence is checkable against the published rows and cannot be stretched into a claim about the web.
The same template works for any small study: name the sample, the method and the date before the finding. If a study does not let you write that sentence, it is not telling you what it appears to tell you.
What a stronger study would need
- Random or stratified selection from a defined population, so the sample can speak to something beyond itself.
- Multiple page types per site, because homepages and articles behave differently.
- Repeat runs, because bot-management responses can differ between two requests seconds apart; the re-run here already exposed that.
- Pre-registered questions, so the analysis is not chosen after the data.
- Published data and code, which this study does: rows, a data endpoint, and the collection script.
Why this matters beyond one study
The GEO field is full of numbers from small samples presented as benchmarks: "X% of sites fail AI search" from a scrape of a few dozen URLs, with no date, no method and no data. The damage is not only to the reader; it is to the trust in every honest small study. A vendor that quotes its own convenience sample as an industry statistic is asking to be ignored, and a reader who learns to check the scope will ignore it.
The standard this site tries to hold is the one in the study’s own limits section: publish the rows, state the scope, and make the claim small enough to be true. A small true finding is worth more than a large one that has to be retracted.