Noindex in a header

WarningMedium priority

Pages whose noindex comes from the X-Robots-Tag header rather than from the page itself.

Why it matters

Nothing in the HTML shows it, so it is easy to miss: the page looks right in a browser and in the source, and is still unfindable.

How to fix it

The rule is set by the server or a CDN rather than by the template, so look there rather than in the page.

Skitter runs this check on every crawl, and lists each page it finds with the evidence: the header, the tag or the link that gave it away.

More in Robots directives