Search results in the crawl

WarningMedium priority

Addresses of this site that look like the site's own search results.

Why it matters

Search pages are generated without limit and hold nothing of their own, so crawling and indexing them wastes the budget that the real pages need.

How to fix it

Block the search path in robots.txt, or mark the results noindex, and stop linking to searches from the site.

Skitter runs this check on every crawl, and lists each page it finds with the evidence: the header, the tag or the link that gave it away.

More in Addresses