← All checkpoints
AI Crawler & Machine Accessibility · 2 points

sitemap.xml present & referenced in robots.txt

Why it matters

A sitemap.xml referenced in robots.txt is the standard way crawlers discover all your indexable pages efficiently, rather than relying purely on link-following.

How to fix it

Generate a sitemap.xml listing your pages and add a "Sitemap:" line pointing to it in robots.txt.