robots no sitemap

info robots-no-sitemap robots.txt

The Sitemap line in robots.txt is the one place every crawler looks without being asked. Leaving it out means relying on each engine being told separately. Submitting a sitemap through each search engine's own panel works, and stops working the moment a new engine matters or an account changes hands. The line in robots.txt is the version that keeps working without anyone remembering it, and it is the only discovery route the crawlers behind AI answers reliably follow.

What indexgap reports

The finding reads:

info  robots-no-sitemap  robots.txt names no Sitemap — the line `Sitemap: https://…/sitemap.xml` costs nothing

How to fix it

Add a `Sitemap:` line to robots.txt.

How it behaves at scale

This is a site-level finding: it belongs to the site, not to any one page, and is reported once however many pages are audited.

Finding it

pipx install indexgap
cd your-project
indexgap init
indexgap check

Then indexgap brief turns every finding on this page into a work order in indexgap-briefs/, with the thresholds the result has to satisfy and the command to check it.

Related checks in robots.txt

All checks · robots.txt