Skip to content
Sitemap Low severity By Lewis Jones

How to Add Your Sitemap to robots.txt

Your site has an XML sitemap but robots.txt never names it. The one line to add, where it goes, and what changes for crawlers that have not been told.

To add your sitemap to robots.txt, put one line anywhere in the file with the sitemap's full address:

Sitemap: https://www.example.com/sitemap.xml

That line is how any crawler finds your sitemap without being told about it somewhere else. This finding means your site has a sitemap, but robots.txt does not mention it.

What this finding means

Crawl Cove's sitemap-not-in-robots check fires when a crawl finds no Sitemap: line in robots.txt, tries the two usual addresses (/sitemap.xml and /sitemap_index.xml), and finds a working sitemap there that lists at least one URL. A real finding reads like this:

robots.txt has no Sitemap: line. This audit found the sitemap at https://www.example.com/sitemap_index.xml (184 URLs) by trying the usual address.

or, when the site has no robots.txt at all:

The site has no robots.txt (HTTP 404), so nothing names its sitemap. This audit found it at https://www.example.com/sitemap.xml (184 URLs) by trying the usual address.

Crawl Cove found the sitemap because it guessed the common address. Other crawlers may not guess, or may guess differently, which is the point of the finding. The same crawl also uses that sitemap for its other sitemap checks, so it can still compare your sitemap with the site.

Why it matters

There are only a few ways a search engine learns where your sitemap is: a Sitemap: line in robots.txt, a submission in that engine's own webmaster tools, or a guess. A Search Console submission reaches Google alone and a Bing Webmaster Tools submission reaches Bing alone. Google retired its anonymous sitemap ping in 2023. The robots.txt line is the one place every well-behaved crawler already reads before it crawls a site.

The cost is small, which is why the finding is Low and advisory. Google and Bing will often find the pages through links anyway, and if you have submitted the sitemap to both, they already have it. The line matters most for new pages that few links point to yet, and for the crawlers you have never submitted anything to.

It is common. In our October 2026 sweep of small business websites, 161 of the 934 sites with a sitemap at one of those two addresses (17%) did not name it in robots.txt.

How to fix it

  1. Find the sitemap's real address. Use the URL in the finding, or open /sitemap.xml and /sitemap_index.xml in a browser. If you have a sitemap index, use the index.
  2. Open robots.txt. On WordPress and most hosted platforms, an SEO plugin or the platform's settings usually manage it; otherwise it is a plain text file at the site root.
  3. Add the line, with the full URL: Sitemap: https://www.example.com/sitemap_index.xml. It can go anywhere in the file and applies to all crawlers. Add one line per sitemap if you have several and no index.
  4. No robots.txt yet? Create one at the root containing just that line. A file with no Disallow rules blocks nothing.
  5. Check it. Load https://your-site/robots.txt in a browser and confirm the line is there, then test the file with the free robots.txt tester. A re-crawl in Crawl Cove clears the finding.

False positives and edge cases

  • A robots.txt the server refused is never judged. If robots.txt answers 403, 429 or a server error, the crawl cannot see what is in it, so this check stays silent rather than claim the line is missing.
  • Submitted elsewhere already. If the sitemap is submitted in Search Console and Bing Webmaster Tools, those two engines know it. The finding is still true for every other crawler, so add the line anyway; it costs nothing.
  • A sitemap at an unusual address (for example /post-sitemap.xml only) is not tried, so a site like that gets no finding from this check. Name it in robots.txt all the same.

Related reading

If robots.txt does name a sitemap but the address answers 404, the crawl reports robots.txt pointing at a sitemap that 404s instead. For the rest of the file, see how to write a robots.txt file, and for what the sitemap itself should contain, non-indexable URLs in the sitemap and sitemap lastmod dates Google cannot trust.

Specification: Sitemaps XML format

Frequently asked questions

Where in robots.txt does the Sitemap line go?
Anywhere. The Sitemap line is independent of the User-agent groups, so it applies to every crawler wherever you put it. Most sites put it at the very end of the file, on a line of its own, which keeps it easy to find.
Does the URL have to be absolute?
Yes. The protocol asks for the full URL, including https:// and the hostname: Sitemap: https://www.example.com/sitemap.xml. A relative path such as /sitemap.xml is not valid, and some crawlers will ignore the line. Use the same scheme and hostname your site actually serves, so the URL does not redirect.
I have several sitemaps. Do I list all of them?
If they sit under a sitemap index, list only the index; crawlers follow it to the rest. If you have separate sitemaps with no index, you can add one Sitemap line for each.
I already submitted my sitemap in Search Console. Do I still need this?
It does no harm and it reaches more crawlers. A Search Console submission tells Google only, and Bing Webmaster Tools tells Bing only. The robots.txt line is read by any crawler that follows the sitemaps protocol, and since Google retired its anonymous sitemap ping in 2023 it is the one standard way left to tell every crawler at once.
What if my site has no robots.txt at all?
Create one at the root of the site, for example https://www.example.com/robots.txt, containing just the Sitemap line. With no other rules in the file, nothing is blocked, so adding it cannot stop any page being crawled.

Audit your site the easy way

Crawl Cove finds this on your machine, on every paid plan. Try the XML Sitemap Checker or see every feature.

Download Crawl Cove

Related issues