Skip to content

Noindex Checker

This noindex checker fetches a URL the way a crawler does and tells you whether Google can index the page, and if it cannot, exactly what is stopping it.

Try it on or paste any public URL.

Note

We request the URL from our server, follow each redirect ourselves and read the HTML exactly as the server sent it, before any JavaScript runs, along with every response header and robots.txt for Googlebot. Nothing from the page is stored; only these findings come back to you.

It reads the meta robots and googlebot tags, in the head or stranded in the body, every X-Robots-Tag header including ones aimed at a single crawler, robots.txt as Googlebot reads it for this path, the status code after each redirect, and the canonical. It also catches the combination that trips most people up: a noindex on a page that robots.txt blocks, which Google never gets to read. No sign-up and nothing to install.

Use this tool often? Bookmark this page, so it is one click away next time.

Email me this result

Want a copy for your records or a client? Leave your email and we'll send you this result. It is written by a real person at Crawl Cove, not an automated blast, so it may take a little while to arrive. We'll only use your address to send it, unless you also tick the box below. Privacy policy.

That email address doesn't look right. Please check it and try again.

Something went wrong saving your address. Your result above is unaffected; please try again.

Got it. We'll email your result to you. Nothing else lands in your inbox unless you ticked the tips box.

How to check a page for noindex

  1. Paste the URL

    Enter any public address, with or without https://. If it redirects, the checker follows each hop and checks the page you end up on.

  2. Read the verdict

    Indexable, Not indexable or Can't tell, with the reasons underneath. A page is only indexable when every signal agrees: a 200 status, robots.txt open to Googlebot, no noindex anywhere and a canonical that names the page itself.

  3. See where each directive came from

    The directives table lists every robots tag and header the page sent, which crawler it addresses, and whether Google obeys it. A noindex hiding in a response header shows up here even when the HTML is clean.

  4. Fix it and check again

    Each failed check comes with the change to make. Results are cached for ten minutes, so wait that long before re-checking a fix.

What a noindex tag does

A noindex directive asks search engines to keep a page out of their results. It can sit in the HTML, as <meta name="robots" content="noindex"> in the head, or in the HTTP response, as an X-Robots-Tag: noindex header, which is the only way to noindex a PDF or an image. Google still crawls a noindex page from time to time, but it drops the page from search the next time it reads the tag. That is the right setting for thank-you pages, internal search results, staging copies and thin filter pages. On a page that should rank it is usually an accident: a plugin setting left on after launch, or a template that adds the tag to more pages than intended.

Noindex: what it means and how to fix it · Excluded by noindex tag in Search Console

What this noindex checker tests

Directives: the meta robots tag, the meta googlebot tag, tags addressed to other crawlers such as bingbot, and every X-Robots-Tag header, including a header repeated more than once and a header scoped to one crawler with a prefix such as googlebot:. Google obeys the robots and googlebot versions and ignores the rest, and when two directives conflict it follows the more restrictive one, so the checker does the same. It also flags nofollow, none, nosnippet and max-snippet:0, unavailable_after with its date, and names it does not recognise, such as no-index, which Google ignores. Crawling: the status code, each redirect hop, and robots.txt for Googlebot at this exact path, including a robots.txt that answers with a server error. Canonical: whether it names this page or asks Google to index a different URL.

robots.txt blocks all search engines · 4xx and 5xx errors

Noindex and robots.txt: why using both backfires

A robots.txt Disallow stops Googlebot fetching a page. A noindex tells Google not to list it. Use both on one URL and the block wins in the wrong way: Google cannot fetch the page, so it never reads the noindex, and if other pages link to the URL it can still appear in results as a bare address with no description. To take a page out of Google, leave it crawlable with noindex on it until it has dropped out, and only then block it if you need to save crawl budget. The checker flags this combination on its own line.

Indexed, though blocked by robots.txt · Blocked by robots.txt in Search Console

X-Robots-Tag headers and crawler-specific rules

A noindex in a response header is invisible in the page source, which is why it survives so long. Servers, CDNs and security plugins add X-Robots-Tag headers, sometimes to every response on a staging host that later goes live. A header can carry several directives, can be sent more than once, and can be aimed at one crawler: X-Robots-Tag: otherbot: noindex keeps the page out of that crawler's index and does nothing to Google. The directives table shows each header line separately, with the crawler it addresses.

Indexable is not the same as indexed

Indexable means nothing on the page stops Google indexing it. Whether Google then does is its own decision, made on the quality of the page, whether it duplicates another one, and how easily Google can find it through links and the sitemap. If this checker says Indexable and Search Console still reports the page as not indexed, the cause is in that second group, and the Search Console status names which one.

Crawled, currently not indexed

Noindex added or removed with JavaScript

The checker reads the HTML the server sends, before any script runs. Google renders pages later and does obey a noindex that a script adds, so a site that sets robots tags in JavaScript can be noindexed in a way this check cannot see; the checker warns when a script in the page mentions noindex. The reverse does not work: when Google finds noindex in the server HTML it may skip rendering altogether, so a script that removes the tag after load leaves the page out of search. Keep robots tags in the server HTML.

Frequently asked questions

Is this noindex checker free?
Yes. It is free, needs no sign-up and no email, and there is no daily limit beyond a rate limit that stops one visitor sending hundreds of checks a minute.
How do I check if a page is noindex?
Paste the URL above. Doing it by hand means three places: view the page source and search for noindex, look at the response headers in your browser's developer tools for X-Robots-Tag, and read robots.txt for a rule that blocks the path. The checker reads all three at once.
What is the difference between noindex and nofollow?
noindex keeps the page out of search results. nofollow tells search engines not to follow the links on the page. They are independent: a page can be noindex and still have its links followed, which is usually what you want for a thank-you or search results page. none is shorthand for both.
Should I use noindex or robots.txt to hide a page?
Use noindex to keep a page out of search results. robots.txt controls crawling, not indexing, and a blocked URL can still be listed if it is linked from elsewhere. Do not use both on the same page, because the block stops Google reading the noindex.
Does it check the X-Robots-Tag header?
Yes. It reads every X-Robots-Tag header in the response, including repeated headers and headers addressed to one crawler, and shows which ones Google obeys.
I removed noindex. When will Google index the page?
After Google next crawls the page and sees the tag has gone. Use URL Inspection in Search Console to request indexing, and make sure the page is linked internally and listed in your sitemap so Google comes back sooner.
Does Bing obey noindex too?
Yes. Bing reads the meta robots tag and the X-Robots-Tag header in the same way. A tag named bingbot applies to Bing only, just as a googlebot tag applies to Google only; the checker lists both and bases its verdict on Google.
Can I check a private or staging URL?
No. The checker only requests public addresses on the standard web ports, and it refuses any redirect that leads to a private network address. To check a staging or intranet site, use a crawler that runs on your own computer, such as Crawl Cove.
Can I share the result?
Yes. After a check the page address includes the URL you checked, so copying it from the address bar gives a link that re-runs the same check. Results are cached for ten minutes, so re-check after that to see a change.

From one page to your whole site

This checks one URL. Crawl Cove reads the status, robots.txt rule, meta robots, X-Robots-Tag and canonical of every page it crawls and lists every page that cannot be indexed, with the reason, so a template that noindexes a whole section shows up at once. See check one page's canonical or run a full site audit.

Download free