Skip to content

HTTP Header Checker

This HTTP header checker requests a URL the way a browser does, follows each redirect itself and shows the status line of every hop, the response time and every header the server sent back.

Send the main request as The other one is fetched too, for the comparison.

Try it on or paste any public URL.

Note

We request the URL from our server with a Chrome or Googlebot user agent, asking for gzip or Brotli, and follow each redirect ourselves so every hop's status line is recorded. Headers are shown exactly as the final response sent them; the body is not decompressed, so the size is what went over the wire. The comparison ignores headers that change on every request, such as Date, cookies and cache IDs. Nothing from the response is stored; only these findings come back to you.

The headers that affect search and security are checked for you: X-Robots-Tag, a canonical in the Link header, Content-Type and charset, HSTS, Content-Security-Policy, X-Content-Type-Options, framing, Referrer-Policy, Cache-Control, ETag and Last-Modified, Vary, compression and software version leaks. It then asks again as Googlebot and lists anything that came back different. No sign-up and nothing to install.

Use this tool often? Bookmark this page, so it is one click away next time.

Email me this result

Want a copy for your records or a client? Leave your email and we'll send you this result. It is written by a real person at Crawl Cove, not an automated blast, so it may take a little while to arrive. We'll only use your address to send it, unless you also tick the box below. Privacy policy.

That email address doesn't look right. Please check it and try again.

Something went wrong saving your address. Your result above is unaffected; please try again.

Got it. We'll email your result to you. Nothing else lands in your inbox unless you ticked the tips box.

How to check a URL's HTTP headers

  1. Paste the URL

    Enter any public address, with or without https://. Choose whether the main request goes as a browser or as Googlebot; the other one is fetched too, for the comparison.

  2. Read the status of each hop

    The status lines show every redirect in order, with its code, the Location it pointed to and how long it took, ending with the final response.

  3. Work through the checks

    Each header that matters is marked Pass, Improve or Fail, with what was found and a one-line fix. Fix the failures first: a noindex header or a server error stops a page ranking.

  4. Compare Googlebot with a browser

    The comparison panel lists any status, redirect or header that differs between the two user agents. Different X-Robots-Tag, Location or Vary values are flagged, because those change what Google indexes.

What HTTP headers are

Every time a browser or crawler asks a server for a URL, the server answers with a status line, a block of headers and then the content. The status line says what happened: 200 for a normal page, 301 or 308 for a permanent move, 404 for not found, 500 for a server error. The headers are name and value pairs that tell the client how to treat the response: what kind of file it is, how long it can be cached, whether it was compressed, which security rules apply and, for search engines, whether the URL may be indexed. None of this shows in the page itself or in view source, which is why a header checker is the quickest way to see it.

How this header checker works

We send the request from our server, asking for gzip or Brotli the way a browser does, and we do not follow redirects automatically. Each hop is requested separately, so you see its exact status line, its Location header and its time. The final response is the one whose headers are listed and checked. The same URL is then requested a second time with the other user agent, and the two answers are compared header by header. Headers that change on every request, such as Date, cookies and cache IDs, are left out of the comparison so that only real differences show.

The headers that matter for SEO

X-Robots-Tag is the one to check first. It does the same job as a robots meta tag but sits in the response, so a noindex added by a server rule or a CDN never shows in the HTML and is easy to miss for months. A directive scoped to another crawler, such as bingbot: noindex, does not apply to Google, and the check reads it that way. The Link header can carry a canonical, which is how PDFs and other files without HTML name their preferred URL. Content-Type tells crawlers what they are reading, and the status code decides whether the page is indexed at all: anything other than 200 at the end of the chain is a problem for a page you want to rank.

Noindex: what it means and how to fix it

Security headers: HSTS, CSP, framing and nosniff

Strict-Transport-Security tells a browser to use HTTPS for your domain for a set time, so it never tries the insecure address first. Content-Security-Policy lists where scripts, styles and images may load from, which is the main defence against injected scripts. X-Frame-Options, or the frame-ancestors rule inside a CSP, stops other sites loading your page in a hidden frame to trick clicks. X-Content-Type-Options: nosniff stops browsers guessing a file's type. Referrer-Policy controls how much of your URL is passed to other sites when a visitor follows a link. These do not change rankings directly, but a hacked page full of injected links does, and they are cheap to add.

Missing security headers: how to add each one

Caching and compression headers

Cache-Control says who may store a response and for how long. ETag and Last-Modified let a browser or crawler ask whether a URL has changed and get a short 304 Not Modified answer if it has not, which saves bandwidth on every recrawl. Vary tells caches which request headers change the response; a compressed response should vary on Accept-Encoding, and Vary: * stops caching altogether. Content-Encoding shows whether gzip or Brotli was used. Text responses such as HTML, CSS and JavaScript usually shrink to a third of their size or less, so an uncompressed page is the easiest speed fix there is.

Googlebot and a browser can get different answers

Servers, CDNs and security plugins often treat crawlers differently. A firewall may challenge or block a request that claims to be Googlebot, a plugin may add noindex only for bots, or a site may redirect crawlers somewhere visitors never see. This checker sends both requests and lists every status, redirect or header that differs, flagging the ones that change indexing. One caveat: our request uses Googlebot's user agent but not Google's IP addresses, so a CDN that verifies crawlers by IP will treat it as a fake Googlebot. A block in that column means the site checks the user agent; it does not prove the real Googlebot is blocked. Search Console's URL Inspection is the final word for that.

Redirects: this tool or the redirect checker?

This tool shows the status line of each hop so you can see how a URL arrives at its final response, then concentrates on the headers of that response. If the redirects themselves are the problem, such as a long chain, a loop or a meta refresh, the redirect checker is built for that: it traces every hop in detail and tests http, https, www and non-www variants.

Redirect Checker

Frequently asked questions

Is this HTTP header checker free?
Yes. It is free, needs no sign-up and no email, and there is no daily limit beyond a rate limit that stops one visitor sending hundreds of checks a minute.
Does it check the HTTP status code too?
Yes. It shows the full status line, such as HTTP/1.1 301 Moved Permanently, for every hop in the redirect chain and for the final response, with the time each one took.
Why are security headers missing from my site?
Most servers and hosting platforms do not send them by default. They are usually added in the web server configuration (.htaccess on Apache, a server block on nginx), at the CDN, or by a security plugin. If you added them and they still do not show, check whether a CDN or cache in front of the server is stripping them or serving an older copy.
Is HSTS the same as redirecting http to https?
No. A redirect only works after the browser has already made an insecure request, which can be intercepted. HSTS tells the browser to go straight to HTTPS for every later visit, without trying http:// at all. You need both: the redirect for first visits, and HSTS for the rest.
Why does Content-Security-Policy matter?
It tells the browser which sources may run scripts and load content on your page. If an attacker injects a script through a vulnerable plugin or form, a good CSP stops it running. Start in report-only mode so you can see what a policy would block before you enforce it.
Why is the response time different each time I check?
Network distance, server load and caching all change it. The first request after a quiet spell is often slower because the server or CDN has to build the page; the next one may come from a cache. Our server is in Europe, so a site hosted far away will show more delay than its local visitors see.
Can I check a private or internal URL?
No. The checker only requests public addresses on the standard web ports, and it refuses any redirect that leads to a private network address. To check a staging or intranet site, use a crawler that runs on your own computer, such as Crawl Cove.
Why does the Googlebot request get blocked when Google indexes my site fine?
Some CDNs and firewalls check that a request claiming to be Googlebot really comes from Google's IP addresses. Ours does not, so it is treated as a fake and may be challenged or blocked. That is the firewall working as intended. Use URL Inspection in Search Console to see what the real Googlebot gets.
Can I share the result?
Yes. After a check the page address includes the URL you checked, so copying it from the address bar gives a link that re-runs the same check. Results are cached for ten minutes, so re-check after that to see a change.

From one page to your whole site

This checks one URL. Crawl Cove reads the response headers of every page it crawls and flags status codes, redirect chains, X-Robots-Tag noindex, insecure canonicals, missing security headers and slow responses across the whole site. See run a full site audit or trace a redirect chain.

Download free