Server Error (5xx) in Search Console: Fixes

"Server error (5xx)" is the Page indexing status that is least about SEO and most about infrastructure. Google asked your server for a page and the server failed. The question for SEO is what Google does with that failure, how long you have before it costs rankings, and how to find out whether it was a blip or a pattern. This post covers all three, and the causes that most often produce a 5xx for Googlebot that you never see in a browser.

What "Server error (5xx)" in Search Console means

Google's definition, from its Page indexing report documentation:

"Your server returned a 500-level error when the page was requested."

Any status in the 500 range counts: 500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout and the less common ones. Google got a response, but not a page. The URL stays in the report under this status until Google fetches it again and gets something else.

What Google does with a 5xx

Google's documented handling of server errors is more forgiving than most people assume, with a limit:

• At first, nothing changes. Google treats a 5xx as a temporary problem. The last indexed version of the page stays in the index, and the URL is scheduled for another try.
• The crawl rate drops. Repeated 5xx responses tell Google the server is struggling, and Googlebot slows down across the whole site to avoid making it worse. That means every other page on the site gets crawled less often while the errors continue.
• Persistent errors drop the page. If a URL keeps returning 5xx across repeated crawls, Google eventually removes it from the index, on the reasoning that a page that never loads is not a page.

The practical window is days to weeks, not hours. A server that falls over for an evening costs you nothing in the index. A server that returns 500 to Googlebot every night for a month costs you rankings and crawl rate together.

The usual causes

A 5xx that appears in Search Console but never in your own browser almost always has one of these behind it.

• Overload. Googlebot crawls in bursts. A server that handles normal traffic fine can fail under a crawl burst, especially when each page runs expensive database queries. The errors cluster at the same times of day.
• Timeouts upstream. A 502 or 504 means a proxy, load balancer or CDN was waiting on the application and gave up. The application may have been working, slowly. These show up as slow server response time long before they become 5xx errors.
• A failed deploy or a broken dependency. A template error, an exhausted database connection pool or a missing environment variable produces a 500 on every page of a section, from the moment of the deploy until someone notices. Google's crawl often notices first.
• Bot protection returning the wrong code. Some firewalls and rate limiters answer unwanted automated clients with a 503 rather than a 403 or 429. If the rule catches Googlebot, Google sees a server error. The page loads for you because you are not a bot.
• Resource limits on shared hosting. Entry-process caps, memory limits and CPU throttling on shared plans return 503 when exceeded, and a crawl is exactly what exceeds them.
• Pages that error only under certain conditions. A product page that throws when stock is zero, a listing that fails when a filter returns no results, an archive page for a month with no posts. Googlebot visits combinations humans do not.
• Maintenance mode without a proper 503. Taking the site down for maintenance and serving a 500, or a 200 with a maintenance message, instead of a 503 with a Retry-After header.

How to diagnose it

1. Read the last crawl date on the inspection result. Then run a live test. If the live test returns 200, the error has cleared and the question is whether it will return.
2. Check the server logs for that timestamp. Filter for Googlebot's user-agent and look at the status codes it received around the last crawl. One 500 among hundreds of 200s is a blip. A run of 503s at 03:00 every night is a scheduled job starving the web server.
3. Look at the Crawl Stats report in Search Console, under Settings. It charts server errors over time and breaks down responses by status code, which shows whether the problem is one URL or the whole host.
4. Group the affected URLs. If they share a path, a template or a parameter, the fault is in that code. If they are random, the fault is capacity.
5. Request the URL as a bot would. No cookies, a bot user-agent string, several requests in quick succession. If that reproduces the error and a browser does not, a protection rule is the cause.

How to fix it

The fix depends on the cause, but a few rules apply across all of them:

• Serve a 503 with Retry-After for planned downtime. It tells Google the outage is deliberate and temporary, and Google comes back when you say.
• Never serve a 200 for an error page. A maintenance page with a 200 status gets indexed as the page's content. A 200 for a page that does not exist is a soft 404.
• Give crawlers the same code as users. A firewall that answers bots with 503 is lying about the server's health. Use 403 for a deliberate block, or better, let verified search engine crawlers through.
• Fix capacity before fixing rank. A server that errors under crawl load will keep producing this status, and the reduced crawl rate affects every page. Caching, query optimisation or a bigger plan all count.
• Resubmit when it is fixed. Request indexing for important pages and resubmit the sitemap. The status clears per URL as Google recrawls each one.

Not every error code is the server's fault

Two related statuses are worth separating from this one. A 404 means the server worked and the page is not there; the not found 404 in Search Console status covers it. A 401 or 403 means the server worked and refused the request; blocked due to access forbidden (403) covers those. And a 429 Too Many Requests is a 4xx that asks the crawler to slow down; it is not a server error, though a server that sends many of them is one step from sending 503s.

Finding 5xx errors across the site with a crawl

Search Console reports the 5xx responses Googlebot received, as of its last visit to each URL, days after the fact. It cannot tell you what the site returns right now, and it cannot show you a whole section failing until Google has worked through it.

Crawl Cove requests every page on the site and its status-code check reports each one that returned a 5xx at Critical severity, with the status code and the page, so a section that is erroring shows up as a block of findings under one path. A page that could not be fetched at all is Critical too, under its own check. It treats the two near-misses differently on purpose: a 429 is reported at Low, as a page that was not checked rather than a broken one, and a timeout is reported at High as a slow page rather than a dead one, so the crawl's own pace does not get reported as your outage. The crawler retries 429, 502, 503 and 504 responses before reporting them, so a single transient failure does not become a finding. The slow-response check flags pages whose response time is heading towards a timeout before they get there.

The Live Index Status panel calls Google's URL Inspection API for any page and shows the page fetch state and last crawl date, so you can confirm Google has seen the fix. Run a full SEO audit to see every page that currently errors, in one list, before Google does.

Wrap-up

"Server error (5xx)" is Google reporting that your server failed it. Google is patient about it at first and unforgiving if it persists, and it slows its crawl of the whole site while it waits. Find the cause in the logs at the time of the last crawl, look especially for errors that only automated clients trigger, serve honest status codes, and fix capacity rather than symptoms. A crawl of your own site shows you the errors as they are now, which is the version Google will see next.