Watch the Status Code Your Custom 404 Page Returns

A custom 404 page makes a site friendlier, and it can keep visitors browsing instead of closing the tab and leaving.

I use a custom 404 page myself — but I noticed a problem. After I switched to a custom 404 page, requesting a page or directory that doesn’t exist returned HTTP status 200 instead of 404. That’s serious.

Why serious? A small site like mine lives on search engine traffic, and spiders don’t recognize “a 404 page” — they go by the HTTP status. When you request a page that doesn’t exist, the server returns a 404 status, telling the search engine the page doesn’t exist or has been deleted.

So what happens if your custom 404 page returns “200” instead of “404” for an invalid URL? Obviously, the search engine concludes that this “nonexistent” page does exist on your site — and it may index your custom 404 page. If you have a pile of dead links all returning the same page, the spider gets very confused.

I asked the data center’s technical support, who said it was my program’s problem. I solved it myself, and it wasn’t a program problem.

Cause: custom error pages are supported, but the error page is defined by URL (you can define it as a file, a URL, or a 302 redirect), which makes HTTP always return 200 rather than 404, 500, and so on — only the file form returns the correct code.

Solution: fix it through web.config or a dynamic page. Here’s the web.config approach for a custom 404:

<configuration>
  <system.webServer>
    <httpErrors errorMode="DetailedLocalOnly" defaultResponseMode="File" >
      <remove statusCode="404" />
      <error statusCode="404" path="404.htm" />
    </httpErrors>
  </system.webServer>
</configuration>

Add the above to your web.config as appropriate (IIS 7 only).

With that, my site now correctly returns a 404 status along with the custom 404 page.