It seems that this will not be an automatic process, but will require web developers to tweak their applications slightly, so they are in accordance with the new standards that Google is proposing. However, Google is hoping that this will be possible with minimal modifications to existing AJAX-based applications. Although Google has not specifically stated how web developers will know if their modifications are correct or not, they do mention that there will be a way to verify what parts of your AJAX-driven content are crawlable, perhaps through Google Webmaster Tools?
Although this proposal and implementation are still in its infancy, it has been a long time coming in my opinion. This will no doubt make the web a better place. Let’s hope that other search engines will soon follow suit.
Best web development and SEO practices dictate that any webpage which does not exist, return an HTTP response code of 404, or Not Found. Basically, this response code means that the URL that you’re requesting does not exist. It could have existed in the past, may in fact exist in the future, but definitely does not exist right now. More often than not, websites will issue the generic 404 page (which I’m sure you’ve all seen many times) when the requested resource cannot be found. While this tells you the page that you’re looking for does not exist, it also takes you outside of the site’s design and navigation structure, which can be quite annoying. To combat this annoyance, web developers can create a custom 404 error page, which issues the correct 404 response code, shows custom error handling text, and keeps the browser within the site’s design and navigation.
Fortunately, ASP.NET provides several ways to issue a custom 404 error page. For this to work properly all of these procedures should be in place simultaneously or unanticipated results may occur.
The first step in creating a custom 404 error page is to actually create the HTML for the page. How to do this is outside the scope of this post, but basically the page should mimic the site’s design, navigation and present the user with a description of the error.
The next step is to create an entry in the website’s web.config file that points to the page you created when a 404 error occurs:
Now when an ASP.NET page is requested that does not exist, the user will be presented with the custom error page you created. An important, an often overlooked factor, is that this only covers ASP.NET pages because (unless you do a lot of finagling) the web.config page will only process ASP.NET pages. In my experience, this is where most web developers leave off with their custom error handling, which quite frankly, just isn’t good enough.
OK, so what about the pages in your website that aren’t ASP.NET pages, such as images, PDFs and HTML pages? To get those to work, you need to edit the Custom Errors tab in the Website properties in IIS. If you are using a shared hosting environment, there is usually a Custom Error section in the control panel. Basically what you want to do here is create an entry for a 404 error. In Windows, you need to make sure it is of type ‘URL’ and provide the URL to the page. This will cover all other non-ASP.NET pages on your site.
There’s still one more thing you need to do to really make sure you’re handling all your 404 errors properly. Sometimes an ASP.NET page will in fact exist, but critical, dynamic information is not supplied for it to display properly. The information could be a missing query string variable or perhaps the server is requesting a row from the database that has been deleted. Rather than show the page with the wrong information (a topic that requires another blog post altogether) here is what you need to do:
Create the following function; this one happens to be in VB.NET:
Public Sub Thow404Error()
Response.StatusCode = 404
Now anytime you are relying on dynamic information to properly display a page, you want to call the above function if that data is not present:
If PageHasNecessaryData() Then
If you implement the procedures mentioned above into your ASP.NET website, then you can be confident that you are covering all of the necessary aspects of proper 404 error handling. In a future post, I will explain why failing to implement the last procedure can cause all kinds of negative SEO implications.