Domain Misconfigurations and Duplicate Content

Lee Zoumas - May 15, 2009

An often overlooked part of establishing a web presence is not paying close attention to how a domain name is configured. Usually, we buy a domain name from a registrar, buy some hosting, and then point that domain name to the host, using the registrar’s control panel. However, some registrars or hosts, contain a wildcard setting that will allow all possible combinations of subdomains to resolve back to your main domain. For example, imagine you buy a domain called example.com. Typically, you would want users to browse the site in one of two ways:

http://example.com
http://www.example.com

Ideally, you would want the non-www to 301 redirect to the www version, otherwise you would have duplicate content. However, sometimes there are wildcard subdomains enabled by default in registrar or hosting control panels. Typical settings for a domain configuration will look like this:

The problem with the above settings lies in the “* (All Others)” wildcard setting. This basically says “route any subdomain to the main domain.” So all possible combinations will lead back to the main website:

http://wwww.example.com
http://abc.example.com
http://spam.example.com
http://duplicatecontent.example.com

The list could go on and on and could get downright nasty. Basically, what this means is that someone could create a page that links to all these subdomain variations and those pages will get indexed by search engines, which will create duplicate content for your website. That is why it is imperative to make sure that your domains are not configured this way.