Articles in The 'Google Crawling' Tag

Google’s Search for Your Website’s Pages

by Melanie Wahl

Google can be a valuable source of traffic for your website.   Googlers who search for a specific keyword or keyphrase benefit from Google’s curated results.   These results, separated into Search Engine Results Pages, deliver the best quality content that makes sense with the query entered.   Behind the scenes, Google goes through a number of steps before displaying (or serving) the queried content to the user.   These include: Crawling, Indexing, and Serving.

Crawling refers to the GoogleBot, Google’s web crawling bot (or spider), that “crawls” or discovers new and updated pages by following links from site to site.   This is why the “nofollow” attribute (rel=”nofollow”) was created, to prevent GoogleBot from following a link.

Indexing refers to the process of sorting which GoogleBot conducts to organize different content types.   Information processed to help GoogleBot sort a page includes tags and attributes.   Some rich media files or pages with dynamic features are not able to be processed, which is why it is best to try to simplify coding on your website if you find that a page is not showing up in Google’s Index.

Serving is the end result, the displayed snippet when a Google searcher enters a query and results are “served” to the Search Engine Results Page (SERP).   Google strives to serve the most relevant pages to a search query and it is a very complex process algorithm which weights results and orders accordingly.

If you are not already familiar, we urge you to read Google’s Webmaster Guidelines to learn Google’s best practice suggestions for helping find, crawl, and index your website.