Articles in The 'Bot Traffic in Google Analytics' Tag

Bot traffic in Google Analytics

by Theo Bennett

As I mentioned in my last post, AdWords Bots in Google Analytics (GA), we’ve seen more and more bot traffic registering as visitors in GA. (Bots are automated programs that normally don’t fire JavaScript or images, so this traffic shows up in web server logs, but not in Google Analytics.)

In this post we’re going to discuss bots from Yahoo! and Microsoft, why it’s important and how to identify the traffic and see if it’s affecting your site.

Why is this important? Well as you’ll see, all of this bot traffic comes into your site as Direct traffic, has exactly one pageview and then does nothing, and that is the problem. We have to remember that a visit as just described equals a bounce — which is a bad thing. So as you look at your reports over time, you may wonder why your goal conversion rates or Ecommerce conversion rates from Direct traffic have plummeted while your bounce rates have increased. Part of the answer could very well be bots. And if you don’t account for this traffic in your quest for the analytics intelligence that will turn your site from a business cost to a profit center, you may never get there!

So how do you know if this traffic is affecting your site? Well by looking from 30,000 feet, you may never know — you have to dig deep. So if you haven’t already been digging for answers about your Direct traffic performance, let me walk you through how to identify these bots.

First of all we know that the focus area is Direct traffic that bounces; so the first step is to create an advanced segment to “filter” all of our reports for these visits in Google Analytics Reports.

If you’d like to “play along” as you read this post here is a link to the segment:

https://www.google.com/analytics/web/permalink?uid=BH3Nk-LCSpmv7UQyPXgGDA

This is a view of the service providers in the Network sub-section of the Technology Report that have been the source of our Direct, bouncing traffic.

Why look here? Well there isn’t much to glean from other reports. Content reports are varied. Traffic Sources and Conversions we know, so the best place to try to find some answers is in the Audience section of Google Analytics. Any report here is a good starting place and in this case we can see from the screen shot above that we’ve gotten a lot of traffic directly from Microsoft and yahoo! inc.

So let’s take a closer look at this bouncing, Microsoft and Yahoo! traffic by applying the segments below.

Microsoft:
https://www.google.com/analytics/web/permalink?uid=c8BzLRzjTnmR9JFa48cbsw

Yahoo!
https://www.google.com/analytics/web/permalink?uid=xXjmJTD8RuakRpcLeR9cQA

So the data that points to bots here is pretty straight forward:

• 100% Bounce Rate

• Nearly 100% New Visits

• No Java Support

So let’s look closer at the Internet Explorer Traffic:

We can see that most visits are from IE7 and again with no Java Support. (Java Support by itself isn’t necessarily a “bot indicator”, there are other supporting traits that we don’t have the space to include or address in this post.)

While Yahoo! ignores IE, both companies are leveraging Mozilla Agents.

While Microsoft eschews Firefox 3.5:

So what does this all mean? First, it’s highly likely that Google, Microsoft and Yahoo! are using automation to explore websites and on the highest level, that it’s no longer safe to say that bots are not tracked in Google Analytics. As Analysts, Marketers or business owners, we all need to make sure we’re accounting for their presence as we explore analytics data.

In my next post I’ll share some strategies to filter out this traffic and more segments to help you remove the unwanted effects of this traffic.