Blocking traffic in Google Analytics

As I mentioned in my last posts, AdWords Bots in Google Analytics (GA), and Bot Traffic in Google Analytics we’ve seen more and more bot traffic registering as visitors in GA.  (Bots are automated programs that normally don’t fire JavaScript or images, so this traffic shows up in web server logs, but not in Google Analytics.)

In this post we’re going to discuss how to block these bots from your GA reports so that you can have clean data from which to make smart business decisions and make more money!

First let’s understand that “well behaved” bots usually have one of two fingerprints:

1. Technology profile – this includes browser user agent, version, java profiles, etc.
2. ISP – Yes that’s right, Microsoft, Yahoo! and Inktomi bots come from Microsoft, Yahoo! and Inktomi ISP’s.

What kind of tracks does your bot traffic leave?  Well it’s best to start with an Advanced Segment that begins to whittle away this bot traffic from the real traffic.  Apply this segment to your Google Analytics data and you’ll be viewing only direct traffic that bounces.

https://www.google.com/analytics/web/permalink?uid=BH3Nk-LCSpmv7UQyPXgGDA

Do you see any patterns in the browser version reports?  (Audience>>Technology>>Browser & OS)

In the screen shot above we’ve selected “Mozilla Compatible Agent” and version 5.0 with no Java Support looks like a likely bot suspect.

What about identifying bots by ISP? (Audience>>Technology>>Network)

So from the data above we can see some interesting sources of bouncing traffic.  Next I created another segment to view each of these ISP’s (1. microsoft corp , 6. yahoo! inc. and 16. Inktomi Corporation).

https://www.google.com/analytics/web/permalink?uid=nt7BieC5RR24dRcai6Kb_g

As you can see from the data above, all of the traffic from these ISP’s bounced.

Your next action is to decide if you want to go a step further and actually filter out this traffic from your analysis profiles.

If you decide to block traffic based on browser profile, then you’ll need to construct a series of filters to do this.

The first two combine browser data together with Java Support (yes or no) to allow you to then filter only the offending browser profile (Third filter listed in the image above)  that is utilized by the bots.

If you decide that the ISP route is the way to go, then you’ll have a much easier path:

Where the pattern is equal to   yahoo|microsoft corp$|inktomi

Either way, you should remember to:

• Always have an unfiltered profile that collects all data

• When possible, test your new profile filter as a segment

• Always apply a filter first to a test profile and when happy with the resulting data, to your analysis profile(s).

• Consider creating a profile to capture the traffic you’re excluding so that you can monitor it more easily and ensure that you’re not excluding any “real” traffic.

In summary, it really does not matter why the bot is on your site, what’s important is that they are triggering nuisance pageviews that can skew your numbers and conversion rates.  Are they affecting your GA data?  Apply the segments above and find out for yourself!