Articles in the Advanced Analytics Techniques Category

May 13 2009

What are we going to do at 3:14:07 UTC, January 19, 2038?

by

Start the countdown right now! In a little under 29 years from now in the year 2038, Web Analytics engineers at Google, Yahoo, Omniture, Coremetrics, and WebTrends will have some very tough choices to make – and it’s never too early to start thinking about them!

This isn’t a trivial issue like Y2K or something like the digital TV transition day on June 12th of this year – no, no, no! This has the potential to seriously compromise cookie integrity, and potentially “break” visitor tracking, industry-wide!

What is happening in 2038?

On Tuesday, January 19th at exactly 3:14:07 UTC, all computer software programs (including Web Analytics Cookies) that store system time as a signed 32-bit integer (like a Unix timestamp) will start to “wrap around”, storing time as a negative number, causing every system using signed 32-bit integers to interpret time as 1901, and not 2038.

Whoa, Whoa! Back Up – I have no clue what you’re talking about.

Okay, let me try to break this down for you. Almost every 20th century computer uses a signed 32-bit integer which keeps track of system time on your computer, on servers, ATM machines, iPods and iPhones, and so on. This “signed 32-bit integer” business is also known by another name – Unix Time (or also “POSIX” time). This time is represented by the number of seconds since January 1, 1970.

If you take a look at your browser’s cookies, you’ll see endless strings of numbers and dots, like this:

My Cookies and the Unix Timestamp

The cookie selected here in this image is the __utma cookie from Google Analytics, and the 10-digit number that I have highlighted represents the first time I visited the Google.com website. This number – 1239628694 – is a Unix Timestamp, and when you do the math (or use a conversion tool somewhere online), this number translates to Mon, 13 Apr 2009 13:18:14 GMT (of course, I most likely cleared my cookies – yes World, I clear cookies from my computer, too!)

So what’s the problem again?

Okay – the problem with this comes due to the way modern computer programs calculate this 10-digit number. That’s what you need to know (Warning: This next party is very geeky). They almost all use a very standard 4-byte integer to count up the seconds, which is 31 bits long, able to contain a maximum value of 2 to the power of 31. The 32nd bit is the sign, which of course is positive (+). When you do the math, the maximum number that computer software programs can reach and stay positive is 2147483646. When you add one more second to it – 2147483647 – the positive sign will become a negative sign, and instead of Tuesday, 3:14:07 on January 19, 2038, computers everywhere will display the time as Friday, 8:45:52 on December 13, 1901.

Can’t this be fixed? Can’t we just ignore the date and move on?

Unfortunately, it’s not that simple. Most every operating system stores system time as a 32-bit integer, and system time is a very big component of a functioning software program (they absolutely need to be able to come up with a positive time stamp). So, it’s not an easy fix – most likely, entire software programs will need to be re-written and re-programmed to avoid Y2038K.

This includes personal computer operating systems, ATM machine software, other electronic devices with computer-like components, and, yes, Web Analytics cookies.

Okay – Y2038K? Give me a break – this is TWENTY-NINE and a HALF years away! I think you’re jumping the gun here.

You’ll be surprised how fast 29 and a half years goes by in computer programming. Think of this – we’re in the year 2009, and we’re using a timestamp that starts counting seconds from 1970 (39 Years Ago), which was first published in 1988 (21 Years Ago). Most of us are still using Office 2003 (6 Years Ago).

29 Years is right around the corner – so I hope that we can come up with some kind of conversion tool, some type of new timestamp calculation, some new 64-bit integer system that can seamlessly transition all software programs and Web Analytics Cookie Timestamps for the next generation!

*Note: Some of this blog post is obviously “tongue and cheek”. I am not really sounding the general alarm about what will happen in 2038 – but hey, it’s never too early to start planning for the future! :)”

January 5 2009

What is your competition up to these days?

by

Happy New Year readers! 2009 is going to be a comeback year for everyone – I can really feel it! Let’s make the most of this fresh and exciting New Year by stepping away from our Analytics kingdom for just a little while and focusing on our Site Intelligence efforts, such as, stepping onto the dark side…I mean…your competition!

While it is illegal to use “black hat” techniques to keep tabs on your competition (such as using “spy ware” type software programs), keeping your eyes and ears open to what your competitors are doing is a critical part of being successful online. Knowing what your opponents are up to can give you great ideas and inspiration for your own website or marketing efforts. It can (and should) also serve as an alert or warning system as to what not to do online, which can be equally as important for you.

Here are just some of the many different things you can do to stay on top of your competitor’s efforts:

1. Visit their website!
Pretty simple, right? If you know the URL of your competitor’s websites, check it out to see what they have going on. Pay close attention to how they market to their potential customer base, the language they use, and the type of sales angle that they incorporate. Be observant of the layout of their website, color scheme, navigation, and – of course – products and services offered.

2. Search for them online
See how successful (or how futile) their online marketing efforts are by searching for their brand name. You don’t have to click on their ads or organic search results – just look them up on Google and Yahoo a few times. Here, you’ll want to take note of how aggressive / passive their marketing language is, and what incentives / discounts / promotions they are offering.

3. Sign up for their newsletter / monthly email alerts / RSS Feeds
This is an excellent way to learn “what’s hot” with you business adversaries. Normally, your competitors will promote the latest and greatest product or service to their email database, including any speaking engagements or other important announcements that they have. Learn how they speak to their database and what re-marketing efforts they are using, and consider similar methods for your own Email marketing efforts (if they are using good methods).

4. Check out their social media / blog / viral marketing programs
Are your competitors present in Facebook, LinkedIn, and Twitter? Do you know what StumbleUpon and Digg are? When was your competitor’s last blog post? If your competition isn’t focusing on any of these newer mediums, then this is an excellent opportunity to gain ground and establish a presence before they catch on. If they are already engaged in Social Media efforts, consider subscribing to their blog, their RSS feed, and “follow” or “connect” with them. They will most likely speak to their audience much differently in these mediums than they will on their website or newsletter – take note of what they are saying and doing here.

5. Do your competitors advertise / market offline?
Have you seen your competitor’s brand and products in a magazine or newspaper? Are they running a late-night infomercial or day-time TV ad? Have you heard their phone number repeated 9 times in a 30 second radio spot on your drive home from work? You may not be able to afford these mediums as they are FAR more expensive than pay-per-click advertising, but it’s good to pay attention to their offline messaging – visit their site the next time you’re in front of your computer and see if there is a connection between the ad that you read or heard and their website.

6. Look at your Referring Sites / Hostname / Domain Name Reports
Your analytics package should be able to tell you what websites have been sending you traffic, and, what domains are delivering you traffic. This is a great way to tell if your competitors are checking you out. Fight the urge to block out or exclude this traffic from appearing in your reports – keep this valuable data in your analytics package. If your competitors are really checking you out, chances are that you are doing something that has caught their attention, and you are most likely going down the right path.

7. Enable Data Sharing / Benchmarking (with Google Analytics)
Google Analytics allows for you to compare your basic website’s metrics against the averages of websites that are a similar size to yours. This is available within the Visitors >> Benchmarking report. The catch: You must anonymously share your data with Google and other services, such as AdWords, to be able to have access to this section. The benefits of knowing how you stack up against websites in your industry – and across every available industry in this section – far outweigh the risk of anonymously sharing your data with Google (keep in mind they already have your website data when you use Google Analytics, so it’s not that much of a leap of faith to enable Data Sharing in your Google Analytics account).

8. Use online research tools like Google’s Insight for Search!
Finally, get a grip on historical and current trends of keywords and key phrases with free programs like Google Trends for Websites and Google Insights for Search. You can perform searches for your competitor’s brand names and products, and you’ll be able to gauge the level of interest at global, national, and regional levels. If there are terms or key phrases gaining popularity that your competitors are using, you may want to consider jumping on those while they’re hot.

Checking out what your competition is doing can help guide your own efforts, as you learn what to do and what not to do. However, always keep in mind to play fair and behave in an ethical fashion – NEVER slander, defame, or bad-mouth your competitors on your website, your blog, or on your MySpace page. Don’t click on their pay-per-click ads, never subscribe their contact or info email to spam mailings, and refrain from posting negative reviews of their YouTube videos or their Local Submission listings.

Have a great 2009!!!

December 11 2008

The Three Evils of Analytics Tracking: Images, Javascript, and Cookies

by

Tag-Based (or script-based) Web Analytics programs have excellent, business-friendly advantages that help organizations make intelligent, insightful decisions about their website. This applies to websites from businesses across every imaginable industry and size, from the local flower shop to the U.S. Government. These benefits include (but are definitely not limited to):

– Full suites of reporting options
– Colorful, easy-to-use graphics and reporting interfaces
– Data that is “good enough” for any marketer or decision-maker to use
– Fast and almost always reliable data

But with script-based web analytics – and like anything in life, really – there are pros and cons, or, the good and the bad. Unfortunately, not every visitor can be tracked with script-based web analytics programs. Some individuals purposely configure their browser settings to block web analytics tools from tracking and collecting data; others have no idea that their browser settings are configured in a fashion that would block or interfere with the data collection process. Most website visitors using mobile phones simply do not have the technical capabilities to be tracked by web analytics programs.

What this means is that tag-based web analytics solutions can only track visitors that allow themselves to be tracked. There are three separate elements that users can restrict on their browser of choice, rendering script-based analytics programs completely helpless. Users can block or restrict images, JavaScript, and cookies from loading or processing – blocking or restricting any one of these means “No Soup For You!” I call these elements the “Three Evils of Analytics Tracking” (Sounds scary, doesn’t it?).

Images
A part of how web analytics programs (like Google Analytics) operate is by requesting a 1×1 invisible GIF image to the Google Analytics servers for storing and processing the data it has just collected. If a browser does not have images enabled for whatever reason, this request cannot be satisfied, and data – although collected – cannot be sent to Google Analytics for processing, hence, no data in reports.

This doesn’t affect too many folks, as almost everyone has their browsers set to load images, and only a very small percentage of the population even knows how to do this in the first place. However, this is a major problem when tracking things like Email Open Rates, which in most (if not all) cases are handled by a request for a 1×1 clear pixel GIF image to the necessary server. If a person does not click on “Download Images”, that person is not able to be tracked.

JavaScript
The main logic behind all tag-based web analytics programs is JavaScript. JavaScript is easy and fast to implement, and it’s the type of web analytics tracking solution that makes the most overall sense across the board. It is with a few lines of JavaScript code that a website can set cookies on a person’s computer, collect data, and send that data to the appropriate processing server, be it an in-house server or a data warehouse of some kind. However, not all that glitters is gold. If users have JavaScript disabled, they cannot be tracked – it is that simple.

Luckily, not many folks disable JavaScript, as it is such a commonly used language, present on almost every website out there. However, a very small percentage of folks do block JavaScript, which is unfortunate for anyone involved with Web Analytics. This really affects mobile phone users in a big way – since the browsers on a lot of mobile phone platforms cannot execute or understand JavaScript, they cannot be tracked by default. The only thing that anyone can do about this is to hope that soon enough, all mobile phones will be equipped with a JavaScript-executing browser.

Cookies
Cookies are very small files that get set by websites on a person’s computer. These small files collect information pertaining to their activity on a website: when they entered the site, when they left the site, where they came from, what source of traffic brought the person there, how many times a person has visited the site, and so on. Cookies come in many different shapes and sizes, life spans, and security levels, but if any of them are blocked or disabled by users on their favorite browser, web analytics programs cannot store or collect data about these individuals.

Unlike Images and JavaScript, Cookie “management” is a very big concern, and it’s the biggest evil of the three. Some people block only specific cookies from specific sites. Some people block all cookies, regardless of where they originate from. Others have daily, weekly, or every first of the month cookie deletion parties on their personal computers, where they wipe off every cookie imaginable. All of these actions hurt tag-based web analytics programs, un-purifying data and distorting figures. This affects a very sizeable portion of the population – some independent reports have this figure at 3 or 4% of all internet users, while other reports have this figure in the high teens / low 20’s.

So how do I know that my data is “good”? Should I be worried about this?

This shouldn’t be something that you lose sleep over, but you definitely need to be aware. If data quality is something that your organization simply cannot live without, tag-based web analytics solutions are going to give you a lot of headaches – you want to consider using log-file parsing programs or packet-sniffing programs, although there aren’t too many of those programs available anymore. You may also want to consider using raw server log information to help.

If tracking every single person that visits your website is not the most important thing – that is, you can live with being a few percentage points “off”, and a little margin of error, then you really have nothing to worry about. Web analytics programs weren’t designed to collect the exact number of hits or queries like your server is configured for – web analytics programs were designed to give you valuable insights about your website’s performance, which can effectively be accomplished with the percentage of data that they can collect for most companies.

© 2018 MoreVisibility. All rights reserved