A recent comparison undertaken by our team examines the difference between Google Analytics and Omniture Site Catalyst’s tracking of page views for a specific section of a heavily trafficked website.
The methodology is simple: over a specified time frame, we pulled page view data for the top 100 viewed pages in that section of the site, using each tool. Since the order is a bit different on each list, we used some Excel wizardry to match up the two page view lists with the appropriate page.
the relationship between page view counts on the two tools is extremely strong,
Even though the individual numbers are different, the overall data sets can be considered roughly equivalent.
When the Google count increases, so does the Site Catalyst count, albeit at a slightly different rate; this led us to our second analysis.
In order to further examine the data, we needed to see how the difference in page views between the two tools was distributed.
The overall message to the client should be that the Google data will be just as accurate as the Site Catalyst data, but Google will, on average, return page view results that are ~2% lower, which is the difference between the mean of our data, 1.02, and 1 (which represents an even ratio).
After presenting the client an analysis like this, they should be encouraged that Google’s tool should return equivalent results at a significant cost savings.
As an analyst, it is also important to explain that for data results gained over time, you must avoid comparing data from multiple tools, since we have shown an inherent difference between the two; this path will only lead to madness.