HTTP SNIFFING FOR WEB ANALYSTS - WHY?
As web analysts we need to be able to see and test the technical details of tags, so that we know exactly how the data is being collected. For Coremetrics, however, one thing that can be difficult about viewing and debugging tags is that the official Coremetrics tag viewer, TagBar, only works for Internet Explorer and some early versions of Firefox. So this post will walk you through how to use an HTTP sniffer to view tags, audit, and debug Coremetrics implementations. Don’t worry if you’re not very familiar with HTTP sniffers – they are just software that lets you see all the traffic being sent from your browser to the internet. There are several benefits to this technique:
- it will work for browsers like FF and Chrome, not just whatever old version of IE TagBar currently works with.
- it can validate what you're seeing in TagBar if you need a second opinion 🙂
- the output can be easier to read than TagBar since it both stores the calls and displays them separately, rather than listing them all one after the other
- there might be some cases where you don’t have access to the official Coremetrics TagBar since their tool can only be downloaded while logged into a Coremetrics account.
- the method here can actually be easily adapted for other web analytics tools. I’m just focusing on Coremetrics since Omniture and Google Analytics have quite a good ecosystem of tools that are even better designed for this purpose, while Coremetrics does not -- for example I previously wrote about tools for debugging Google Analytics, but none of those tools work very well (or at all) for Coremetrics.
So, here we go.
HTTP SNIFFING FOR WEB ANALYSTS - TUTORIAL
1) Get an http sniffer (also known or available as web debuggers, packet sniffers, http analyzers, etc, but we don't need anything too complicated here)
I’m going to recommend HttpFox. It is a free, simple plug-in for Firefox that works great for this purpose. Download it here: https://addons.mozilla.org/en-us/firefox/addon/httpfox/
(Chrome has a built-in sniffer that you can access by navigating to the "network" panel of Chrome's developer tools and clicking on the "Headers" tab. However, this tutorial will focus on HttpFox for Firefox because it's easier to use).
2) Install the plugin and restart Firefox. Activate it by navigating to Tools > Web Developer > HttpFox > Toggle HttpFox (or just hit Shift + F2)
3) Click the “play” icon and make sure Start is greyed out.
4) Now the fun starts. Navigate to the site you’re checking out. For the purposes of this tutorial, I’m going to Costco.com.
5) Oh no! There are hundreds of calls.
6) We need to filter these so we just see what we’re looking for: the calls to the Coremetrics servers. Type in “ci=” into the search filter box. This will limit it to just calls with the text “ci=” in them, which shows up in all Coremetrics tags (more on that in a minute). You could probably choose some other text but I’ve found this to be reliable across any site running Coremetrics, and excellent for filtering out the non-Coremetrics calls as well. Now we'll see just the Coremetrics calls.
7) You can click around the site and these calls will persist in the top section, with the most recent calls appearing on the bottom. Click on the Query String tab to see a parsed list of what is being sent to Coremetrics.
8) Now let's look at an example result and dig into it. Here is the first call that gets sent from Costco.com to Coremetrics.
You can see a long list of query strings. Let's go through one by one:
|st||Session start time|
|vn1||Coremetrics library version|
|ec||Character set encoding|
|vn2||Coremetrics library version #2|
|pi||Page ID (page name)|
|ul||Destination URL. The URL of the page.|
|tid 1 = page view tag|
|tid 2 = registration tag|
|tid 3 = order info tag|
|tid 4 = shop tag (combine with action 5 for shopping cart tag, combine with action 9 for order receipt tag)|
|tid 5 = product view tag|
|tid 6 = technical properties tag|
|tid 7 = custom details tag|
|tid 8 = link click tag (for real estate/ site promotions click tracking)|
|tid 9 = link impressions tag (for cm_sp impression tagging)|
|tid 10 = form action tag (auto-generated for forms)|
|tid 14 = conversion event tag|
|tid 15 = element tag|
|tid 404 = error tag|
|cg||category. This needs to match an entry in the Category Definition File (CDF)|
|rnd||Random # (used for cache-busting)|
|pc||Boolean answering “is this a page view?”|
|np0...np11||Technical browser properties|
|je||Boolean answering "is Java enabled?"|
As you can see, this was a Technical Properties Tag, triggered since it was the first page of the visit. The standard Page View tag and Product View tags would look very similar.
If we next add a product to cart, a Shopping Cart Tag (aka Shop Action 5 tag) fires and HttpFox displays the following:
Now we can see some additional query parameters, described as follows:
|at||action tag ID tag|
|at 5 = Shop Action 5|
|at 9 = Shop Action 9|
Besides these, you will come across Order Tags, Registration Tags, Element Tags, Conversion Event Tags, and Error Tags (deprecated since 2010, but you may see them in older implementations). Each of these will contain slightly different parameters (and a lot of overlap) but the basic method of viewing them is the same.
Just for completeness, here are the remaining parameters you're most likely to come across:
|hr||href (look here to check cm_sp site promotions and cm_re real estate tagging)|
|cid||conversion event ID|
|ccid||conversion event category ID|
|pr_a1...pr_a5||extra product attributes|
|pv_a1...pv_a5||extra page view attributes for use in Explore (will not show up in regular reports)|
Again, HttpFox will persist all the calls in the top, so you can click around on the pages you're interested in, and then review what was sent.
I hope this tutorial helps you understand exactly how to debug Coremetrics when you aren't able to use TagBar.
**9/19/12 Update: This post contains the complete list of query strings for reference.