Quickstart Guide to Google Analytics Clickstream Data

This post provides downloadable Google Tag Manager (GTM) scripts and a walkthrough of the necessary steps for getting clickstream data out of Google Analytics. This clickstream data already exists behind the scenes; every time a user takes a tracked action on your website, like viewing a page or clicking a button, the data about that action is sent to Google Analytics as a hit. But by default, Google Analytics doesn’t expose this granular hit-level data — instead all reports are aggregated together by location, marketing channel, and so on.

clickstream-data-img0

This makes sense within the Google Analytics interface where a firehose of raw data would be more of a hindrance than a help, but the underlying raw clickstream data has many use cases — valuable enough that access to this data in BigQuery costs $150,000 annually via Google Premium (don’t worry, of course we’re going to get it for free here 😀 ).

NEW! API CONNECTOR ADD-ON FOR GOOGLE SHEETS

Check out my new API Connector Add-on to import data from thousands of platforms (e.g. Shopify, Harvest, Mailchimp, ActiveCampaign, VWO, YouTube, etc.) directly into Google Sheets.

THE VALUE OF CLICKSTREAM DATA

If you’re here, you probably already know why you want clickstream data. It enables you to merge Google Analytics data with a CRM, integrate GA data into a data warehouse, perform cohort analysis, answer questions about lifetime value, and run all kinds of complex queries and advanced analytics that are inaccessible from the standard reports.

Simo Ahava gets a lot of credit as this system was inspired from his posts on improving data collection, using custom tasks, and leveraging the ‘transport : beacon’ field. So definitely read his posts for background information. This article slightly modifies, condenses, and simplifies the steps, and compiles the variable scripts into a downloadable container so you can get a quick start.

STEP 1: SET UP CUSTOM DIMENSIONS

In Google Analytics Admin, navigate to Admin > Property > Custom Dimensions > +New Custom Dimension.

In the new custom dimension interface, set up the following 4 custom dimensions:

  1. Name = Timestamp, Scope = Hit
  2. Name = GA Client ID, Scope = User
  3. Name = Session ID, Scope = Session
  4. [optional] Name = User ID, Scope = Hit. Use this if you’re passing an internal ID to the data layer that identifies your logged-in users, as described here

Once these have been added, note the numbers assigned by Google Analytics in the Index column. We’ll need these later, in Step 5.

clickstream-data-img1

STEP 2: DOWNLOAD THE SCRIPTS

Here I’ve compiled the scripts for these 4 dimensions into a downloadable container you can import into your own GTM account. Right-click the link and click ‘Save link as’ to save them to your own computer: GTM Clickstream Variables

For reference, this is the content of these scripts:

  1. JS Timestamp
  2. GA Client ID
  3. Session ID
  4. User ID
    Data Layer Variable Name =

STEP 3: IMPORT CLICKSTREAM SCRIPTS INTO YOUR OWN GTM CONTAINER

Follow the directions below to import the scripts into your own GTM container without affecting anything you have there already.

In Google Tag Manager, navigate to Admin > Container and click Import Container

clickstream-data-img2

Click Choose Container File and import the json file from step 2

clickstream-data-img3

Choose New workspace. Name it Clickstream and click Save.

clickstream-data-img4

Choose the options Merge and then Rename conflicting tags, triggers, and variables

clickstream-data-img5

Click Confirm and complete the import. Following the above steps will ensure that none of your existing tags are changed or overwritten.

You will now have 4 new Variables in a workspace named Clickstream: {{JS – Timestamp}}, {{JS – Session ID}}, {{JS – GA Client ID}}, and {{DL – UserID}}.

STEP 4: ADJUST THE NEW VARIABLES

These variables are almost ready to go. You just need to make one modification:

In the {{JS – GA Client ID}} Variable, click into it and add in the index number assigned by Google Analytics in Step 1 to your GA Client ID custom dimension. In my case it was “4”, but remember to use your own assigned index number.

clickstream-data-img6

STEP 5: ADD THE NEW VARIABLES TO YOUR TAGS

The most convenient way to add these new variables to your tag is to update your Google Analytics Settings variable (located at Variables > User-Defined Variables). Make the following changes:

  1. Add Field Name = customTask, Value = {{JS – GA Client ID}}
  2. Add Field Name = transport, Value = beacon
  3. For each of the other 3 new dimensions, add in the index number assigned by GA in Step 1, alongside the Variable name.

clickstream-data-img7

Tracking ID should of course be your own GA account ID. If you aren’t using the Google Analytics Settings variable, you can find these same options by clicking into your tags and clicking the “Enable overriding settings in this tag” checkbox.

Note: the {{DL – UserID}} Variable requires that you are pushing your internal user ID to your site’s data layer. You can leave it out if you’re not using it. If you do include it as a custom dimension without actually pushing a user ID to the data layer, it just won’t populate.

STEP 6: TEST AND PUBLISH

Always test to make sure everything looks good in Preview Mode before publishing to production. Click around your site and manually verify that the Variables are populating correctly, like this (note that JS – GA Client ID will contain a function rather than the actual ID):

clickstream-data-img9

Also click into your page view tag and verify that the Variables are being passed inside it, and are populating the correct index slot number.

CONCLUSION

Once you’ve followed the above steps, you’ll have access to raw clickstream data from your GA reports. This is an example of what the data will look like (Click to import this custom report into your own GA account: https://analytics.google.com/analytics/web/template?uid=sbXi8C3HQGukKWmO_39GPg)

clickstream-data-img8

Of course you can also access this data via the API, or send elements of it directly to your CRM upon form submission (if your form links directly to your CRM), or programmatically send this data to your data warehouse daily so you can run SQL queries on top of it, or push the data into BI software like Tableau… having this data available opens up all kinds of custom possibilities.

Have you implemented clickstream reporting or enabled any useful integrations? Let me know below.

Additional Reading:

  1. Google’s documentation on Integrating CRM Data with Google Analytics
  2. a complete example of how to pull the GA Client ID and pass it in a form

Comments:2

  1. Great post, thank you. I have used GA free in this way to bring clickstream to Snowflake Db and report in Tableau. I havent hit limits with volume, I did have to split the API queries as I could only pull 7 dimensions at a time, and also they had to be phased hourly as there is an API limit of querying <500k Sessions. Are you aware of any other limitations we should be aware of if depending on GA (free) for clickstream?
    Thanks for your time!

    1. Those are really good points. Querying via the API does have several limits:
      1) as you noted, you can supply a maximum of 7 dimensions in any query. (documentation) If you need more, you need to pull them separately and group them back together using a shared key like the client ID and/or session ID.
      2) Not all dimensions and metrics can be queried together. Certain combinations of dimensions and metrics are invalid, so you may not be able to query all dimensions at the same time. The Dimensions and Metrics Explorer tool shows what combinations are valid.
      3) Some combinations may be technically valid but won’t make sense together, for example pages and events. This combination would just provide the pages that events occurred on, rather than a list of all pages and a list of all events. For a clickstream report you’d usually want both page views and events, so these need to be split into separate queries.
      4) You can not send a query composed only of dimensions: Requests must specify at least one metric (maximum of ten).
      5) The Analytics Core Reporting API returns a maximum of 100,000 rows per request, no matter how many you ask for.
      6) I’m not sure about an API limit of querying 500k sessions, but there are various limits and quotas on API Requests, as described here: https://developers.google.com/analytics/devguides/reporting/core/v4/limits-quotas. The main one is a 10,000 limit on the number of requests per view per day.

      Unless you have massive volume, you should be able to extract all your data daily with the limits of 10,000 requests a day and 100k rows in each request. Further, #2, #3, and #4 apply to the paid version as well. So none of the above should be deal breakers, though ultimately it will depend on how much work you want to put into extracting and manipulating the data vs. paying for Google to push it into BigQuery for you.

Leave a Reply

Your email address will not be published.