Validate test scenario steps with the data recorded and stored by the platform used for reporting, segmentation and re-targeting. A step-by-step guide on how to use DDX API and Postman to test IBM Digital Analytics data.

Recently I was working with a retail customer who wanted to perform some testing of their website tags and the data received by the analytics platform. Their chosen analytics platform is IBM Digital Analytics which provides metrics for eCommerce activity (product views, purchases etc), marketing acquisition, conversion attribution and segmentation capabilities. It also has pre-built integration into other marketing tools and platforms, and provides APIs for reporting and custom data integration.

The context for this test scenario was that we wanted to test both the analytics tags firing when viewing the site pages and validate the data received by the analytics platform. So, we perform a test user journey on the customer site and then extract this session data from the platform to validate the pages visited are recorded on the platform and are therefore available for reporting and segmentation. In this scenario I focus on the page level session data, but you may extend this to included session data about on-page events, conversions, etc.

The feature in IBM Digital Analytics that enables you to retrieve this session data without having to wait for a full ETL routine is the Digital Data Exchange (DDX) API. DDX allows you to generate API URLs to extract data that is loaded every 5 minutes from the analytics platform.

  1. Define and execute a test scenario
  2. Create the DDX API to locate the first page of the test scenario
  3. Use Postman to execute the DDX API (record the session id value)
  4. Create a second DDX API using the session id as a filter
  5. Use Postman to execute the DDX API to retrieve the specific analytics session data

Define test scenario

The first task is to define the test scenario which defines a web site journey:
Step 1: Open Chrome and enter http://www.website.com?journey=test
Step 2: Click Furniture
Step 3: Click Bedroom Furniture

The above is a simple test journey but could be much longer. The key thing to note is the query string on step 1 which will allow us to identify the test session id using the API.

Complete the above test scenario in a browser. You can also use Dev Tools to validate tag execution.

Create the DDX API URL

To create your DDX API URL you need to login to DDX (http://tagmanager.coremetrics.com) and then access the API left-hand menu option.

Blog DDX picture

You can build your own API from scratch or access some pre-created ‘Best Practice API Feeds’. In this case we will make use of the ‘Best Practice Page View Events’ feed. Add a filter for Destination URL contains journey=test. You can also add or remove fields to the Selected Data Fields list.

Blog Best practice API

Then you can click ‘Generate API URL’ and DDX will generate an API that you can use to extract the data you require. Copy the generated API URL.

https://welcome.de.coremetrics.com/analyticswebapp/api/1.0/report-data/ddx/pageviews.ftl?clientId=90397088&username=user1&appendtofilter=load_id+eq+1460462100&userAuthKey=01c26b5d-9436-403d-ac04-938292f13d4d&viewID=default.ftl&format=xls&language=en_US&filter=DESTINATION_URL+cnt+”journey=test”&metrics=LOAD_ID,CLIENT_ID,COOKIE_ID,SESSION_ID,EVENT_DATE,PAGE_ID,DESTINATION_URL,REFERRAL_URL,ONSITE_SEARCH_WORD,SITE_LOCATION_ID,SITE_LOCATION_NAME,SEARCH_RESULT_COUNT

Use Postman to execute the API URL to retrieve the Session ID

Postman is a Chrome app used for testing APIs which if you haven’t already got it can be installed from http://www.getpostman.com.

Paste the DDX API URL into the request field and then click ‘Params’. Edit the following parameters:

appendToFilter: load+gt+0
format: csv

Editing the appendToFilter parameter tells the Digital Analytics platform that you are filter all of the DDX analytics data. Then click ‘Send’ to make the API request. The returned data will be displayed at the bottom of the Postman window.

Blog Append to Filter

Make a note of the value in the Session ID column.

Use Postman to execute the API URL to retrieve the full session data

Now we can use the Session ID value retrieved in the previous step to create a new API request in Postman. Either edit the previous API URL directly or save it as a new API request. The only parameter we need to edit is the filter field:

Filter: SESSION_ID+eq+ <YOUR_TEST_SESSION_ID>
e.g.
Filter: SESSION_ID+eq+ 4546415550955596761

Then click ‘Send’ to make the API request. The data returned will be all of the test scenario page views recorded by the analytics platform. The data returned can be saved in various formats, and Postman even generates code snippets in various languages so that you could automate this testing process.

Blog test scenario

You are then able to validate the test scenario steps with the data recorded and stored by the Digital Analytics platform used for reporting, segmentation and re-targeting.