Img-Reboot

Enabled a global retailer to build a Marketing Analytics Platform to maximize marketing ROI

Summary

One of our largest retail clients wanted to build a comprehensive solution that could provide a detailed impact of their marketing spend on revenue to better understand which marketing campaigns were and weren’t effective and what variables drove ROI across different global regions.

Tredence’s world-class marketing analytics and data engineering teams joined forces to work with the client to design and implement a next-generation Marketing Analytics Platform on GCP.

The Goal

The goal was simple: build an exceptional Marketing Analytics Platform that would provide valuable insights across the organization. The platform needed to be able to provide some of the following features to analysts and users:

  • Actionable campaign direction from the previous week’s campaigns, not historical campaigns
  • Enable analysts to perform ad hoc analysis on the last day’s marketing data
  • Reduce total cost of ownership compared to earlier and alternative solutions
  • Deliver a suite of powerful and useful tools to enhance analysts' productivity
  • Provide access to a wide range of data sources, types, and formats, both internal and 3rd party in origin
  • Enable robust security features allowing analysts only to see what data they need to see and gain access to

The Challenge

The solution was ambitious and came with a variety of technical and business challenges that our team had to overcome:

  • Migrating legacy workloads into the Google Cloud Platform came with its challenges, as many of the pipelines were inefficient and had to be completely redesigned
  • The timeline was extremely aggressive – our team used an agile, iterative approach to quickly deliver value while still maintaining quality execution
  • The platform leveraged many new GCP services that our clients had not used before as part of our world-class solution architecture

Tredence worked closely with the client’s enterprise architecture team, our business stakeholders in the marketing analytics organization, and Google to design the optimal solution architecture to meet our client’s needs.

Approach

The Marketing Analytics Platform is based on a robust, scalable architecture that leverages many of the most advanced GCP services available. Our solution had a few key technical requirements that needed to be met, including:

  • Seamless data ingestion
  • Scalable data processing
  • A model-ready data lake and in-house attribution model
  • Ad hoc data exploration and data visualization capabilities

The services we leveraged within the GCP as part of this solution are listed below.  As can be seen, we used various services to provide a marketing analytics solution that met the client's requirements. 

Technology requirement

GCP service

Rationale for using the selected GCP service

Virtual Machines

Compute Engine

Google Compute engine provides the ability to create virtual machines that are required to -

  • Run the attribution model as a Restful service result consumed by a React application
  • Run the PM2 server that manages the React applications used by marketers and consumes the results of the attribution model
  • Run the Jupyter hub server that serves the Jupyter Notebooks used by analysts for ad-hoc data exploration

Object Storage

Cloud Storage

  • Provides a robust storage solution to store incoming flat/semi/unstructured source files such as marketing performance data, external factors, etc.
  • Provides a scalable storage solution to host the file system needed for the data lakes

Event Processing

Pub/Sub

  • Pub/Sub provides the ability to process incoming marketing performance data from 3rd party vendors as and when they arrive and update the marketing data lake 
  • Pub/Sub provided trigger notification for the Spark solution running on compute engine to start the data validation process and update the marketing data lake

Data Processing Engine

Dataproc

  • Dataproc is the recommended data processing engine when migrating on-premises Hadoop clusters to Google Cloud 
  • Dataproc gives the maximum features and flexibility of operating the clusters in the cloud while maintaining a low learning curve for the developers
  • Dataproc’s auto-scaling feature will help seamless processing during peak workload periods such as Black Friday and Cyber Monday sales. Scaling the clusters for such events previously with the on-premises Hadoop clusters was an extremely cumbersome process for developers and analysts.

Logging

Cloud Logging

  • Cloud logging provides a flexible and scalable logging mechanism that can store the log files on Google Cloud

Datawarehouse

BigQuery

  • Ad-hoc reporting over large historical data sets is a very resource-intensive operation and has been a pain point for the client. BigQuery is a highly scalable data warehousing solution that can support these use cases

The solution architecture diagram below showcases how we used the different GCP services to build the Marketing Analytics Platform.

Solution environment assessment with design diagram

Key Benefits

The Marketing Analytics Platform has created tremendous value since it was implemented. The solution has modernized how the marketing team thinks about, consumes, and uses data and insights to power decision-making. Post implementation, the Marketing Analytics Platform has helped to generate $150M+ of additional revenue through optimized marketing analytics campaigns during the Black Friday events. Additionally, optimizations that were generated by analysts using this platform generated an additional 15% lift year over year in performance marketing campaign efficiency.

Results

Icon Boost

The UDP since its inception have been processing 250 Tb’s of data weekly

Icon Boost

Overall reduction in turn-around time by 70 percent for computationally heavy jobs

Icon Boost

30-35 per-cent overall cost savings

Talk To Us