A View of the Product Analytics Stack
Explore ways to get data insights to generate more actionable insights
Today, every company is a data company, whether they leverage data as a core asset. Companies with the strongest data-informed cultures have a thoughtfully created agile infrastructure and a culture of monitoring and experimentation.
“Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day.” — Jeff Bezos
“One of the things I’m most proud of that is really key to our success is this testing framework.… At any given point in time, there isn’t just one version of Facebook running. There are probably 10,000.” — Mark Zuckerberg
The rise in data-driven products has created a demand for an entire industry focusing on multiple parts of the data stack, helping companies effectively utilize data to drive impact and value.
Here’s an attempt at looking at some of the players in the booming data landscape — and oversimplifying what they do to build a mental map.
Data Collection
Customer data comes in many forms. Here’s a few examples:
- User behaviour and clickstream events on web and mobile apps
- Transactional data from customer SaaS apps for marketing, sales, and customer support.
- Third-party data from external platforms like Google Ads.
- Processed Data ( “computed” transactional data, customer scores, recommendations from AI/ML systems )
The most common way to collect product data is Google Analytics JavaScript snippets embedded in the website. You can make more advanced analytical calls to fetch deeper customer insights.
Tools like Segment simplify collecting data from apps, servers, websites, etc., and connecting new tools. For example, Event calls and Identify calls in Segment help you get visibility of user info and granular events.
Companies like Freshpaint and convizit provide no-code solutions to capture data. Once installed, a non-technical user can easily set up a collection of new data.
Customer data platform
A customer data platform (CDP) is software that collects and unifies first-party customer data — from multiple sources — to build a single, coherent, complete view of each customer, powering the marketing stack.
Customer data platforms — like Segment, RudderStack, mparticle — process the raw data to make it more usable and are designed to drive customer interaction. Data warehouses and data lakes are not tailored to a marketer’s needs; CDP fills that gap.
Data Storage
Data store brings the data into one central location to be analyzed, reported, and used across the company.
- Traditional on-premises BI/DW (on-premises Business Intelligence and Data Warehouse): Oracle, Teradata, Netezza
- Big Data Platforms: Hadoop, Spark, Elasticsearch
- Cloud data warehouse, data Lake: Amazon Redshift, Hive, Snowflake, Databricks, Azure
Cloud data storage is the most popular because of its ease and pricing. DataWarehouse, Data Lake, and Data Lakehouse are different cloud storage patterns for different needs and kinds of data.
Transformation tools
Sophisticated analysis on top of the warehouse data requires transformations and processing — these are more effectively handled in-database rather than in some external processing layer. dbt enables data analysts and engineers to transform data in their warehouses more effectively.
dbt does the
Tin ELT (Extract, Load, Transform) processes; it doesn’t extract or load data, but it’s extremely good at transforming data that’s already loaded into your warehouse.
Metrics Store
With multiple data sources, data being transformed multiple times, updated at different places, and multiple teams handling it — there’s no central repository for defining a metric. This is scattered in dashboards, recreated, and redefined inconsistently at different places.
This technical debt is already creating issues and is proposed to be solved by a metrics store — a layer missing in most modern data stacks.
Metrics Store is a central repository so teams can consistently reuse metrics across BI, automation tools, business workflows, or even advanced analytics. Read more on understanding metrics store as implemented by Pandora here.
The benefits of a Metric Store in achieving metric consistency, scalability, and speed were seen most famously at Airbnb.
Analytics Tools
Business Intelligence (BI) Tools
A BI tool provides direct access to data to be visualized and analyzed — they’re great for a wide range of structured and unstructured data from different kinds of datasets — marketing, finance, and product.
Popular BI Tools are Tableau, Power BI, Mode, and Looker. Some new tools like Transform and Trace also provide a metrics layer.
Product Analytics Tools
BI tools are excellent for visualising any data, but this generality comes with the drawback that it is time-consuming to set up and harder to derive deeper insights. Product analytics tools like Amplitude, Heap, Pendo, and Mixpanel fill the gap by providing easier integrations and deeper product insights like funnel conversion, cohort analysis, etc. Check out this report by G2 analyzing top players in product analytics.
As the number of metrics and dashboards explode, there is a growing need for products like Thoughspot, Sundial, and Tellius to provide automated, actionable insights.
Business Applications
There are a few other common downstream SaaS apps. Some of them could serve multiple purposes and don’t fall into one bucket:
- Marketing automation, analytics, and testing tools like Marketo, Customer.io, Iterable, Mailchimp, and mobile and push notification tools like Braze, Twilio
- Experimentation tools like VWO, optimizely
- Advertising platforms like Google, Facebook, Twitter, AdRoll, LinkedIn Insight Tag
- Sales and customer relationship management systems like Salesforce, HubSpot, Pipedrive, Freshworks, and customer support systems like Intercom and Zendesk
Some of these apps may be a part of the stack, depending on their use. They use data insights and feed data into the system to generate more actionable insights.
If you find this useful, hit follow and subscribe here: Khyati Jain. It goes a long way in encouraging me to keep writing :) You can also find me on twitter jnkhyati
