Simulator for Learning Data-Driven Product Management
You will learn about basic metrics, the difference between correlation and causation, percentages and percentage points, arithmetic mean, and median. You will also get to know Instachat, a messenger that uses LLMs to predict replies and save users time and attention.
The story of how Slack became a market leader and then took on Microsoft Teams, plus the history of the browser wars as a bonus case. A good analogy to what your new company wants to achieve.
The first task from a Chief Product Officer. You will learn why key risks should be removed before a test launch, set up the Claude connector for Amplitude, calculate the basic metrics of the app, and make the first conclusions about the product and its potential.
While everyone prepares for the launch, you have time to study the market using publicly available data. You will evaluate the audience and download figures of competitor apps and make your conclusions about the target market for the company. You will also learn how to use AI tools, including deep research, for market analysis — and why their results always need to be validated.
Big day for the company — the public launch of the product! Plus a first look at a new AI feature for selfies.
You will understand the nuances of retention metrics and how retention is calculated in Amplitude. You will get the idea of how retention, new users, and active audience are connected. With the help of Claude, you will create a model for predicting the audience to assess how changes to key metrics impact growth.
You will find out more about different kinds of metrics and what they’re used for. You will also get up-to-date retention benchmarks for different product categories.
You will use cohort analysis and the funnel tool in Amplitude to understand how the changes made in the latest version of the product have affected the key metrics — and learn why it’s so important to compare the right things.
To compare the metrics of two versions of the product, it is not enough to calculate them. You must verify whether the change is statistically significant or not. You will learn to apply mathematical formulas — and to run the calculations in any AI chat — to draw relevant conclusions while comparing metrics.
Your manager tells you about a new AI feature that will be developed soon. You decide to take the initiative and evaluate its potential effect and usage: you segment users with the help of AI and create a product model to evaluate conversions of the key steps in the funnel.
You will learn what a value hypothesis is. Then you will formulate it and build a product model to better understand the product and how it actually works.
You will learn the difference between hypotheses and facts and check your product spidey sense by predicting the results of real experiments. You will learn to design experiments and interpret their results. You will design an experiment to test the value of the product’s AI features, calculate the sample size, and get unexpected results.
The unexpected results of the experiment have put the confused team in a bind. But you will find a solution by applying qualitative user research methods (interviews, surveys, etc.).
Time for one more educational story, uncovering WeChat’s path to dominating the Chinese messaging app market. This chapter will help you look at the product and things happening to it from a different angle.
You will learn how to apply a number of frameworks to find the hidden value of a product. You will observe something unexpected — or maybe not. It’s all up to you, remember?
You will use the insights from the previous chapter to change the product. And who knows where this will lead you?
You will have to step into the leader’s shoes and make tough choices. Using real-world examples, you will understand why it’s so important to manage product risks as early as possible.
You will explore the second key hypothesis while creating a product. You will learn why distribution is often the main bottleneck, study which growth channels well-known products used at the early stages, figure out which channels are worth testing, and prioritize growth hypotheses for the product.
It’s time to deal with marketing analytics. You will learn the unit economics of growth channels — from gross profit to ROI — and how to apply cohort analysis to them. You will also explore approaches to forecasting LTV (probabilistic and ML models) and understand why attribution and incrementality are so hard to measure.
The product is improving, but to make this process meaningful, you need to understand the target market. You will learn the Jobs to Be Done approach, identify the key jobs your product can do, and analyze competitors, market size, structure, and dynamics. You will also see how product, channel, and market are connected — and why value should be created at the product level rather than by competing on AI model quality.
The team begins testing paid acquisition. You will learn why creative is the new targeting, how to research competitors’ creatives and funnels, and how to figure out the maximum you can pay for a user (CPA vs. LTV). You will analyze the metrics of a creative campaign, see how channels like Meta Ads and TikTok Ads work, how ad tracking has changed with user privacy restrictions, and why some companies operate at negative ROI. Then you will decide whether to invest in the channel.
A theory chapter on search as a growth channel. You will learn the types of scalable growth channels, see examples of products that grow through ASO, SEO, and AEO/GEO, and learn how to assess the potential of each channel.
Despite the progress in many areas, you are far from the goal. The situation is complicated by the fact that the company is running low on cash. To get the next round of investment, you need to change something. You have to use all your accumulated knowledge about the product to make a tough decision.
The decision you made leads to unexpected results. You have to conduct an investigation using all the skills you have mastered. Who knows where it may take you?
Resolution of the story. Repetition and consolidation of all the skills and knowledge gained in the training process.