Category posts
Series
Oleg Ya
When I worked at Facebook, the Workplace analytics team had a cool tradition: The team’s weekly meetings always started with a small data quiz.
The winner of the previous week’s competition would prepare a question about the product’s key metrics. For example, “what was last month’s MAU?” or “how many new users joined last week?” or “what proportion of the new companies reach 10 users?” or “what was last month’s revenue?” The question had one requirement: Its answer had to be found on the team’s dashboard.

The participants were to write down the answer without getting help from computers, which meant we could only use our memory to do so. The person whose answer was closest to the correct number got +1 point in the chart, and the person who was the farthest lost 1 point. Every six months, a winner was chosen and the game started again.
I participated in five seasons and won three of them. In one of the final rounds, I was tied with another analyst. The team arranged the final round, where we had to answer five questions in a blitz quiz. I managed to score the winning point and won the mug that you see in the photo below.
I told this story not because I wanted to brag about winning the quiz (well, this too, to be honest). In almost every quiz, the respondents’ guesses on metrics were distributed across a wide range, which I found surprising.
Why? Well, first of all, it was the analysts who played the game. They were the people who worked with data most of their time and should have been good at navigating it. Second, these analysts were working at Facebook, a company that has a very advanced and strong data culture. At Facebook, each team has clear goals, dashboards are available to all the company’s employees, and all meetings start with progress updates on key metrics. How could these people be so wrong in answering questions about the product they were working on?
If you decide to play this game with your company’s employees, you will most likely be as surprised as I was. It will turn out that most people have very vague ideas about the key metrics of your product and business. And some people will have no idea at all.
In this essay, we will discuss why it is important for team members to remember at least approximate values of the key product metrics, why this usually doesn’t happen, and how to get there.
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
→ Join our discussion on LinkedIn. New topics to talk about every week.

Oleg Ya
The most common metrics gaming companies focus on are Day-1, Day-7, and Day-30 retention rate. While these metrics are of great help early in the journey, it’s long-term retention which is key to lasting success and a seat in the top-grossing charts. This post makes a case for long term-retention and why your focus should be first and foremost there.
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
→ Join our discussion on LinkedIn. New topics to talk about every week.

Oleg Ya
After my recent essay “Slack vs Teams vs Workplace: The intriguing dynamics of the work messenger market,” I didn’t plan to revisit the competition between Slack and Microsoft Teams just yet. Despite the rapid development of the work communication market, it is still a B2B market that is changing relatively slowly.
However, something extraordinary has happened: We are in the midst of one of the greatest “experiments” of our time, and a great part of the world has switched to remote work due to the COVID-19 pandemic. The outbreak has greatly sped up the development of remote work tools. This situation has propelled us several years ahead in time, much faster than it was destined to happen, allowing us to look into the future of the market today.
Another reason to take a closer look at Slack vs Teams is that recently, a lot of new data has surfaced about Slack. Even before the coronavirus outbreak, several large companies decided to adopt Slack as the communication tool for all their employees, including IBM with its 350,000-strong workforce and Uber, which has more than 38,000 employees. The number of paid Slack customers has grown by more than 7,000 over the past one and a half months, surpassing the growth in the entire previous quarter. This week, Slack CEO Stewart Butterfield shared the sequence of the recent events in a series of posts on Twitter, all of them conveying a clear message: Slack is growing very fast.
All of the above made me doubt the assumptions I’ve made in my past essays, in which I didn’t put too much faith in Slack’s chances of winning the race against Microsoft Teams. In light of the new data, I decided to take another look at the market and figure out what was going on.
And here’s what I realized: Slack is a great business, but it still stands no chance against Microsoft Teams in dominating the messenger market. The recent weeks have further confirmed this assumption.
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
→ Join our discussion on LinkedIn. New topics to talk about every week.

Oleg Ya
Watching new and rapidly changing markets can teach you many things. The work communication market led by companies such as Slack, Microsoft Teams and Workplace by Facebook is something I have been following for a long time.
Last year, before Slack went public, I did an analytical review of the data disclosed in Slack’s S-1 filing. At the end of that review, I shared my opinion that Slack experienced problems in the enterprise segment: the competition from Microsoft Teams and Workplace by Facebook for this market segment threatened Slack’s long-term growth prospects and its $20+ billion valuation.
A lot of things have happened in the eight months that have passed since I published that essay. A lot of new data has surfaced, with one of the biggest market intrigues fading away and a new one appearing. The leading characters once again reminded us of a number of fundamental rules the market plays by. And this is exactly what I am going to talk about in this essay.
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
→ Join our discussion on LinkedIn. New topics to talk about every week.

Oleg Ya
When conducting experiments, teams usually include all the active users in their tests, or sometimes they tend to add all the new users who join the app during this test. So when calculating the metrics for different test groups, all the data from the moment the A/B test kicks off is taken into account.
Today I’ll talk about how you can reduce the time required to get the signal on the change you are testing in a product. You can do that by changing the process of adding users to the A/B test, and in this essay I will show you how you can do it.
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
→ Join our discussion on LinkedIn. New topics to talk about every week.

Oleg Ya
Let’s start with a practical task.
Say a company’s management wants to allocate significant resources to the development of infrastructure that would increase their app’s speed. The hypothesis is that increasing the speed of the app will have a positive effect on the user experience and the key metrics.
Think of an experiment (an A/B test) to validate this hypothesis.
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
→ Join our discussion on LinkedIn. New topics to talk about every week.

Oleg Ya
It is common to evaluate your product performance and the impact of changes you make by using engagement metrics with active audience in the denominator. Examples of these metrics include the time spent per active user, the occurrence of certain actions (messages sent, levels played, chapters read, etc) per active user, or ratio metrics (what percent of active users perform a specific action) .
In most situations, these engagement metrics will be helpful. But in some cases, they can be misleading. And it is important to understand why and when this can happen, and what you can do about it.
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
→ Join our discussion on LinkedIn. New topics to talk about every week.

pic from http://mediainjection.com
(more…)Oleg Ya
Product/market fit is an important concept when working on a new product. All entrepreneurs and product managers are committed to it. But if you ask what the term means, very few will be able to give a clear answer. Even fewer will have an understanding of how we can measure product/market fit using metrics.
Without a clear definition, even the most useful concepts will be of little help when making decisions. In this post, we will discuss some of the most common product/market fit definitions and their advantages and disadvantages, as well as tell you about PMFsurvey.com (Product / Market fit survey by Sean Ellis) developed in collaboration with GoPractice, which is designed to give you an objective metric of how close are you to Product / Market fit.
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
→ Join our discussion on LinkedIn. New topics to talk about every week.

Oleg Ya
Finding insights and answers to questions in data is a key skill in product analytics. And developing this skill is the area where analysts usually see their growth potential.
Talking from experience, I strongly recommend paying attention to another aspect of analytical work: communication skills. Key here is not only finding insights, but also turning them into projects and making sure they convert into real value for users.
Getting to this point requires building relationships with the team, participating in key discussions, gaining credibility, and learning to present information in an effective way.
This article provides a series of recommendations for product analysts. However, it will be equally useful for product managers and executives who want to maximize the impact of analysts working in their teams.
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
→ Join our discussion on LinkedIn. New topics to talk about every week.

Oleg Ya

Retention rate is one of the fundamental metrics in product management. We all use it regularly, yet few of us know that there are many different ways to calculate retention rate. And it is very important to know which one to use when you’re making decisions based on retention data.
Let me start with a story. When I worked at Zeptolab (popular game development company, creator of Cut the Rope, King of Thieves, CATS) once we got an email from a gamedev studio that wanted us to publish their game. We were getting many similar emails, but that one got our attention. We were impressed by the metrics of the game, which had just recently soft launched. According to the developers, Day 1 retention rate of the game was over 55%, and Day 7 retention rate was over 25%.
However, when we started playing and testing the game, something felt wrong. The gameplay was not engaging enough to justify >55% Day 1 retention rate. And the meta game design was not good enough to users in the longer term.
Further investigation revealed that what this game development company called retention was actually “rolling retention.” Classic Day N retention of the game happened to be unimpressive.
This is just one example of how retention metrics can misguide you. There are many nuances in how you can calculate it.
(more…)→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
→ Join our discussion on LinkedIn. New topics to talk about every week.
Other content series
that you might find useful
- Addressing user pain points vs solving user problems better
- Product manager skills: evolution of a PM role and its transformation
- Product metrics, growth metrics, and added value metrics
- Customer retention levers: task frequency and added value
- How to measure the added value of a product
- Should a product be 10 times better to achieve product/market fit?
- Product/market fit can be weak or strong and can change over time
- Two types of product work: creating value and delivering value
- What is the difference between growth product manager, marketing manager, and core PM
- When user activation matters and you should focus on it
- User activation is one of the key levers for product growth
- The dos and don’ts of measuring user activation
- How “aha moment” and the path to it change depending on the use case
- How to find “aha moment”: a qualitative plus quantitative approach
- How to determine the conditions necessary for the “aha moment”
- Time to value: an important lever for user activation growth
- How time to value and product complexity shape user activation
- Product-level building blocks for designing activation
- When and why to add people to the user activation process
- Session analysis: an important tool for designing activation
- CJM: from first encounter to the “aha moment”
- Designing activation in reverse: value first, acquisition channels last
- User activation starts long before sign-up
- Value windows: finding when users are ready to benefit from your product
- Why objective vs. perceived product value matters for activation
- Testing user activation fit for diverse use cases
- When to invest in optimizing user onboarding and activation
- Optimize user activation by reducing friction and strengthening motivation
- Reducing friction, strengthening user motivation: onboarding scenarios and solutions
- How to improve user activation by obtaining and leveraging additional user data
- Tax/benefit framework for analyzing user activation
- How well do you articulate value during user activation? Check with the value communication framework
- How product teams get the “aha moment” wrong
- Slack vs Teams vs Workplace: the intriguing dynamics of the work messenger market
- How the “Slack vs Microsoft Teams” race evolves as the world switches to remote work
- How Revolut Trading was built. The importance of industry expertise and the balance of conservative and new approaches
- The values and principles of Wise. Key ideas from the Breakout Growth Podcast by Sean Ellis
- How to calculate customer Lifetime Value. The do’s and don’ts of LTV calculation
- How to calculate unit economics for your business
- Guide to ARPU: formula, calculation example, LTV vs ARPU
- Experiments where you make your product worse – the most underrated product manager tool
- Why your A/B tests take longer than they should
- Peeking problem – the fatal mistake in A/B testing and experimentation
- Mistakes in A/B testing: guide to failing the right way
- Designing product experiments: template and examples
- To reduce your product’s churn rate, first find out why users stay
- What is product/market fit and how to measure PMF
- How engagement metrics can be misleading
- How to forecast key product metrics through cohort analysis
- Cohort analysis. Product metrics vs growth metrics
- Correlation and causation: how to tell the difference and why it matters for products
- How product habits are formed and what dopamine has to do with it
- Hook Model: encouraging a product habit to improve retention
- Not every product is habit-forming, but all products can have loyal users
- How to design and run JTBD research interviews: guide and templates
- How to move from marketing to product management?
- Key data skills for product managers: experienced PMs sharing their thoughts
- Data mistakes to know and avoid as a product manager
- Data cherry-picking to support your hypothesis. What is it? Why is it bad?
- How can PMs encourage more teammates to use data?
- Metrics to focus on before and after product/market fit. How to better understand your product at different stages?
- How to establish effective collaboration between product managers and data analysts
- How to move from engineering to product management?
- Losing sight of real users and their needs behind the metrics. How can product teams avoid this?
- Quantitative vs qualitative data: what is the difference and when should you use one instead of the other
- Pros and cons of a data-driven culture
- Data-driven, data-informed, and data-inspired product decisions. What are the differences and when should you use each one?
- Using data to understand competitive and market dynamics
- Moving from a startup to an enterprise as a product manager
- The downsides of a data-driven culture
- Using data for strategic decisions
- How to increase the effectiveness of your product analysts
- Why every team member should know the key product metrics
- How to move from marketing to product management?
- Key data skills for product managers: experienced PMs sharing their thoughts
- Product manager skills: evolution of a PM role and its transformation
- What is the difference between growth product manager, marketing manager, and core PM
- How to move from engineering to product management?
- Product growth, reinvented: what growth hacking is (and isn’t)
- Moving from a startup to an enterprise as a product manager
- Product manager interview: real questions plus guide for employers and candidates
- Rolling retention, Day N retention, and the many facets of the retention metric
- Long-term retention—the foundation of sustainable product growth
- Errors in calculating ROI and unit economics. Impact of attribution models and incrementality on the ROI calculation of marketing channels
- Traffic attribution models. Why attribution models need to change along with growth channels, product, business objective and external environment