Category posts
Series

When I worked at Facebook, the Workplace analytics team had a cool tradition: The team’s weekly meetings always started with a small data quiz.

The winner of the previous week’s competition would prepare a question about the product’s key metrics. For example, “what was last month’s MAU?” or “how many new users joined last week?” or “what proportion of the new companies reach 10 users?” or “what was last month’s revenue?” The question had one requirement: Its answer had to be found on the team’s dashboard.

The participants were to write down the answer without getting help from computers, which meant we could only use our memory to do so. The person whose answer was closest to the correct number got +1 point in the chart, and the person who was the farthest lost 1 point. Every six months, a winner was chosen and the game started again.

I participated in five seasons and won three of them. In one of the final rounds, I was tied with another analyst. The team arranged the final round, where we had to answer five questions in a blitz quiz. I managed to score the winning point and won the mug that you see in the photo below.

I told this story not because I wanted to brag about winning the quiz (well, this too, to be honest). In almost every quiz, the respondents’ guesses on metrics were distributed across a wide range, which I found surprising.

Why? Well, first of all, it was the analysts who played the game. They were the people who worked with data most of their time and should have been good at navigating it. Second, these analysts were working at Facebook, a company that has a very advanced and strong data culture. At Facebook, each team has clear goals, dashboards are available to all the company’s employees, and all meetings start with progress updates on key metrics. How could these people be so wrong in answering questions about the product they were working on?

If you decide to play this game with your company’s employees, you will most likely be as surprised as I was. It will turn out that most people have very vague ideas about the key metrics of your product and business. And some people will have no idea at all.

In this essay, we will discuss why it is important for team members to remember at least approximate values ​​of the key product metrics, why this usually doesn’t happen, and how to get there.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

The most common metrics gaming companies focus on are Day-1, Day-7, and Day-30 retention rate. While these metrics are of great help early in the journey, it’s long-term retention which is key to lasting success and a seat in the top-grossing charts. This post makes a case for long term-retention and why your focus should be first and foremost there.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

After my recent essay “Slack vs Teams vs Workplace: The intriguing dynamics of the work messenger market,” I didn’t plan to revisit the competition between Slack and Microsoft Teams just yet. Despite the rapid development of the work communication market, it is still a B2B market that is changing relatively slowly.

However, something extraordinary has happened: We are in the midst of one of the greatest “experiments” of our time, and a great part of the world has switched to remote work due to the COVID-19 pandemic. The outbreak has greatly sped up the development of remote work tools. This situation has propelled us several years ahead in time, much faster than it was destined to happen, allowing us to look into the future of the market today.

Another reason to take a closer look at Slack vs Teams is that recently, a lot of new data has surfaced about Slack. Even before the coronavirus outbreak, several large companies decided to adopt Slack as the communication tool for all their employees, including IBM with its 350,000-strong workforce and Uber, which has more than 38,000 employees. The number of paid Slack customers has grown by more than 7,000 over the past one and a half months, surpassing the growth in the entire previous quarter. This week, Slack CEO Stewart Butterfield shared the sequence of the recent events in a series of posts on Twitter, all of them conveying a clear message: Slack is growing very fast.

All of the above made me doubt the assumptions I’ve made in my past essays, in which I didn’t put too much faith in Slack’s chances of winning the race against Microsoft Teams. In light of the new data, I decided to take another look at the market and figure out what was going on.

And here’s what I realized: Slack is a great business, but it still stands no chance against Microsoft Teams in dominating the messenger market. The recent weeks have further confirmed this assumption.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

Watching new and rapidly changing markets can teach you many things. The work communication market led by companies such as Slack, Microsoft Teams and Workplace by Facebook is something I have been following for a long time.

Last year, before Slack went public, I did an analytical review of the data disclosed in Slack’s S-1 filing. At the end of that review, I shared my opinion that Slack experienced problems in the enterprise segment: the competition from Microsoft Teams and Workplace by Facebook for this market segment threatened Slack’s long-term growth prospects and its $20+ billion valuation.

A lot of things have happened in the eight months that have passed since I published that essay. A lot of new data has surfaced, with one of the biggest market intrigues fading away and a new one appearing. The leading characters once again reminded us of a number of fundamental rules the market plays by. And this is exactly what I am going to talk about in this essay.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

When conducting experiments, teams usually include all the active users in their tests, or sometimes they tend to add all the new users who join the app during this test. So when calculating the metrics for different test groups, all the data from the moment the A/B test kicks off is taken into account.

Today I’ll talk about how you can reduce the time required to get the signal on the change you are testing in a product. You can do that by changing the process of adding users to the A/B test, and in this essay I will show you how you can do it.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

Let’s start with a practical task.

Say a company’s management wants to allocate significant resources to the development of infrastructure that would increase their app’s speed. The hypothesis is that increasing the speed of the app will have a positive effect on the user experience and the key metrics.

Think of an experiment (an A/B test) to validate this hypothesis.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

It is common to evaluate your product performance and the impact of changes you make by using engagement metrics with active audience in the denominator. Examples of these metrics include the time spent per active user, the occurrence of certain actions (messages sent, levels played, chapters read, etc) per active user, or ratio metrics (what percent of active users perform a specific action) .

In most situations, these engagement metrics will be helpful. But in some cases, they can be misleading. And it is important to understand why and when this can happen, and what you can do about it.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

pic from http://mediainjection.com

Product/market fit is an important concept when working on a new product. All entrepreneurs and product managers are committed to it. But if you ask what the term means, very few will be able to give a clear answer. Even fewer will have an understanding of how we can measure product/market fit using metrics.

Without a clear definition, even the most useful concepts will be of little help when making decisions. In this post, we will discuss some of the most common product/market fit definitions and their advantages and disadvantages, as well as tell you about PMFsurvey.com (Product / Market fit survey by Sean Ellis) developed in collaboration with GoPractice, which is designed to give you an objective metric of how close are you to Product / Market fit.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

Finding insights and answers to questions in data is a key skill in product analytics. And developing this skill is the area where analysts usually see their growth potential.

Talking from experience, I strongly recommend paying attention to another aspect of analytical work: communication skills. Key here is not only finding insights, but also turning them into projects and making sure they convert into real value for users. 

Getting to this point requires building relationships with the team, participating in key discussions, gaining credibility, and learning to present information in an effective way.

This article provides a series of recommendations for product analysts. However, it will be equally useful for product managers and executives who want to maximize the impact of analysts working in their teams.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

Retention rate is one of the fundamental metrics in product management. We all use it regularly, yet few of us know that there are many different ways to calculate retention rate. And it is very important to know which one to use when you’re making decisions based on retention data.

Let me start with a story. When I worked at Zeptolab (popular game development company, creator of Cut the Rope, King of Thieves, CATS) once we got an email from a gamedev studio that wanted us to publish their game. We were getting many similar emails, but that one got our attention. We were impressed by the metrics of the game, which had just recently soft launched. According to the developers, Day 1 retention rate of the game was over 55%, and Day 7 retention rate was over 25%.

However, when we started playing and testing the game, something felt wrong. The gameplay was not engaging enough to justify >55% Day 1 retention rate. And the meta game design was not good enough to users in the longer term.

Further investigation revealed that what this game development company called retention was actually “rolling retention.” Classic Day N retention of the game happened to be unimpressive.

This is just one example of how retention metrics can misguide you. There are many nuances in how you can calculate it.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.