Category posts
Data and analytics

Data is essential for every product. These essays will help you improve your skills in analytics and avoid data mistakes.
Author:
Editorial
Data cherry-picking to support your hypothesis. What is it? Why is it bad? 
Data cherry-picking to support your hypothesis. What is it? Why is it bad? 

Data is an essential part of the work of every product manager. It helps to form and validate hypotheses, provide more insights about user behavior, and make better decisions and track product changes.

But the misuse of data can be harmful. One important example is selecting only data that confirms a particular hypothesis and ignores relevant contradictory evidence.

(more…)
Author:
Editorial
Data mistakes to know and avoid as a product manager
Data mistakes to know and avoid as a product manager

Data is a key part of product management. We gain our intuition by looking at data. We come up with hypotheses based on our observations of data. We test and validate these hypotheses using data. And we make key product decisions and monitor and track changes in data. In a nutshell, data helps us go from ideas to facts to decisions.

However, working with data is also wrought with pitfalls that every product manager should avoid. Data can sometimes be misleading or incomplete. It might not tell the whole truth and lead you in the wrong direction. And it might amplify your own erroneous assumptions.

(more…)
Author:
Oleg Yakubenkov
Mistakes in A/B testing: guide to failing the right way

Failing fast and often will help you learn from your mistakes sooner rather than later. This is an advice you hear often from successful product managers. But what you hear less often is that not every failure is a successful learning experience.

A product’s success is largely dependent on coming up with a hypothesis and designing the right tests. Without those elements, you might draw the wrong conclusions and steer your project in the wrong direction.

In his guest post for the GoPractice blog, Ethan Garr, VP of product at TelTech.co, shares some hard-earned experience in product testing. Through concrete case studies, Ethan shows us how to avoid key pitfalls when designing tests for hypotheses.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

Learn data-driven product management in Simulator by GoPractice.

Learn growth and realize the maximum potential of your product in Product Growth Simulator.

Join our discussion on LinkedIn. New topics to talk about every week.

Mistakes in A/B testing: guide to failing the right way
(more…)
Author:
Oleg Yakubenkov
Peeking problem – the fatal mistake in A/B testing and experimentation
Peeking problem – the fatal mistake in A/B testing and experimentation

You can make many mistakes while designing, running, and analyzing A/B tests, but one of them is outstandingly tricky. Called the “peeking problem,” this mistake is a side effect of checking the results and taking action before the A/B test is over.

An interesting thing about the peeking problem is that even masters of A/B testing (those who have learned to check if the observed difference is statistically significant or not) still make this mistake.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

Learn data-driven product management in Simulator by GoPractice.

Learn growth and realize the maximum potential of your product in Product Growth Simulator.

Join our discussion on LinkedIn. New topics to talk about every week.

(more…)
Author:
Oleg Yakubenkov
Why your A/B tests take longer than they should

When conducting experiments, teams usually include all the active users in their tests, or sometimes they tend to add all the new users who join the app during this test. So when calculating the metrics for different test groups, all the data from the moment the A/B test kicks off is taken into account.

Today I’ll talk about how you can reduce the time required to get the signal on the change you are testing in a product. You can do that by changing the process of adding users to the A/B test, and in this essay I will show you how you can do it.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

Learn data-driven product management in Simulator by GoPractice.

Learn growth and realize the maximum potential of your product in Product Growth Simulator.

Join our discussion on LinkedIn. New topics to talk about every week.

Why your A/B tests take longer than they should
(more…)
Author:
Oleg Yakubenkov
Experiments where you make your product worse – the most underrated product manager tool

Let’s start with a practical task.

Say a company’s management wants to allocate significant resources to the development of infrastructure that would increase their app’s speed. The hypothesis is that increasing the speed of the app will have a positive effect on the user experience and the key metrics.

Think of an experiment (an A/B test) to validate this hypothesis.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

Learn data-driven product management in Simulator by GoPractice.

Learn growth and realize the maximum potential of your product in Product Growth Simulator.

Join our discussion on LinkedIn. New topics to talk about every week.

Experiments where you make your product worse – the most underrated product manager tool
(more…)
Author:
Oleg Yakubenkov
Setting up event tracking for a mobile app

Creating the app of your dreams can be a daunting challenge. But an even greater challenge is making sure your users find and experience its value. To do this, you need to have visibility on how users navigate through your app and interact with its features.

This is where analytics and event-tracking platforms come into play.

Analytics help you answer important questions about your users such as what do they like about your app, where are they struggling, which features need improvement, which features are hindering the user experience, and many more.

The foundation of analytics is data collection. Every analytics platform comes with a software development kit (SDK) that you can integrate into your app to send events. Developers add code that calls special application programming interfaces (API) in the locations where an event must be fired (for example, when users launch the app for the first time or when they send a message). If event logging is set up poorly, you will be blinded when working on your product, or you will work with erroneous data, which in my opinion is even worse than not having any data.

In this article I will be talking about the following:

  • Common mistakes in setting up analytics for mobile apps
  • The right approach to integrating analytics in mobile apps
  • Some hacks that will allow you to use data more efficiently

→ Test your product management and data skills with this free Growth Skills Assessment Test.

Learn data-driven product management in Simulator by GoPractice.

Learn growth and realize the maximum potential of your product in Product Growth Simulator.

Join our discussion on LinkedIn. New topics to talk about every week.

Setting up event tracking for a mobile app
(more…)
Author:
Oleg Yakubenkov
How game developers can leverage the hidden potential of their data

The general perception is that data analytics and data-driven product management is more suited for business applications, social media apps, and communications platforms. But the reality is, any kind of product can benefit from a data-driven perspective.

One of the domains where the value of data is often underestimated is games.

In my experience, most teams working on mobile games don’t fully use the potential of data. They tend to track topline metrics, measure effectiveness of paid marketing campaigns, analyze the impact of product changes all while running meticulous AB tests. This may sound like enough, but it really isn’t. Not if your goal is to climb onto the top of the grossing charts and stay there.

There are many more ways how data can increase your chances of building and operating a successful mobile game. The key is to stop thinking of data as a way to look back at what you have done, but instead start using data as a tool that can help you make decisions, decrease uncertainty and remove main product risks as early as possible.

In this post, I will walk you through a few examples of how data can drive key product decisions at different stages of product development cycle. But first, let me tell you a story…

→ Test your product management and data skills with this free Growth Skills Assessment Test.

Learn data-driven product management in Simulator by GoPractice.

Learn growth and realize the maximum potential of your product in Product Growth Simulator.

Join our discussion on LinkedIn. New topics to talk about every week.

How game developers can leverage the hidden potential of their data
(more…)
Author:
Oleg Yakubenkov
Looking for spikes. How to increase the effectiveness of your dashboard

Consider this:

A product manager asks an analyst: “Please make a dashboard where we can see Day 1, Day 3 and Day 7 retention rates in dynamics.”

“Are you sure?” the analyst asks. “These charts will be quite noisy. Just look how much the metrics alter from day to day. Maybe it’s a better idea to monitor the weekly retention rates instead. In this case, any random fluctuations will be smoothed out. ”

They called it a deal.

Now a new dot appears on the dashboard once a week. This dot has “Everything is fine, nothing has changed” written all over it. But sometimes, the storms of everyday life are hidden behind this apparent calm: days of ups and downs, victories and defeats that happen during weekdays and weekends.

But no one on the product team finds out about them because everyone is looking at weekly metrics. Consequently, they’re missing both the random and meaningful fluctuations.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

Learn data-driven product management in Simulator by GoPractice.

Learn growth and realize the maximum potential of your product in Product Growth Simulator.

Join our discussion on LinkedIn. New topics to talk about every week.

Looking for spikes. How to increase the effectiveness of your dashboard
(more…)