Category posts
Data and analytics

Failing fast and often will help you learn from your mistakes sooner rather than later. This is an advice you hear often from successful product managers. But what you hear less often is that not every failure is a successful learning experience.

A product’s success is largely dependent on coming up with a hypothesis and designing the right tests. Without those elements, you might draw the wrong conclusions and steer your project in the wrong direction.

In his guest post for the GoPractice blog, Ethan Garr, VP of product at TelTech.co, shares some hard-earned experience in product testing. Through concrete case studies, Ethan shows us how to avoid key pitfalls when designing tests for hypotheses.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

You can make many mistakes while designing, running, and analyzing A/B tests, but one of them is outstandingly tricky. Called the “peeking problem,” this mistake is a side effect of checking the results and taking action before the A/B test is over.

An interesting thing about the peeking problem is that even masters of A/B testing (those who have learned to check if the observed difference is statistically significant or not) still make this mistake.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

When conducting experiments, teams usually include all the active users in their tests, or sometimes they tend to add all the new users who join the app during this test. So when calculating the metrics for different test groups, all the data from the moment the A/B test kicks off is taken into account.

Today I’ll talk about how you can reduce the time required to get the signal on the change you are testing in a product. You can do that by changing the process of adding users to the A/B test, and in this essay I will show you how you can do it.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

Let’s start with a practical task.

Say a company’s management wants to allocate significant resources to the development of infrastructure that would increase their app’s speed. The hypothesis is that increasing the speed of the app will have a positive effect on the user experience and the key metrics.

Think of an experiment (an A/B test) to validate this hypothesis.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

Creating the app of your dreams can be a daunting challenge. But an even greater challenge is making sure your users find and experience its value. To do this, you need to have visibility on how users navigate through your app and interact with its features.

This is where analytics and event-tracking platforms come into play.

Analytics help you answer important questions about your users such as what do they like about your app, where are they struggling, which features need improvement, which features are hindering the user experience, and many more.

The foundation of analytics is data collection. Every analytics platform comes with a software development kit (SDK) that you can integrate into your app to send events. Developers add code that calls special application programming interfaces (API) in the locations where an event must be fired (for example, when users launch the app for the first time or when they send a message). If event logging is set up poorly, you will be blinded when working on your product, or you will work with erroneous data, which in my opinion is even worse than not having any data.

In this article I will be talking about the following:

• Common mistakes in setting up analytics for mobile apps
• The right approach to integrating analytics in mobile apps
• Some hacks that will allow you to use data more efficiently

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

The general perception is that data analytics and data-driven product management is more suited for business applications, social media apps, and communications platforms. But the reality is, any kind of product can benefit from a data-driven perspective.

One of the domains where the value of data is often underestimated is games.

In my experience, most teams working on mobile games don’t fully use the potential of data. They tend to track topline metrics, measure effectiveness of paid marketing campaigns, analyze the impact of product changes all while running meticulous AB tests. This may sound like enough, but it really isn’t. Not if your goal is to climb onto the top of the grossing charts and stay there.

There are many more ways how data can increase your chances of building and operating a successful mobile game. The key is to stop thinking of data as a way to look back at what you have done, but instead start using data as a tool that can help you make decisions, decrease uncertainty and remove main product risks as early as possible.

In this post, I will walk you through a few examples of how data can drive key product decisions at different stages of product development cycle. But first, let me tell you a story…

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.

Consider this:

A product manager asks an analyst: “Please make a dashboard where we can see Day 1, Day 3 and Day 7 retention rates in dynamics.”

“Are you sure?” the analyst asks. “These charts will be quite noisy. Just look how much the metrics alter from day to day. Maybe it’s a better idea to monitor the weekly retention rates instead. In this case, any random fluctuations will be smoothed out. ”

They called it a deal.

Now a new dot appears on the dashboard once a week. This dot has “Everything is fine, nothing has changed” written all over it. But sometimes, the storms of everyday life are hidden behind this apparent calm: days of ups and downs, victories and defeats that happen during weekdays and weekends.

But no one on the product team finds out about them because everyone is looking at weekly metrics. Consequently, they’re missing both the random and meaningful fluctuations.

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

→ Join our discussion on LinkedIn. New topics to talk about every week.