Category posts
Data and analytics

As product managers, we regularly use percentages in different ways to measure different metrics. But despite their intuitive simplicity, percentages have different nuances that can make them confusing or misleading if used improperly.

In this article, we’ll discuss percentages, percentage points, and percentiles, along with their uses in product management along with best practices and pitfalls to avoid.

Data-driven decision-making is essential for success in product management. And one of the important parts of any kind of data-driven work is measuring and understanding central tendencies in data samples. 

Central tendencies refer to the typical or average values of a set of data points, and they can provide insights into the overall performance of your product. Measuring central tendencies is becoming increasingly important in any kind of decision-making that involves data. And product managers have to make decisions based on data on a daily basis.

There are several ways to measure central tendencies, but the most commonly used methods are arithmetic mean and median. In this article, we will explore these two measures, compare them, and see how they can be used in product work.

Data drives product management. Product managers rely on data for everything from creating an easy buying experience to determining if a product should even be developed. And while data is used to inform all aspects of managing a product, are there different considerations for using data for long-term strategy vs. daily decisions? 

It turns out that the answer is yes. While there are many similarities, like needing to ensure success is clearly defined, for strategic decisions more context, expertise, and qualitative data is required. Data used for strategic decisions tends to be higher quality, take longer to collect, and be more expensive than data used for everyday decisions. Data mistakes, such as bias or incorrect benchmarking, are also more costly to the organization when they happen in the context of creating a strategy.

A quantitative data-driven culture is generally seen as very positive. It’s great for understanding performance and tweaking strategy. Quantitative data is useful for reducing risk and can be used for decisions that need a high degree of certainty. These situations are usually high enough of a priority that adequate time is allotted to use data to verify various experiments before the decision is made. 

But there can be very real downsides. Data alone isn’t particularly useful for big transformations or concepts that don’t exist in the market. Henry Ford, founder of Ford Motor Company, famously said, “If I had asked my customers what they wanted they would have said a faster horse.” Ford’s customers actually wanted a faster way to travel, but lacked the language and vision to describe an automobile. Ford had to make the leap from “faster horse” to “car.” Data cannot do that.

Product managers or analysts will often say something like: “My analysis of the data shows that users who do action X are more likely to buy the premium version or become successful.” Based on this insight, they decide to invest time and effort into making more users do X.

But this is a classic example of assuming causation where there is only correlation. Maybe one thing really does cause the other—or maybe it’s just another case of two metrics happening to grow at the same time.

We have all heard the time-old caution: “Correlation does not imply causation.” It sounds obvious, almost too obvious. Yet time and again, even very experienced people, whether by accident or design, treat them as the same thing.

Here we will look at why it’s so easy to gloss over the difference between correlation and causation, how to prove causality, and why it’s important to keep all this in mind when working on a product.

Product managers know that understanding their market and competition inside and out is vital to the success of their products. Comprehensive market knowledge tells you what problems your customers are trying to solve, what they want, and ultimately what new features or products to build. Knowing the competitive landscape helps to set your business apart, allows you to create a strategy to deal with new competitive developments, and helps to arm your sales team to win against the competition. 

When an organization says they are data-driven, they typically mean that they base decisions on data. But there can be vast differences with how data is used to make these decisions. Is data only being used to validate straightforward decisions? Are multiple sources of data combined with other factors to determine priorities like the features to be worked on next quarter? Or is an exploration of data being used to spark innovation and determine new strategy? Each situation requires different skills, tools, and ways of working with data to be successful.

This is why the concepts of data-informed and data-inspired are being added to the data-driven discussion; they allow for a more nuanced definition of how data is actually used in an organization. Data-informed and data-inspired decisions consider depending not only on data for clear-cut decisions, but on using data in conjunction with other important influences and to invent something new.

Some may argue that adding the terms data-informed and data-inspired to the data-driven discussion adds complexity and muddles the discussion around data. While that may be true in some cases, really understanding how to correctly use data based on a particular need is critical to creating products that customers love. In the end, the terminology isn’t as important as making sure you’re getting the most out of data. 

Product managers use data at the heart of their decision making. There are two types of data that they rely on: quantitative and qualitative. Quantitative data refers to numerical data that can be measured; examples include number of clicks, number of users, and monthly recurring revenue. Qualitative data cannot be counted, but instead describes traits or features such as ease of use, user likes and dislikes, and motivations behind user actions. 

Working with data helps companies across the board to unlock their potential and become more productive and better at making decisions. However, making people in the team and company rely on data involves a lot of work. Product managers must often set a strategy, reinvent processes, and change organizational behavior. 

Data is an essential part of the work of every product manager. It helps to form and validate hypotheses, provide more insights about user behavior, and make better decisions and track product changes.

But the misuse of data can be harmful. One important example is selecting only data that confirms a particular hypothesis and ignores relevant contradictory evidence.