When an organization says they are data-driven, they typically mean that they base decisions on data. But there can be vast differences with how data is used to make these decisions. Is data only being used to validate straightforward decisions? Are multiple sources of data combined with other factors to determine priorities like the features to be worked on next quarter? Or is an exploration of data being used to spark innovation and determine new strategy? Each situation requires different skills, tools, and ways of working with data to be successful.

This is why the concepts of data-informed and data-inspired are being added to the data-driven discussion; they allow for a more nuanced definition of how data is actually used in an organization. Data-informed and data-inspired decisions consider depending not only on data for clear-cut decisions, but on using data in conjunction with other important influences and to invent something new.

Some may argue that adding the terms data-informed and data-inspired to the data-driven discussion adds complexity and muddles the discussion around data. While that may be true in some cases, really understanding how to correctly use data based on a particular need is critical to creating products that customers love. In the end, the terminology isn’t as important as making sure you’re getting the most out of data. 

For an enhanced understanding of these terms and concepts, we asked product managers skilled in this area the following questions:

  • What is the difference between data-driven, data-informed, and data-inspired decisions?
  • When is each approach best for making a decision?
  • What common mistakes typically occur with these approaches? 

We’d like to give a big thank you to the experts who discussed this topic with us:

→ Test your product management and data skills with this free Growth Skills Assessment Test.

→ Learn data-driven product management in Simulator by GoPractice.

→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.

Q. What is the difference between data-driven, data-informed, and data-inspired decisions?

Data-driven, data-informed, and data-inspired approaches have similarities and differences. And while exact definitions vary by company and team, the distinction generally lies in how the data was instrumental to the decision. A data-driven decision typically answers a distinct question and data is the only factor that the decision is based on. The data needed is available and provides a straightforward answer.

A data-informed decision also uses data, but in a different way. Usually there’s not just one data source, and data isn’t the full picture, just taken into consideration. Other factors like strategy, user interviews, organizational strengths, and resource availability also factor in. Being data-inspired is being inquisitive. There is no one answer; many data sources are explored to generate new directions or inform strategy. It’s useful for identifying new opportunities and fresh ideas for prototypes. 

However, not everyone agrees that all three terms are needed. Most organizations struggle to use data at all in their decision making and many people have only heard of the term data-driven. Breaking it down into three different categories could cause unnecessary confusion, especially since upper management usually says they want data-driven, but what they actually expect is data-informed or data-inspired. Keeping terminology simple may work best for some teams.

Arushi Mishra (Product Leader, Cloud Connectivity and Customer Experiences at Ford Motor Company)

While at first glance all three terms appear to be the same, they are drastically different. Data-driven means you have exactly the data you need to make the next decision. You will get a binary answer to your question by looking at this data, yes or no. For example, are customers using a specific feature on your app? This will help you decide if you need to remove a feature due to low to no usage.

Data-informed means that you are aware of why a product is performing the way it is. This means that you are aware of certain user behaviors and how they affect certain metrics. By being data-informed, you can prioritize and plan your near- and long-term strategies to improve your products. 

Data-inspired is not fixated on finding an answer to a very specific question or on the immediate next step. It is more exploratory in nature. The key is to look at data from different sources, like customer interviews and industry expert feedback, to identify data patterns or commonalities. It provides new ideas that can be taken up for product research and prototyping.

Nicholas Pezarro (Senior Product Manager at LinkedIn)

The main difference between data-driven, informed, and inspired decisions is the level to which the data contributed to the decision. Was the data that you found the primary motivator (data-driven)? Did you take data into consideration when you made your decision (data-informed)? Did some data spark a direction that you decided to pursue (data-inspired)? 

Here’s a sample scenario. You’re a product manager in charge of checkout for an ecommerce company. You look at your dashboards and notice that your cart abandonment rate seems high based on industry standards. 

  • Data-inspired: You find an article about how Amazon reduced cart abandonment by removing the top navigation at checkout. You run an experiment where you strip the top navigation when users checkout and see a reduction in cart abandonment. (Don’t forget to check your guardrail metrics!)
  • Data-informed: You see that users who pay with preset payment profiles (Google Pay, PayPal, etc.) have higher purchase completion rates. You decide to run an experiment where those payment options are shown in priority order for eligible users.
  • Data-driven: You segment the cart abandonment data based on a few categories, size the opportunity for each segment, and launch a survey to users in these groups. Based on the results, responsive billing information and integrating Giropay are your top solutions. You decide to prioritize the responsive design since it can be completed quickly and you can start to get data on how effectively you’ve shifted cart abandonment rate.

Afrina M (Product Manager at Google)

The data-driven decision-making process is the ability to answer a specific question using a large sample set that precisely covers all of your customer segments. You’d need a ton of help from your instrumentation team to gather the right data and data scientists to build the models. 

With a data-informed decision, there isn’t one data source that was created for its purpose. It is more analytical and derives conclusions based on logical reasoning backed up by data. An example is a recommendation engine that shows recommendations based on your shopping or browsing history.

Data-inspired decision-making is using data to validate your gut feeling with generalized data sets. For example, when I decided on product management as a career, I looked up the need in the industry, hiring trends, pay scales, and companies that hire globally, along with taking my passion into consideration.

Julia Winn (Lead PM at Shopify, ex-Google)

Exact definitions may vary by company and team, but in my experience:

  • Data-driven implies data is the primary factor guiding the team’s roadmap and decisions.
  • Data-informed suggests data is one consideration among others, such as longer-term strategy, PR, or team morale, factored into the roadmap.
  • Data-inspired tends to refer to the practice of combing through data, often product usage data, to identify opportunities for improvement in the user experience. While none of my workplaces ever used this exact phrase, the approach itself was widely used everywhere.

However, over the years I’ve heard colleagues use data-driven and data-informed interchangeably. Even in companies where data best practices are clearly defined, day-to-day practices still vary from team to team. It’s important to work with your team to define exactly how data will be used for each project.

Simon Loftus (Senior Product Manager at Barclays)

Senior leaders often say that decisions need to be data-driven; in reality what they want is something between inspired and informed. A truly data-driven decision will have all non data-driven influences removed. A typical example may be a loan pre-approval process which uses data reference points, run through an analytical model to determine credit worthiness.

This works wonderfully for large scale repetitive processes, however business decisions on a larger scale typically need more than just data. This is because data, no matter how good, can’t tell you the whole story. The product manager will often be the guide as to whether a decision should be informed or inspired by data.

Vitor Seabra (Head of International Expansion at Pluga)

Splitting the meaning of “using data for decision-making” into three different concepts is not helpful when we need to guide a company to understand the importance of data. Simplicity is a key factor for success in this field, thus I rather use only one term: data-driven.

When data-informed and data-inspired came to the discussion, the main argument was that the term data-driven excludes important factors such as intuition, creativity, and interpretation. Taking that into consideration, data-informed would complement this lack of human sense on the data-driven approach, adding some quality inputs. On the other hand, data-inspired would bring an inspirational touch when people start thinking about new hypotheses for problem solving using data.

What I want to bring to this discussion is the real goal when creating new buzzwords. Do we really need to add data-informed and data-inspired definitions? We already have data-driven as a solid concept which most companies have heard about. Sometimes I think that these new terms are made up to sell more books or to justify the fact that most of the companies can’t be considered data-driven because they are still led by gut feeling.

Q. When is each approach best for making a decision?

Because each approach is different, the types of decisions they are used for are also different. A data-driven approach is best for straightforward decisions, like deciding between two designs with an A/B test. If the data is there, the time to make this decision is relatively small, which can be advantageous.

A data-informed approach works well when you need to use multiple data sources and take additional factors into account. Things like prioritizing features and creating a roadmap are perfect for a data-informed approach because you need to rely on multiple sources of data and also need to consider your organization’s strategy, priorities, skills, and resources. 

A data-inspired approach is useful for identifying opportunities, informing strategy, and generating innovation. It’s a great technique for the initial phase of a project.

Here are examples of when each approach works well:

Data-drivenData-informedData-inspired
A/B tests
New feature performance		Roadmap
Feature prioritization		Strategy
New opportunities

Afrina M (Product Manager at Google)

During a product life cycle, I’d use data-driven decision-making when I need to validate two designs. We’d run a UX study with our targeted customers and make a decision based on the results of the study. 

Data-informed decision-making is when I have to prioritize features. For example, based on the requests from the field, issues reported by customers, and UX feedback, we are prioritizing this particular request. 

I would use data-inspired decision-making when I have to come up with a strategy for a launch. I would look into what’s working well for other product managers, how that might fit into my situation, and what’s doable.

Julia Winn (Lead PM at Shopify, ex-Google)

I only recommend using the data-driven approach to make decisions when all other factors, for example usability and design consistency, are equal. In practice, this is relatively rare and tends to be limited to narrowly focused A/B tests. For example, let’s say you are deciding between variants A and B after running an experiment to evaluate the best text for the signup button. If variant A performed better and the designs are the same otherwise, then choosing variant A is a reasonable choice.

I always recommend a data-informed approach when setting a roadmap and prioritization. As an example, an engineer has some spare cycles and asks which of two bugs they should work on. Bug A might be more severe, but Bug B has much wider reach. You might choose to focus on either of these bugs given this information, but at least you’ll be able to factor in both the severity of the issue AND the number of impacted users when making your decision.

If your team regularly collects usage data for the product, being data-inspired can be a great source for identifying opportunities. Some common starting points include:

  • Funnel drop-off. At what stage of the funnel is drop-off highest? Are there any major differences in who is dropping off where?
  • Search terms. If your site enables search, what do visitors try to search for? How many searches result in zero results?
  • Dead ends. If you have an error page, how are users reaching it? How often does this happen?

Arushi Mishra (Product Leader, Cloud Connectivity and Customer Experiences at Ford Motor Company)

Now that we have talked about the differences, let’s look at when to use each one.

Data-driven, use it when:

  • There is a critical business question to be answered.
  • The end goal is to ensure that changes to the product will not have an adverse impact.

    Example situations where you can use a data-driven approach:
    • What design performs better, design A or design B?
    • How will a specific feature impact the users?
    • How is feature X performing since it was rolled out last month?

Data-Informed, use it when:

  • Product needs to be optimized for improved user efficiency.
  • You need information or data to make an informed decision about prioritizing the backlog.

    Example situations where you can use a data-informed approach:
    • What can we do in the future to improve the lower calls to action on a button?
    • Given that our website has low customer acquisition, how do we improve this in future?

Data-Inspired, use it when:

  • A new innovative direction needs to be discovered for a product.
  • During the design thinking (initial) phases of developing a product idea. Think about how to use data from various sources to ease customer’s pain-points.

    Example situations where you can use a data-inspired approach:
    • A new product needs to be identified within a business or product line.
    • Determining common data trends in the market to leverage them for a competitive advantage.

Sriram Suresh (Product Manager at Capital One)

You can think of being data-driven, data-informed, and data-inspired as a spectrum. If the decision is more complex, i.e. it cannot be answered with a yes or no, a data-informed or data-inspired approach works best. But these approaches take more time because you need to analyze multiple data sources and use a more holistic approach to make product decisions. 

The following matrix shows when each approach works well.

Decision complexity# of Data SourcesTime to make decisionRecommended Approach
LowSingle – FewLowData-driven
MediumFew – ManyMediumData-informed
HighManyHighData-inspired
  • Decision complexity is defined as how much thought or effort it is going to take you to make a decision. For example, let’s say your decision is whether to change a button color on your website. This would be considered low complexity. Whereas a decision to roll out a new payment system for your product would be considered high complexity, would require much more thought and effort, and may not have an obvious answer.
  • # of data sources is defined as the number of data sources you can use to make your decision. If you only have a few data sources you might not have the opportunity to take a data-inspired approach as the data richness is just not available.
  • Time to make a decision is defined as how much time you and your team have in making your product decision. 

Vitor Seabra (Head of International Expansion at Pluga)

My point of view is different. I consider the term data-driven to cover all of the three buzzword concepts since we are humans, not robots. So I don’t get stuck on which term is more suitable for each situation.

Q. What common mistakes typically occur with these approaches? 

A basic mistake is simply using an incorrect approach for the situation. A clear-cut decision that can be made just using numbers doesn’t require an elaborate data exploration project. Similarly, an organization’s strategy cannot merely be built on numbers from one data source. More exploration, validation, and data sources are needed. 

Additionally, blunders occur when the data is either inaccurate or there is misinterpretation of the data. If the data is wrong, there’s a good chance your decision will be wrong as well. And if data is misinterpreted, for example if you decide your low conversion numbers are because your sign-up page is difficult to use but it’s really an issue with your marketing, you won’t fix the problem. Issues can also be hidden if anomalies are lost in averaged data. Getting overwhelmed with too much data and delaying decisions, or inversely making decisions too fast based on a limited amount of data, are also common mistakes.

To summarize, typical mistakes seen with data-driven, data-informed, and data inspired decisions are:

  • Using the wrong approach
  • Inaccurate data
  • Misinterpretation of data
  • Overlooking problems hidden by averaged data
  • Analysis paralysis
  • Making decisions too quickly

Simon Loftus (Senior Product Manager at Barclays)

I see two common mistakes when it comes to data-driven decisions:

  1. Inaccurate data. Decisions made off of this information, whether fully driven or even simply inspired, can cause huge mistakes. Often product managers will rely too heavily on a data team who aren’t close to the end product or customer and therefore are less likely to spot irregularities. Or teams are often stretched in multiple directions, so the likelihood of human error is more likely than not.
  1. Misinterpretation of data. Data can often be vast in its size, particularly in a field like transaction banking where multiple transactions per day by multiple customers quickly scales. However, in addition to the volume of data, the complexity of data is often very high. This increases the chance of misinterpretation, particularly where product managers are not experienced or technically capable to interrogate directly, meaning they lean on data teams.

It’s too easy at times to overlook problems because they’re washed out in averages. Product managers need to be particularly careful when receiving high level analysis for this reason, as a proposition may look sensible on mass however fundamental flaws may be masked.

Arushi Mishra (Product Leader, Cloud Connectivity and Customer Experiences at Ford Motor Company)

It is easy to misunderstand the application of these data analysis techniques and use one over the other.

  • If it’s an answer to a specific business scenario, use a data-driven technique.
  • If it’s a product improvement strategy, use a data-informed technique.
  • If it’s a new product idea or product discovery phase, use a data-inspired technique.

However, each of these approaches has a limitation.

  • A data-driven approach will never have the answer to a long-term strategy. It talks about “immediate next.”
  • The data-informed approach will give you a long-term strategy and data points to refine your backlog. It will tell you about past failures but will never give you a new inspiring idea.
  • The data-inspired approach will lead you to a new idea but needs a cautious analysis as the data could be spurious or random. Having concrete deductions from this kind of data can be hard.

For example, let’s say you rolled out a feature where users can add a nickname to their devices in an app, but for some reason, the data shows you that people are not using the feature much. In this case, a data-driven technique only tells you that a feature is not performing well. In order to make an informed decision about the cause, you need to look at other aspects of the app, get the data, and make changes to the product strategy. This is why business needs are critical to determine the right approach.

Sriram Suresh (Product Manager at Capital One)

One of the biggest pitfalls I have noticed, especially with the data-inspired approach, is analysis paralysis. This is common with product managers at very large companies, where you have access to vast amounts of data. This is both good and bad as you can make stronger data-inspired decisions but you end up taking time that may delay product deployment. One of the differentiators of an effective product manager is knowing when to recognize and stop analysis paralysis. 

Another common pitfall is using the data-driven approach on the other end of the spectrum of analysis paralysis. I would call this “too quick to make a decision.” This is where you don’t take the time to think through the decision complexity and don’t consider the multiple data sources that you have available. This can lead to a poor experience for your customers if you did not take the right approach to deliver a strong product. 

Julia Winn (Lead PM at Shopify, ex-Google)

The most common mistakes I’ve seen involve experiments. People often default to one of two mindsets:

  1. We know this version is better, so why bother with an experiment? (data agnostic)

    OR
  2. Everything should always be an experiment (rigidly data-driven).

We know this version is better, so why bother with an experiment?

Sometimes, I do think this is the right approach! Especially if you are removing something known to be misleading or confusing users. However, if the change impacts a high traffic surface or a critical step in the user funnel, small changes can sometimes make a big difference, and nobody’s intuition is perfect 100% of the time. I once saw sign ups drop 40% after launching a redesign everyone was certain improved the experience. No one could understand why this happened, the old site wasn’t even mobile responsive, and some images were blurry. But the new design performed terribly. Ultimately, the new site was rolled back, but not until weeks passed. Had we rolled the new site as an experiment, not only would we have seen the dip in performance sooner, we would have known it wasn’t caused by other factors like seasonality. We switched back to the old site, and sign ups went back up, but we could never figure out why the new site performed so poorly.

Everything should always be an experiment

While I love enthusiasm for experiments, insisting everything go through an experiment can be problematic. Setting up and evaluating the results of experiments takes time. It’s even worse to run a poorly designed experiment that returns misleading results. Some cases to consider skipping an experiment include:

  • The sample size of the surface or user group is too low. An example of this would be making a change to the settings page of an app or the footer of a website. Maybe you have lots of daily active users, but how many actually visit these surfaces? 
  • Large structural or taxonomy changes that may take months or years to show results. One example of this would be Netflix changing its rating system, most recently from 1-5 stars to thumbs up/thumbs down. Maintaining different recommendation models simultaneously would require extra resources and staffing and results would not come for a long time. Furthermore, these decisions to change the model were already highly data-informed and influenced by user research. Therefore, it was reasonable to just launch the change without running a months-long, messy experiment.

We’d like to thank Stephanie Walter for incredible help in creating this article.