While most product managers know that using data in their daily activities can have tremendous benefits, sometimes they find themselves in an environment where doing so is not easy. Perhaps they don’t have access to the data they need, the data is unreliable, or there is no support in place to incorporate data into their processes. These product managers are not in a data-driven culture.
A data-driven culture is when an organization embraces data to make decisions at all levels. The organization has the infrastructure and talent needed to collect, transform, and analyze data, along with reliable and trustworthy data sources. There is an importance on using data to support hypotheses and resolutions. Data-driven cultures embrace data and bake it into their everyday processes.
But a data-driven culture doesn’t just happen on its own. It needs both top-down and bottom-up support in the organization. Upper management must make the decision to invest in data and infrastructure while the teams must believe that using data in their daily jobs is beneficial. And while data enthusiasts in an organization can plant the seed, the entire organization’s support is needed for a data-driven culture to blossom.
For a deeper understanding of this topic, we interviewed a group of product managers with knowledge of data-driven cultures and asked them the following questions:
- Does a data-driven approach have any advantages? Disadvantages?
- What makes a company data-driven? How can a data-driven culture be developed from scratch?
- Which approach is better for transforming a company to be data-driven: top-down (from upper management) or bottom-up (from data enthusiasts)?
Thanks to our experts for their outstanding insight:
- Igor Akimov (Head of AI Solutions at Wrike)
- Nick Allen (Group Product Manager at Booking.com)
- Elaine Chao (Senior Product Manager at Adobe – Creative Cloud)
- Karl Lillrud (Professional Speaker, Business advisor and Mentor)
- Kumar Samanvaya Misra (Product Manager, Data Monetization at BASF)
- Laura Onu (Product Manager at Meta – Reality Labs)
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
Q. Does a data-driven approach have any advantages? Disadvantages?
A data-driven approach can have numerous advantages for your organization. Many of our experts credit being data-driven with long term success of their companies, better decision-making, and less risk. This approach can help with deeper customer understanding and allow you to gauge the health of your business. It can even ensure that resources aren’t wasted on projects or features that won’t achieve an organization’s goals.
While most product managers will say that the benefits of a data-driven approach outweigh the drawbacks, it’s important to be aware of the disadvantages. Being overly reliant on data can inject increased time into your processes and delay decisions. There’s also the downside that while the past is often a good indicator of future behavior, it can never fully predict what will happen. Not every aspect has enough data to be analyzed, and many intangible, yet important, elements of your business cannot be measured.
Highlights from our experts of the advantages and disadvantages of a data-driven approach are summarized in the table below.
| Advantages | Disadvantages |
| — Increases confidence and decreases risk in decisions — Enables understanding of user behavior, engagement, and health of the business — Focuses resources on proven priorities — Brings forward trends and sizes opportunities | — Can slow decision-making — Past information cannot predict the future — Not everything can be measured, e.g. customer happiness, reputation, or company leadership — There may not be enough data to make all decisions — When misused, it can hamper innovation and limit product growth |
Igor Akimov (Head of AI Solutions at Wrike)
Instead of internal feelings, expert opinions, or the wishes of the most vocal customers, which lead to wasting resources, we are coming to a world where quantified decision making is the cornerstone of any process, and that is awesome! We get better output and more value to the customer instead of just releasing some big feature.
But this approach also has disadvantages:
- Not all things can be measured. I really like Tony Shea’s quote, “Just because you can’t measure the ROI of something doesn’t mean you shouldn’t do it. What’s the ROI of hugging your mom?” Things like customer happiness, reputation, or leadership are hard to measure.
- There may not always be enough data to make decisions. There may be a long sales cycle, little traffic, little time, poor data quality, or human bias, which can lead to bad decisions.
- Focusing only on metrics will never make a radical shift in the company strategy, so balance is important.
In general, the ubiquitous data-driven approach is more suited to large audience B2C projects, when you can run thousands of A/B-tests. But continuous discovery and evaluating your decisions to evolve your strategy should be the heart of any company.
Nick Allen (Group Product Manager at Booking.com)
A disciplined approach to decision making using data is a key driver of long term success. This discipline can be lacking in a lot of earlier stage startups, especially with first time founders or if there is a lack of experience on the senior team. A data-driven approach involves maturity that develops over time.
With a data-driven approach, it can be necessary to make tradeoffs between confidence and speed of decision making. The reward of making a fast decision can sometimes be very great, giving first mover advantage. However there is also an increased risk of making a wrong choice.
Laura Onu (Product Manager at Meta – Reality Labs)
A data-driven culture is having your team agree on where you’re headed and why by using a reality-informed framing. On the opposite spectrum is the activity of gathering a conglomerate of data points and using them to derive the most risk-averse product strategy. Unfortunately this approach risks hampering innovation and limits your product growth.
We should think about data as a way to understand your customers and meet their needs starting with your company mission. From this perspective using data is not tied to an industry or type of product offering but rather a general way to deepen and refine your connection to customers.
Kumar Samanvaya Misra (Product Manager, Data Monetization at BASF)
Decisions commonly used to be made based on gut feeling or optimizing the risk. Today with the availability of data, we are increasing our confidence in those decisions. Too much reliance on data-driven decision-making takes us into a habit of using past information to predict the future. This makes us helpless to deal with disruptions caused by black swan events. When such an event as a pandemic hits unexpectedly, all data-driven approaches go for a toss, and this anomaly needs to be fed into the system to fine-tune itself further.
Q. What makes a company data-driven? How can a data-driven culture be developed from scratch?
Being data-driven is not a set of checkboxes to complete; it is a mindset that must be pervasive at every level of the organization. It is an integral part of the company culture, not a side project that only a few people work on. In a data-driven culture, every team member relies on data to do their jobs and evidence-based decision-making is emphasized. The organization’s confidence is bolstered by data points, not opinions.
To develop this type of culture, an organization must assess how its decision making is done today. The company must agree on the vision for a data-driven culture and also the level of risk it is willing to accept in decisions. Processes and protocols need to be updated to make room for data-based decision-making. For example, data requirements need to be included in the feature definition and completion phases, while reviewing data as a team on a regular basis. Data should be part of every product manager’s daily activities. Since this could be a massive task, it may be helpful to start with areas of the business where using data is quick and easy to get started, then expand over time.
Nick Allen (Group Product Manager at Booking.com)
To be a data-driven company, every individual team member, team, and department respects and relies on data for all elements of their jobs. Decisions cannot be made without data. Team members should be free to exert their opinions and ideas based on their expertise and knowledge, but there must be an environment that allows challenge.
In order to foster a real data-driven environment, there needs to be clear alignment around the level of ambiguity or risk the organisation is comfortable with for decision making and there needs to be recognition that there is a trade off to be made. For more certainty, you sometimes need to accept that things will move a little slower. Gathering data takes time and investment.
Booking.com is a very heavily data-driven company. Many will have heard about the number of experiments that can be running at any time on the accommodation booking website. This model works really well for the extremely well established core part of the business. For newer, less mature products we work on, such as the Rides business I work in, there are sometimes tradeoffs we need to make. We can operate with a little more risk as there is more benefit in moving quickly when making our product decisions.
Elaine Chao (Senior Product Manager at Adobe – Creative Cloud)
A data-centric culture has to permeate every part of an organization from top to bottom; executives need to understand the value, and even the most junior of developers should be asking the question: “What do we need to measure?” An organization should ensure data requirements are included in the definition phase, set an expectation that data is part of the completion of a feature, and then review data together on a regular basis, reporting success, and identifying further questions. To develop a data-driven culture, data should be a part of every product manager’s day-to-day activities, but it has to be a part of a daily workflow.
Karl Lillrud (Professional Speaker, Business advisor and Mentor)
The best way to develop a data-driven culture is just to get started and not overcomplicate things. Find small things that you can track. For example, decide you will be data-driven within a particular segment or specific feature. Find something where you think you might be able to bring lots of value. Basically, start with the low hanging fruit, where becoming data-driven is not that difficult and then you can prove that being data-driven is the right way to go about growing your business.
Kumar Samanvaya Misra (Product Manager, Data Monetization at BASF)
In my opinion, being data-driven is not a process, it’s a mindset. And in order to promote the right mindset it has to be made a priority in the company culture. Making an evidence-based decision is a key identity of a data-driven company. I have seen that providing psychological safety to your team and allowing them to fail fast helps in exploring various opportunities to discover rapid growth.
Laura Onu (Product Manager at Meta – Reality Labs)
A data-driven culture starts with the vision. Here are some questions to ask your team:
- What are the macro trends that support this need?
- How do you size the opportunity?
- What net change will your vision bring?
Ask who, why, when, where, and how often again and again. If your team’s confidence grows as data points are gathered, you have a data-driven culture.
Q. Which approach is better for transforming a company to be data-driven: top-down (from upper management) or bottom-up (from data enthusiasts)?
Our experts agree that companies need both a top-down and bottom-up approach for a data-driven transformation to be successful. Support from upper management is imperative because significant investment is necessary, including hiring experts, allocating resources, educating the team, and updating existing processes. Without the support from the top, a data-driven transformation cannot be sustained.
However, a top-down approach alone will not work. Everyone in the organization needs to be aligned around a common vision and define how data-driven decisions will be achieved. Without the support of the teams doing the work, data could be incorrectly gathered or not used at all. If the team doesn’t believe data is useful, it won’t be.
The table below shows how everyone in the organization has the responsibility for a data-driven culture.
| Top-down | Bottom-up |
| — Hire needed talent — Allocate resources — Invest in updating processes | — Gather data correctly — Use data properly in daily activities — Rely on data for decisions |
Igor Akimov (Head of AI Solutions at Wrike)
Transformation can start with an ambitious individual, but without significant support from management this initiative will stay as a local activity for a long time. So it is better to “infect” the data-driven approach to someone at the very top. Without top management’s support and belief in the project, and lacking common processes, practices, meetings, and artifacts, the initiative won’t be sustained.
Nick Allen (Group Product Manager at Booking.com)
A combination of both approaches is best. Leadership has to want the change and needs to hire experts in roles that will execute the data-driven model. Leadership’s buy-in is important because there is significant investment to make sure that the right data strategy is in place. Data must be gathered and stored in a way that allows the value to be extracted to answer the organisation’s questions. This should be a continual investment with teams owning the data strategy, quality, and management in the long term.
Elaine Chao (Senior Product Manager at Adobe – Creative Cloud)
Transformation isn’t easy in any organization, and it has to happen from both the top-down and bottom-up to be successful. Without top-down support, critical resources won’t be allocated for data management (including collection technology, infrastructure maintenance, software licenses, and defining a useful and performant data structure), advanced data analysis, and data compliance. If organizational leadership doesn’t hold data as a value, there also won’t be accountability for implementation across the board or time allocated for each individual contributor to work on data as a part of implementation.
At the same time, without bottom-up support, data could be inconsistently or thoughtlessly gathered. If it’s just a checkbox and not providing real value, or if a team doesn’t take action on the available metrics, data is useless. I work with our data scientists to ensure that we have the right metrics in place to measure utility and specific feature use patterns before we start implementation. We then add more measurements as the questions change or deprecate the measurements if they’re no longer necessary to gather.
Karl Lillrud (Professional Speaker, Business advisor and Mentor)
Not being just top-down is one of the most important steps to actually empowering the people within the company to make their own decisions. Everybody should align around a common goal and/or vision. And with that common vision it’s easier to define how to become data- driven within areas that are of highest value.
Laura Onu (Product Manager at Meta – Reality Labs)
A cultural shift will always happen organically with support from all people regardless of their level. The future always exists in the present within a few small groups. Those groups will create high impact products that will create curiosity and desire for other teams to try and test out the approach.
We’d like to thank Stephanie Walter for incredible help in creating this article.
