Working with data helps companies across the board to unlock their potential and become more productive and better at making decisions. However, making people in the team and company rely on data involves a lot of work. Product managers must often set a strategy, reinvent processes, and change organizational behavior.
To find out how to make more people in the team use data in decision-making and daily work, we spoke to product managers from different companies and industries. Their answers provide insights about the following:
- Which members of product teams can benefit most from using data?
- What are the key barriers to using data by all members of your product team?
- How to overcome the barriers mentioned above?
- What specific tactics can help to increase the adoption of data use in a product team?
- Which tools and apps are helpful for product teams?
We would like to thank all the experts who shared their experience with us and helped us answer these important questions.
- Adi Debel (Senior Product Manager at Walmart Global Tech)
- Sunetra Virdi (Senior Product Manager (Technical) at Azure Purview | Azure Data | Microsoft)
- Rodion Lyayfer (Senior Product Manager, Data at Zillow)
- Gavin Deadman (Lead Product Manager, Betfair at Flutter Entertainment Plc)
- Julia Markhadaeva (Founder and CEO at to Stories, Ex PM at Soundcloud)
- Jashyard Johnson (Senior Product Manager, Finance at Carvana)
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
Q: Which members of product teams can benefit most from using data?
Data is helpful to each and every member of the product team. Using a data-driven approach will make it easier to understand your customers, analyze metrics and anomalies, prioritize features, and be objective about decisions.
Let’s dig deeper into different roles in product teams using data:
Adi Debel (Senior Product Manager at Walmart Global Tech)
Data is helpful to PMs, engineers, designers, and every other team member joining the ride. As a Product Manager, I try to bring data to every conversation to help us as a team align our priorities and the “why.” One of my first initiatives after joining my current team was ensuring we have the right data infrastructure and dashboards in place, so every member has access to key metrics easily and when I say “DAU is down 10% MoM” we are all on the same page on the definitions.
Sunetra Virdi (Senior Product Manager (Technical) at Azure Purview | Azure Data | Microsoft)
The first group of people who can use data most are the engineers. Customer telemetry helps teams understand customer pain points and develop the product. In my experience, if customers can’t complete a workflow or process end to end, it usually indicates the steps are not clear or there is a bug. I reach out to customers to understand what’s blocking them, and as I learn and share information with my engineering team, they redesign the feature to clear the blockage, hence resulting in higher customer satisfaction.
Rodion Lyayfer (Senior Product Manager, Data at Zillow)
I can see a big impact of using data specifically in Product Management roles. Whether you’re trying to understand your customers, analyzing metrics and anomalies, prioritizing features, troubleshooting — a data-driven approach will help justify the direction you take.
For example, I am working at online real-estate marketplace Zillow and I receive a bug report from my team and need to decide on its priority. I must analyze how many listings and which markets are impacted to identify business and user impact to prioritize.
The same approach will help you in troubleshooting. Once I received a request to dig into email and app notifications issues. The first thing I did was to look into notification logs to understand if the issue is systemwide or localized. It helped me understand that the issue is coming from a data ingestion system. The next step was to look into logs of that system to understand what exactly is causing the issue. In the end, the appropriate fix was prioritized.
And one more example of when you use publicly available data for storytelling: When presenting a new feature, I used information published by competitors and other reliable sources from the industry to explain the market condition and justify why a specific feature is important.
Gavin Deadman (Lead Product Manager, Betfair at Flutter Entertainment Plc)
As product designers and developers conduct experiments to validate the impact of a product change, it will be crucial for them to first make sure they can measure success and then monitor the data as it goes live. Otherwise, it will be impossible to understand the ROI and celebrate success.
Q: What are the key barriers to using data by all members of the product team in your experience?
Data can help improve decision-making, gain competitive advantage, and transform the way the business operates. However, achieving these benefits can sometimes be challenging.
Based on PMs’ answers, product teams face three primary challenges to make their teammates use data:
- Building a data culture
- Consolidating data from different sources and making it accessible
- Providing quality of data and training in data interpretation
Challenge 1. Building a data culture
Jashyard Johnson (Senior Product Manager, Finance at Carvana)
It’s important to build a data-centric culture. I believe making data top-of-mind allows people to spend more time using data in their day-to-day work. This can be a barrier if your organization doesn’t value data and analytics. Having messy data typically leads to bad output and frustration for the business and users.
Rodion Lyayfer (Senior Product Manager, Data at Zillow)
Sometimes people don’t know the data they need exists at all. For example, you might be a PM on the backend team that owns transaction information but you don’t know that there is also a clickstream data available somewhere.
Julia Markhadaeva (Founder and CEO at to Stories, Ex PM at Soundcloud)
Encouraging people to make a shift to something they’re not used to is difficult. It takes a lot of practice before new behavior is adopted and becomes a habit. I guess it’s just human nature. Even if we all agreed that something was useful and important, that wouldn’t mean that we all could instantly apply a new routine to our daily work.
Challenge 2. Consolidating data from different sources and making it accessible
Julia Markhadaeva (Founder and CEO at to Stories, Ex PM at Soundcloud)
Everyone should know how to navigate through the company’s existing dashboards and charts, how to get the analysis they need, and how long they have to wait. A problem with accessibility could be easily discouraging and usually require a lot of effort and discipline to overcome.
Adi Debel (Senior Product Manager at Walmart Global Tech)
Whether it impacts operation, product, or business, data needs to be maintained in a usable format for analytics and aggregates to provide insights. It’s important to set clear success metrics to every project and initiative that have clear measurement practices. Not all team members are going to feel comfortable running Python scripts or SQL queries to pull data, and even if they do, it’s often too much hassle for them to do so. Providing easier ways to pull data is crucial to make your team more data fluent.
Rodion Lyayfer (Senior Product Manager, Data at Zillow)
People know there are multiple data sources but it’s very hard to connect them together. For example transactions data might be stored in a relation DB on one server while clickstream data is stored in a data lake on a different server with no easy way to connect those servers.
Also data usage restrictions can cause people to use gut feeling instead of a data-driven approach. For instance real estate data in the US is highly regulated. You can’t just use any data for analytical or derivative purposes even if you have it.
Gavin Deadman (Lead Product Manager, Betfair at Flutter Entertainment Plс)
All transactional, analytical, and qual data should ideally be in one tool, making it easy to access. Also, the speed of pulling the data is important. If data takes more than 10 seconds to load after each query it discourages people from using the tools.
Challenge 3. Providing quality of data and training in data interpretation
Sunetra Virdi (Senior Product Manager (Technical) at Azure Purview | Azure Data | Microsoft)
The quality of the data can become a key barrier to more data-driven decisions in a product.
Julia Markhadaeva (Founder and CEO at to Stories, Ex PM at Soundcloud)
There’s no value in data if you can’t make sense of a bunch of charts. So it always comes down to the skill of reading the data and telling a story. Also important is the ability to distinguish between cause and the effect.
Q: How did you overcome the barriers mentioned above?
As a product manager you should break silos, create a data-driven culture, and encourage members of your team to learn and provide accessible data.
Here is what the product managers we spoke to recommend:
Julia Markhadaeva (Founder and CEO at to Stories, Ex PM at Soundcloud)
1) Make sure that data is both available and accessible to team members
2) Invest in educating everyone on translating numbers into actionable product insights
3) Create a culture that encourages everyone to be more data informed and also lead by example
Jashyard Johnson (Senior Product Manager, Finance at Carvana)
I started building a better relationship with my data engineers and analysts in order to build a data-centric focus to align goals and improve our planning. This allowed me to prioritize having clean data flow through our systems, which ultimately provides value back to the business through analytics.
Rodion Lyayfer (Senior Product Manager, Data at Zillow)
Some of the blockers are removed by having dedicated teams who consolidate data (usually Data Engineering teams) and present data in reports and dashboards (Business Intelligence teams). As a Product Manager you should work across teams and across orgs to break silos and learn.
Talking about data restrictions, one of the approaches is to acquire other data sources that give you the extra coverage you need for your product development.
Gavin Deadman (Lead Product Manager, Betfair at Flutter Entertainment Plс)
It’s helpful to prioritize the need to have front-end analytical data to connect to transactional data in one system and ask for updates weekly. Mentioning the impact helps to drive action.
Sunetra Virdi (Senior Product Manager (Technical) at Azure Purview | Azure Data | Microsoft)
To maintain good quality of data we start by thinking about the customer data much ahead in the product development process. We clearly define why we need this data and how we would learn from it, or what KPIs will be informed with this data. That helps in maintaining the purpose and hygiene of the data.
Q: Can you share specific tactics that helped you increase the adoption of data use in your team?
There are some practices that can help product teams overcome the barriers to using data.
Our experts had the following key recommendations:
- Ask right questions to uncover challenges you’re facing and generate better solutions
- Use different KPIs to track the team and the product effectiveness and review core metrics on a regular basis
- Encourage team members to share and discuss data
- Set tools and processes for self-service data analysis
- Lead by example in the workplace
Gavin Deadman (Lead Product Manager, Betfair at Flutter Entertainment Plс)
One of the best things which has helped the team use data more is asking better questions to drive action. What do users think when there are multiple design options to choose from? How can we measure success? How will we measure the impact of product development work once we go live? What are our product’s strengths and weaknesses in the market? What are our top-10 customer support queries and how can we reduce them? What data do we have to inform us that the proposed solution will likely solve the problem?
Julia Markhadaeva (Founder and CEO at to Stories, Ex PM at Soundcloud)
It usually comes down to some of these very basic activities:
- Introducing core product KPIs, team KPI, and how everything is related.
- Setting up a dedicated hub with dashboards that is relevant for the team’s current focus.
- Setting up automated reporting on relevant data via emails and/or Slack bots. This helps build awareness within the team and create the right context.
- Supporting any discussion or side chat with data snapshots, so there’s always a quantified insight as context.
- Introducing a habit of sharing data insights proactively.
- Encouraging team members to build their own dashboards or run their own data analysis as part of their product discovery.
Adi Debel (Senior Product Manager at Walmart Global Tech)
Showcase the value of using data. Tactically, what worked for me was reviewing the core metrics with my engineering team on a weekly basis: what’s working, what’s not working, and what can be improved. It helped us all get in the mindset of outcome vs outputs and think about what we would like to achieve in the form of moving the needle on those metrics.
Rodion Lyayfer (Senior Product Manager, Data at Zillow)
One example is from the time when we worked on a massive project to redesign our entire data ingestion pipeline, which meant all the data flow was changing. Here are a few things we did to help the users understand:
- Drew a high-level data flow with key systems to help our data consumers understand the overall process
- Created a UI tool with the ability to look up most used data
- Educated users on how to use the UI tool
This helped data analysis to become more self-service.
Another example: I worked with a Product Manager from a different team on data analysis. He happened to know SQL well and had direct access to the database. I offered to show him how I analyze data, explain more about the dataset itself and shared SQL queries I usually use for requests from his team. That was really helpful and after that he did the analysis on his own.
Q: Which tools and apps are helpful for product teams to increase data usage in decision making?
Special tools and apps can help product teams use data to assess their development efforts, optimize performance, remove roadblocks, and increase customer satisfaction. Such instruments provide access to different types of data, and they have a modern infrastructure, high speed data access, and other capabilities.
The PMs we spoke to recommended these tools and apps for product teams to increase data usage in decision-making:
- Tableau
- Databricks
- Snowflake
- Excel or Google Sheets
- Firebase
- Looker
- Amplitude
- Google Data Studio
Jashyard Johnson (Senior Product Manager, Finance at Carvana)
We use Tableau for all our reporting needs, which is very user friendly for analytics. Our team also uses Databricks and Snowflake, which have amazing infrastructure for our data engineers to work with big data and analytics.
Rodion Lyayfer (Senior Product Manager, Data at Zillow)
My first go-to is to offload data from a DB into an Excel or Google Sheets where you can easily slice and dice data, make almost any analysis you want, and create pivot tables and charts.
I also use different third-party reporting tools to present data to others: from one-time reports to dashboards that are updated in real-time. These days a lot of data management tools have reporting capabilities: you can build reports in Snowflake or Datadog.
Gavin Deadman (Lead Product Manager, Betfair at Flutter Entertainment Plс)
My favorite tool is Tableau. Its data visualization options and data access speed are fantastic if architected appropriately, and it’s quite easy to load different types of data from different sources whether from the transactional DB, Google Analytics, or qual data from surveys. I also like Firebase analytics for app performance. I’ve had experience with Looker, but I’ve found Tableau to be more effective in terms of speed of querying the data, ease of using the tool, and analyzing trends in the tool itself.
Julia Markhadaeva (Founder and CEO at to Stories, Ex PM at Soundcloud)
Tools like Looker, Amplitude and Google Data Studio have email scheduling for dashboards. I found it super useful to have specific dashboards to be regularly sent as weekly or monthly newsletters not only for myself, but also for team members and stakeholders.
→ Test your product management and data skills with this free Growth Skills Assessment Test.
→ Learn data-driven product management in Simulator by GoPractice.
→ Learn growth and realize the maximum potential of your product in Product Growth Simulator.
