We’ve transcribed some of our most popular Product to Product podcast episodes for you to peruse.
In this episode, we chat with Rose Yao, who was the Senior Director of Product Management at Google Maps Platform at the time we interviewed her. (Rose is now the Senior Director of Product Management at Google Nest.)
Rose will talk about how she likes to collect customer feedback and data, what metrics she thinks every PM should have on their dashboard and some of her user interview techniques.
If you’d like to listen to the entire episode, click below.
Highlights
(The highlights have been condensed and edited for clarity.)
Can you tell us who you are and what your role at Google Maps Platform is?
Rose Yao: My name is Rose and my last job is the Senior Director of Product for Google Maps Platform.
The best way to talk about it is actually to maybe give a brief origin story on this enterprise product. When Google Maps launched over 10 years ago, it was very much a consumer product. Then people quickly started hacking the product to make their own map mashups. The most famous one was the Craigslist mapping tool where you can see all the apartments around you on the map, which is super useful. They did that without any support from Google.
Very quickly, Bret Taylor who was the original PM on Google Maps who's now the COO at Salesforce actually said, okay, well, there's clearly demand. Let's actually make an API that people can use legitimately so that we can provide a service for the world. We launched as an API––it was web and JavaScript only. It was very advanced for its time and was completely free for developers. Then within a couple months we had businesses call us and say, hey, can we please pay you for this?
They actually said, we would like to pay you for this for two reasons. One, we want a little more bandwidth and scale than you're providing for your free product. Then two, because we want you to take this seriously, we would like to run our business on this platform and we figure if we pay you, you won't change your mind.
In your experience, what's the best way to collect customer feedback and data and how do you like to do it?
Rose Yao: This is a question a lot of people ask me. I want to say the key is that every product is different, right? Also every company is different. Some companies have a lot of resources and they can invest a lot in this. Other companies are tiny and you have to be resourceful.
I'll give you the three big buckets that I think about when it comes to feedback and talk about the different ways I've used each one.
First and foremost is looking at how your existing customers are using your product. I like to do the macro economics version of that, which is looking at your dashboards and your key metrics. Really diving into the numbers you have and saying, “Is the business or the product trending up or down? Is it growing faster or slower?” Those are just simple questions that you can answer right away.
Then, usually it's about cutting that data a little bit through your intuition. By country, by language, by customer segments, by email business types, by user types, by age––all those different things can give you a different picture. That usually gives us what we call “finding where the smoke is” and then going and diving deeper into that.
That leads to the second bucket from feedback, which is the qualitative feedback. In my 14 years working in tech, I would say there's nothing as powerful as actually talking to a user of your product, whether they're paying you or not. There's also something very motivating about that for product managers and engineers because there's a real person who’s expressing their delight or their annoyance with your product.
I’ve walked out of usability studies being like, “Yeah, I’m going to fix that right now!” I'm empowered to make this person happier. A lot of it is about actually asking people how they use your product. Why do they use your product? What makes them happy? What doesn't make them happy? There's a couple caveats to that one, which I think is very important, especially for new product managers. Talk to at least 10 people. Also, try to make them as diverse and representative of your customer segment as possible because it's too easy to fall into a feedback loop. Make sure that person does not know you! The worst thing I feel like executives or PMs can do is come to an engineering team and say, “My mom told me…”
A lot of users don’t like offending people and it’s difficult to tease out actual customer feedback. What’s your take on this?
Rose Yao: It is actually really tricky. Often when we design these studies, we'll say we're trying to recruit 10 people. We want three of them to be brand new users, three to be lightweight users and three to be power users. The last one is someone who we can easily recruit. We might talk about demographics, like we're looking to understand how this product performs with teenagers versus with moms. Those are going to have very different kinds of feedback.
Designing your user studies and feedback sessions just as thoughtfully as you're designing your product means that you'll get the best feedback and insights from that exercise.
The third thing that I think is valuable and not to be ignored is that it's important to talk to the people who work on the product.
Often my advice for new product leaders, whether they're junior or senior, is to spend at least the first month talking to everyone who works on the product and asking them what they think is the most valuable thing to do. What do they wish they could spend time on? What do they think is going well and not going well? Because if you think they're wrong, at least you know why they have the opinions they do.
More often than not, the team has a lot of great insights. If you listen well, you'll find that commonality and you'll be able to create a more unified vision and a roadmap that everyone can get really excited about.
You mentioned in our pre-interview that you think tying the customer feedback to strategy and using it is actually just part science and part art. Can you explain what this means?
Rose Yao: The science part is a little bit of what I just talked about. You have to have data to back up your intuitions or your assumptions or the strategy that you're proposing.
I'll give you an example––in some ways I might be making fun of my own company a little bit––not everyone has the luxury of running an experiment for a hundred shades of blue. If we have all the time and resources in the world, yes, we could ask every single question and turn over every stone, compare the data on every single possibility and try to find the right answer. But even at Google, we don't have time for that. I know we're a huge company, but we feel resource constraints as well.
The key is we want to hire product managers because we're looking for people with judgment, who have done this job or have thought hard enough and lived in the data, in the feedback, in the product and understand the market, are excited about tech to say, “I want to make some bets.” My intuition says that the on-demand economy is going to continue growing. We should dive deeper into what those customers need and what the next big trend in that industry is.
I know this is somewhat frustrating for people who are just starting a product management career. The intuition will come.
How do you develop this intuition for problem solving when facing customer feedback and data?
Rose Yao: Part of it comes with time and experience. Part of it is, I think you mentioned problem solving.
A lot of product managers have engineering degrees. Not all, but quite a few. Certain people have business degrees. But the key is having the ability to learn and dissect problems.
When I hire product managers, even junior ones, I'm looking for people with pretty good taste, people who intuitively understand or can talk about why a product works and why a product doesn't, even if it's something like a vacuum cleaner. Certain products are truly delightful. The person who can explain it, who can really think through what makes it delightful, it means that you have the potential and the fundamentals of product intuition.
An additional trait is listening to your own instincts. Practice making decisions by using the team that surrounds you. Tap into your mentors, your manager and senior engineers on your team. Tell them, “I’m thinking of X,Y,Z. What do you think?”
When it comes to collecting data, do you think all PMs need to be data savvy PMs?
Rose Yao: Some basic data savviness is necessary, but I don't think all PMs need to be incredibly analytical. I often think that none of us are great at everything and different PMs are great at different things.
When I have a product that is very future-facing, for example when I'm hiring someone for the AR/VR efforts, I'm not looking for someone who is really good at data. I'm looking for someone who's really creative and who's a little bit of a dreamer because that's what that role requires. When I'm looking for someone for the growth team, they need to love data. Their skill involves being able to read data, have the intuition and cut the data to get the insights they need. For infrastructure related efforts, I'm looking for someone very technical, who almost mirrors the skills of an engineer.
What are some things you think a PM should know at all times about their product?
Rose Yao: On a typical morning, I look at about half a dozen key stats and the trend lines that follow them. I usually look at whatever the main dashboard is for my product or business. When I was on Facebook photos, it was the number of photos shared per user per day. I want to see if it’s going up or down. Is it following the trends? What does it look like year over year? I don’t check it every day expecting it to change, but because I want to be familiar enough with the data to point out anomalies. I want to be able to detect changes in the world that the data is telling me about.
The other thing I do every day is I keep track of tech news and headlines because I think one of the areas that PMs should be looking for ideas and should have pretty good passive knowledge on is where is the world going. What startups are gaining traction? What is this big new breakthrough that people are excited about? What's buzzy on Reddit today? I find myself piecing these things together in random ways as we talk about products.
What I'm hearing you say is you have to step back and not just focus on your product, but look at what else is happening around the world in terms of trends.
Rose Yao: That's right. Ultimately we're building products to serve the world. All of us in our ideal state would be building billion person or billion dollar products. In order to do that, it's not about what's inside your company or what's on that dashboard, it's about what's happening outside.
In a scenario where you receive a lot of customer feedback about a big problem with your product or your app, what's the first order of business as a PM?
Rose Yao: Usually it's to identify the right customers to talk to in order to understand the problem better. Make sure it’s the representative set and that you’re not just listening to one segment.
In the case that you have a big problem, I like to identify usual customers that are really unhappy and then customers who are actually pretty happy. Once you identify the customers, don’t ask them what the problem is right away. If you jump straight into that, you'll miss out on the real problem.
I like to take a step back and say, “How are you doing today? Tell me more about your business or your life.” Let's take a problem that probably more people understand. Let's say you're talking to college students about why they like Facebook or not. You might just say, “Tell me how life is going. Are you busy at school? What's your day like?” Then you might say, “Oh, how do you communicate with your friends? How do you keep up on what's going on on campus?” Listen to them as they talk about the products they're using. That actually gives you competitive insights as well. Are they using WhatsApp or Instagram in different ways than the main app? Are they using Google? Are they using TikTok and why?
It's important to be curious about their whole lives. I find that these background conversations only take 10 minutes, but give you a lot of insights.
The best product managers listen a lot better than they talk.
Looking to make your product prioritization more data-driven? Read all about it here.