Pali Bhat on building intelligent products

In this episode of McKinsey on Building Products, a podcast dedicated to the exploration of software product management and engineering, McKinsey senior partner Chandra Gnanasambandam and partner Rikki Singh sits down with Pali Bhat, chief product officer of Reddit. Bhat has been at the forefront of product innovation for several large technology platforms, including Google. During their conversation, Bhat and Gnanasambandam, and Singh discuss how to find untapped opportunities to create lasting value, how technology will shape product management, the importance of clear product principles, and the merits of learning from mistakes. An edited version of their conversation follows.

Bringing technology and people together

Chandra Gnanasambandam: Pali, could you share one or two unexpected events that shaped your career?

Pali Bhat: Two events had a profound impact on me and the journey I took in technology. First was seeing the power of technology firsthand. Unlike some common origin stories you hear of people having a computer when they were eight, or writing programs and selling software by ten, I didn’t get to see my first computer until I was in college. So having the opportunity to go to the University of Illinois Urbana–Champaign (UIUC) was life-changing for me. UIUC, for some of the younger folks in our audience today, was where foundational pieces of the internet were built. It was an extraordinary time when we could see the future of something that was going to change the world. That was profoundly impactful.

The other influential event was seeing an early version of Reddit well before I was the CPO [chief product officer], more than a decade ago. On Reddit, there are communities that are organized by what are known as “subreddits.” Think of it as an open canvas that anyone can find. In 2022, we allowed every user on subreddits to place a single tile on their canvas to create art. Over a span of three days, users placed north of 100 million tiles. The only way you can create something magical like that is by doing it with your community. It was amazing to see people collaborate like that and work with their communities. Seeing the power of technology and the power of people come together shaped my own personal journey and continues to inspire me.

The influence of generative AI on product management and engineering

Chandra Gnanasambandam: Speaking of the power of technology, let’s talk about the next generation of it, which is all about generative AI [gen AI]. At Reddit, what are some examples of ways PMs [product managers], engineers, and architects use AI in a creative way to drive productivity?

Pali Bhat: The first example is from our safety team, which is working on ways to identify time with a homegrown large language model [LLM]. We have millions of data points that are part of this data set that help us identify what is and isn’t spam or what is and isn’t hateful content, for instance. We’re training our own model to automate this process, which will take a lot of toil off our internal teams and moderators. The moderators of Reddit are volunteer moderators, often experts in their communities or in their own areas, so we want them to spend time leading those communities and not worrying about these toils.

Chandra Gnanasambandam: Are there innovative ways in which your engineers are using it?

Pali Bhat: Our teams are trying out various forms of AI. Usually, it takes two aspects. One is not necessarily a full generation of code—I think of it more as a coding assistant. We’re trying it in several of our teams. We’re also looking at use cases such as helping us with better commenting in our coding so that our code base becomes more maintainable over time; or generating test cases, so we can get test coverage for a much lower cost than if we had to write it. That way, engineers can spend their time building awesome products and not having to worry about some of the mechanics. It’s still early days, but the potential we see is enormous in terms of increasing productivity.

How product principles shape product vision

Chandra Gnanasambandam: You’ve talked before about building helpful products for the community. What do you mean by that, and how you go about doing it?

Pali Bhat: Ultimately, it’s about focusing on the user. I often tell my teams: we all have heard of product market fit, and we all understand that, but there’s also a pre-product market fit test that you need to think about as you’re coming up with products.

The first step is determining if you are either solving a real pain point or enabling people to do something they couldn’t do previously. If the answer is yes to either, step two is determining if you are going about it in a differentiated way. What about your take is unique and creates some kind of enduring value? Sometimes you can still work on things that are undifferentiated, things that might not stand the test of time, but you have to think hard about how much you want to invest in that. What are you using that for? Is it just an add-on, or is that the core of your product? If it is the core of your product, then maybe you’re not thinking about it holistically. The third step is determining if you can deliver on these plans with the resources you have—the people and the tech.

For example, more than a decade ago, I worked on building payments into Chrome at Google. The problem was simple: online checkout was hard. There was too much friction. So this was a real pain point that we were solving—a check mark for the first step. For the second step, we differentiated our product by using one of the earliest versions of ML [machine learning] at Google to help complete the information in a checkout form. This sounds simple, but at the time, many websites around the world couldn’t distinguish between a credit card number and a phone number. As we gathered more data, the feature became dramatically better.

We were able to pull it off with a very small but mighty team through a collaboration across the Chrome team and payments team at Google. In the first few weeks, as we were running experiments internally, we were approaching 60 to 70 percent accuracy. Within six months, we crossed 95 percent accuracy. It was amazing. The website was also seeing an increase in conversion in double-digit percentages, which is shocking, because an increase of even a percent is massive for an e-commerce platform. It was a win–win across the board. In the few years after that, we saw a massive drive, and many teams that came after us have built additions to this, so it has enduring value.

Chandra Gnanasambandam: Do you have any counterintuitive product management hacks you’ve learned during your career?

Pali Bhat: There has been a huge emphasis on and awareness of the importance of data, which has shaped how people think and become central to how many products teams function. For example, at Google we were looking at a dashboard to assess the product quality for end users and consumers, and we noticed a tiny blip on the dashboard. It turned out that 0.1 percent of users were experiencing a degradation of service—what is conventionally called a “brownout.” It didn’t sound like much, but for a product that has a billion users, a 0.1 percent degradation is a million users who are having a bad experience. So data is an imperfect substitute for users—focus on the users.

Second, very rarely will you find product teams that have product principles rather than product requirement documents. Product principles can be powerful for team velocity. It may seem counterintuitive, because having a principle and improving velocity are not obviously connected.

Chandra Gnanasambandam: What’s an example of a great product principle?

Pali Bhat: At Reddit, we are committed to democratizing conversations globally, so we have a principle that makes it so no one on Reddit has a megaphone: every comment or post someone makes on Reddit starts on a level playing field. The users in the community then vote on the comment. Moderators then weigh in on whether it is appropriate for that community and satisfies the norms and values of that community. The effect of this is that users see more diverse conversations, and the highest-quality, most authentic conversations rise to the top. That’s the magical thing about Reddit—it’s not a platform where some users can push content to 100 million users, regardless of the quality or authenticity of that conversation, which has a huge impact on safety.

Internally, this principle gives teams the ability to make decisions in a decentralized way because they don’t have to escalate or debate every decision to make a new feature or requirement. When product principles are built in an enduring way and you’re consistent about them, they can drive velocity.

Understanding what AI and ML can do for products

Chandra Gnanasambandam: Gen AI is the topic of the hour because it might be the next computing platform. How should PMs today think about the skill set they need?

Pali Bhat: First, we should take a step back. Often, whenever product teams, product managers, product leaders, and engineers look at some new tech, the first question they usually have is, how can I apply this to my product? I encourage everyone to spend some time understanding what an AI or ML pipeline looks like to understand what it can do well and what it can’t do.

Unfortunately, not every PM and engineer is comfortable with ML concepts. They have to consider what reinforcement learning with human feedback looks like. Or when they have an ML model, are they training it using existing information in their systems or seeking new information to advance it? That will help them understand how a probabilistic model would work. It’’s a different paradigm of how to build software and a different paradigm for how to build products.

Once you commit to understanding how an AI and ML pipeline would work, the next step is to figure out where you sit in the value chain. On the product side, you could be building a portion of the emerging AI stack. For instance, Reddit will offer prepared and curated data sets for training large language models because Reddit is one of the largest corpuses of moderated, curated content that’s available online, and conversations that are available online are organized by topic. Reddit data is a very powerful data set, which is why many of the LLMs today have been trained on it.

The other extreme is taking your existing product and adding gen AI to it. For example, you could use an off-the-shelf API [application programming interface] as a conversational interface for your product. Then you’re operating at the application level as opposed to the stack level. But you have to be thoughtful about your data because if it is being ingested by some gen AI, there could be risks on the regulatory side or confidential information could leak.

Once you understand how you’re using ML and AI, you’ll know what skills you need.

The value of constant learning

Chandra Gnanasambandam: Pali, what mistakes have you made in your career, and what have you learned from them?

Pali Bhat: I have a laundry list of mistakes, longer than the accomplishments that I have. Once, at Google, I was working on payments, and we made a bad mistake involving a large sum of money. The only question my manager had was, “Did you learn that many millions of dollars from this experience?” Whenever there’s an issue, the most common thing teams do is try to find the root cause, but my manager asked, did you learn from this mistake, and did you learn from the magnitude of this mistake? The moment I pivoted my thinking from mistake avoidance to learning, I started to treat all mistakes as learning opportunities. As long as you don’t keep repeating the same mistakes, you’re in great shape. That’s also how you should treat mistakes from your team. Often, it’s easier to learn from your own mistakes than from others. It’s good if you can learn together as a team.

Chandra Gnanasambandam: What advice do you have for new product managers?

Pali Bhat: Be curious and always be learning. You never know where learning comes from—try to learn from everyone and everything. Do not think that only product managers know about products. Your sales and engineering teams have opinions about products, and they all will be valid input for you. Also, focus on the user, whoever they are. It’s so easy to get distracted with technicalities and take your focus away from the user.

Chandra Gnanasambandam: A lot of PMs feel like they get stuck in middle management. What are some ideas for how they could progress?

Pali Bhat: First, try to make an impact. If you have a profound impact on your users and on your business, the promotions will come. If the promotion doesn’t come, then you can move on and try again. At that level, impact comes only in the form of a larger team. So don’t just be a product manager or an engineering manager—be a leader as well. Rally people at scale. You will get recognized if you move the needle for the company and for the users. That’s a function of being able to galvanize a large group of folks to work on a real pain point or a breakthrough piece of innovation, and then delivering on it so you’re creating enduring value.