What API Product Managers Need

Transcript

Goyal: My name is Deepa Goyal. I’m a product management strategist at Postman. I’m going to be talking about what API product managers need. In order to start thinking about APIs as products, I’ll be sharing some frameworks to understand how do you identify the right stakeholders across API product lifecycle? How do you establish user research strategy, understanding developer journey to develop that empathy for your users? Also, we will be thinking about API maturity-based distribution model. Then going into what makes up for a great API experience and looking at all the components. Also, how do we develop effective feedback loops so that we can collect customer feedback and improve and iterate upon our APIs.

Mapping the API Producer and Consumer Lifecycle

Let’s talk about the API producer and consumer lifecycle. The API producer is the team that is producing an API. The consumer, as the name suggests, are the teams or customers who are integrating with your APIs and building their applications. API producers are generally teams, and not just teams of developers, but the lifecycle of the API producer actually spans over multiple teams. The first step being defined, which is where you’re defining what your API is going to be about, who is it for. This is where a product manager writes the requirements and user stories. Then you get to the design phase, where your architects and your team leads jump in to understand those requirements and convert them into an API design. Then you start the development process. Even before you develop, you have to think about what you’re trying to build, who you’re building it for. Those steps happen in the define and design phase before jumping into develop. Then you get into testing. You think about security, where security teams will help you evaluate your APIs in terms of making sure that they meet the security guidance of your organization. You should also be thinking about deployment generally taken care of by SRE teams. Then you get into implementing observability, and establishing monitors and analytics. It’s very important to measure. As you release any product, you should be measuring the results. You should think about observability before the release instead of later. Then, at that point is where you are ready to distribute your API, make it available to your audience.

It’s really important to understand that distribution is not just putting your API out there, as like it’s now available on your site. It’s actually a process that is handled across multiple teams, such as sales and marketing, developer relations being a big factor of it. In terms of API experience components that we’ll be talking about soon, in addition to documentation, there’s also video content, or blogs, and things like that, that go into making your APIs available, and known, and discoverable to your customers. Once your APIs are available and your consumers can actually discover them, then that’s where the consumer lifecycle begins, which is where they go from discover to evaluate. As a customer, you discover a set of APIs, let’s say you’re thinking about payment APIs, and you want to integrate with some payment APIs. You discover that there’s quite a few of them in the market. Then you come to evaluation phase where you try to explore. Your developers are going to maybe do prototyping with a few of those, and then try to identify which one’s right for you. That is the evaluation phase. As you go from evaluation, once you have identified that, ok, these are the set of APIs that’s right for me, right for my application and my use case, then you get into integration. During the integration phase is where your development happens. This is generally followed by testing. You want to make sure integration works. You go through use cases, different scenarios that your customers are going to walk through. For example, in case of payments, it could be that you process a payment, and then your customers end up asking for a refund. What happens? How does that integration work? In the scenarios where the customer does a stop payment, what happens? You try to test all these scenarios. Then you deploy.

Once you deploy, you want to make sure you’re observing your integration. In case of payments, for example, or payment integration, you want to make sure all your payments are going through. Throughout this experience, you want to make sure that you get the support you need from the producer of those APIs that you’re integrating with. That there is a developer community that has information you need in case you get stuck, and there is enough resources for you to debug your integration if you need to. In case you get stuck, you’ll be able to reach out and provide feedback to the company that’s producing the APIs. It’s very important that that feedback loop exists. From the producer standpoint, it’s very important that that feedback loop is designed in a way that helps the customer in a way that is time sensitive. Also, make sure that that feedback across many customers is available as an input to the next iteration of your APIs, so that you are able to empathize with the customers and your APIs will evolve over time to better address customer needs.

Stakeholders Across API Producer Lifecycle

One of the very fascinating things that I find about APIs is how it has evolved from engineering teams, and engineers have led the development of APIs for a long time. Most of the times, a majority of APIs tend to be internal and not public facing. At the same time, now that a lot of companies are starting to think about publishing APIs, monetizing APIs through public facing APIs or partner specific, customer specific APIs, the need for thinking of APIs as products has really come to the front. It’s become really important that we think about various aspects of what that means. One of the important things that really impacts a product is to make sure that the stakeholders across the product lifecycle, in this case, API lifecycle are identified, and that they are able to collaborate in effective production of the product. This is where the role of product comes in in terms of API development.

In this diagram, the blue items in the API producer lifecycle are business functions. When we traditionally did not have a lot of product management in terms of building APIs, now it’s wrapped in more business focused initiatives and thought process. In terms of when we think about APIs, just jumping into developing something, which is very developer-first way of developing APIs, we have to take a step back and approach it with a more design led way of developing APIs. That’s where you start with a product manager leading the definition and requirements of the API that needs to be built. You bring in software architects, and designers, and PMs, who collaborate on the design of the API. Then the actual development begins. This also ensures that in a large organization, you have done the due diligence of making sure you don’t have redundant APIs, that your APIs meet a certain quality criteria, and you have thought about the prerequisites of what is needed, and you’re being more mindful of your developer bandwidth and not just building things that you have to later deprecate. There’s more thought put in upfront before the APIs are actually built.

This is also an opportunity to shift security and testing left where you can think about what will the security needs be for the API. I recommend bringing in security teams as early as the design phase to actually get your feedback on, what are the criteria that a particular API needs to meet in terms of being more secure for your end users? Since your consumers are going to be very focused on security, and security is a very top of mind aspect of how customers evaluate different APIs to integrate with in applications. When we think about development, you have the developers for testing, you have test engineers. For security, you have information security teams. Deployment is generally a different team of release management. Then you come to observability, where you bring in the SRE, DevOps, your platform teams. You could probably even bring in data analysts to help you build dashboards, depending on what metrics you want to establish for measuring your APIs. When you start to think about go-to market, you need to start involving your product marketing manager, who will drive the distribution of your APIs. As you can see, these are different teams that are involved, who are owners of different steps of the API producer lifecycle. As your APIs go through different stages of the lifecycle, all these different owners have to work together and provide their input, set the standards. Generally, the product manager has to be the one who owns the progress of the API across all of these steps. Although they are the originator and they’re part of the definition, they are the ones coordinating and collaborating with all these owners to get the APIs to the finish line.

This is also the same framework you can apply to API governance, because governance is applicable at every step of the producer lifecycle. Because you need certain standards that an API must meet to be considered ready for the next phase. You need a certain level of quality, of definition, to actually jump into design. You need a certain level of design to be able to start development. Owners of each of these steps are the people who should be setting those standards. There should be some SLAs that you establish for those owners to approve and sign off on those steps. If the design doesn’t meet the security standard, for example, then it needs to be worked on further before development can begin, for example. You need to establish API governance with the lens of API producer lifecycle, so that you are making sure that you’re building high quality APIs for your end customers. Another thing that we should be thinking about is supportability. Supportability is another aspect of like, when you start to think about distributing your APIs, do your customers have a way of getting support from your team? How do you distribute? We will be looking at some distribution models.

Creating a User Research Strategy

First, let’s begin with trying to understand the customer journey and develop some customer empathy. The first thing that we need to do when we think about building any product is user research. Let’s say you’re building a new set of APIs, then you need to identify who your customers are, who is your target audience. There are a lot of different approaches you can take to understand your customers. Some of the most straightforward ways is to start thinking about customers in terms of customer segments. For example, are you thinking of making an API that’s for individual developers building applications, or are you thinking about small to medium businesses? You could be thinking about targeting enterprise audience. You could segment in terms of industries. Is your API specifically for FinTech payments, or is it more for healthcare? You can do user research to identify the right audience, and understand what are the use cases that you’re trying to serve, way before you actually start building your API. This user research although should be ongoing even after you build your APIs, but you should really think about what are the different use cases you’re trying to address. What is the customer segment you’re trying to address? Once you do that, you can go deeper to understand your customers better.

Once you’ve identified that, for example, I’m going to build these APIs to address an SMB market with payment APIs. Then, you try to understand that SMB market, who needs payment APIs a little better, by trying to understand maybe using customer interviews, surveys, and various other aspects to understand, for example, what do developer teams look like in SMBs? How big are they? Do they usually have three or four developers? What skills do they have? Do they mostly write Python, or JavaScript? What kind of tools they use. What does that tech stack look like? For example, a very large team can probably build an integration really quickly. Versus if your customer is a startup and the CEO is doing some coding, then tools like a lot of copy paste code might help them better. Understanding the team structure, the skills on the team, the tech stack, things like that, as you go deeper into the target audience will help you decide what tools you can provide your customers to help them integrate better with your APIs.

This leads to also a very important decision is, are your APIs going to be public, private, or internal? Internal being, it’s only available to your internal developers within your organization. If that’s the case, then your total addressable market is the size of your organization, the number of developers you have. If it is private APIs or limited to certain customers, then that is your TAM, that is your total addressable market. If it is going to be a public API, then it is going to be based on the target segment that is the potential size of your audience, which is your total addressable market. That would help you understand how big your audience is, how much potential use your APIs are going to get, which will help your design team and developer team gauge the scale that you would need for those APIs. It would also help you manage APIs in terms of what kind of experience you need to build around those APIs.

Customer Journey Map

Another important aspect once you have your customers identified, let’s say you have published your set of APIs, this is where you start to build a framework to map your customer developer journey. In terms of your consumer developer journey, the first thing that we talked about a couple of slides ago, is discovery. This is where your customer is asking questions like, how do I find your APIs? Discovery is very important, because how do you publish your APIs? Is it just on your site? Is it available on other tools? For example, on Postman’s public network, do you have a collection that allows people to discover your APIs? Are there blogs, YouTube videos that you have published about your APIs that allows people to discover them? There are different customer touchpoints that help the customer get to the next step in their journey. In terms of discovery, the customer is asking questions like, how do I find your APIs? You address it with touchpoints such as your landing pages that you SEO optimize, your newsletters, case studies, white papers, along with social media content. Then, when the customer goes into evaluation phase where they have looked at a couple of different API solutions out there, maybe your competitor products, this is where they start to look deeper into your documentation. They want to look at your FAQs, pricing pages, because pricing is a pretty important aspect of anybody’s decision-making in terms of if they want to go with the product or not. One of the interesting things about APIs is also how important the developer community is. Because a lot of times, developers, when they’re evaluating APIs, they would look at how strong the developer community is around those APIs. Are there developers who are already integrated with these APIs? If I had a question, is that something people have already run into and described? Are there enough developer generated content out there that can help me? For example, on Stack Overflow, people asking questions. That really helps developers think that, yes, these are tried and tested APIs, and really plays an important role in their evaluation process.

The other thing is also understanding technical dependencies. Making sure your documentation outlines the technical dependencies, or limitations of your APIs. It really helps consumers go from evaluation to actually deciding to go with your APIs and start building their integration. This is where your consumers start to actually do the development work and will probably start reaching out to support. Interestingly, if your support teams are not able to handle these support requests, or they’re not equipped, then this is where you would see a drop-off. Because if as a customer, I run into an issue that does not get resolved, then I cannot move forward. It’s an important aspect of conversion for your customer funnel, if you look at it as a funnel. Once the integration is complete, if your consumer and developers have received enough code samples, tutorials, they have learning resources and ability to get support from your support teams, or within the developer community, then they can get into testing, where they might need tools like testing tools or sandbox offerings that you can build either in-house or use third-party tooling. Then, they can go into deploying their application, at which point, you would start to see the spike in usage. This is where you start to see in your data, some API calls going up, and customers coming online and starting to make API calls as they grow and scale.

Customers are also very interested in observability. That is where as a producer, you can publish things like status pages where you list your recent incidences, or you establish SLAs and you publish change logs to make sure that customers are aware of any changes happening to your APIs. You can also create some ambassador programs where customers can participate and engage better with your teams. At every step of this journey, you should establish feedback loops and also measure customer sentiment. It’s very important to understand that every touchpoint is an opportunity for you to improve customer experience. If you provide better support in different aspects, you can get more customers through the journey smoother, faster, and more effectively. That will drive business value for your APIs.

API Maturity Based Distribution Model

Next, I want to share with you a distribution model. As you start to think about building your APIs and publishing them, this framework will help you think through how you publish them and how you think about distribution. Going through, on the left is the API development lifecycle that we talked about, where you go from define, design, develop, test, secure, deploy, observe, distribute. Once you do that, now in the distribution phase, the questions you should be asking is, are these APIs supposed to be for internal release only? Are they only for your internal developers? Then, once your team has developed them, you release it to your internal audience. Hopefully, you can do that with limited set of teams. Let’s say you want to publish it to 20 teams across your company, then you do it one by one, to first publish it to maybe one team, get some feedback, make some iterations before you publish it to all 20. Then scale them little by little, so that you’re improving the experience over time. Then, when it is ready, and all teams that you were targeting are ready to use the API, then you’re done. Let’s say your publishing partner APIs are specifically built for some specific customers, then you still release these APIs as internal APIs. You get some internal teams to test it out. A great way to get early testing done is to use internal audience like support teams, or your sales teams as power users, because they are much more aware of the customer needs than your developer team. Making sure that they can test out these APIs for you, maybe create a hackathon. Get them tested as internal release before promoting them to a private beta for a partner API, where you get some external customers to start testing them in beta. Then based on feedback, you can iterate until you feel confident that they’re ready to promote to being published partner APIs.

If your APIs are supposed to be targeted for public use, then you go through the same cycle, but first you test it out with internal audience, then you test it out with limited set of customers. That could be a private beta. Once you’ve done the private beta, you can publish a public beta, where you invite or make your product available for external customers, but with the type of beta so that they understand that the APIs might be subject to change, to a point where you feel ready that your APIs are ready to be promoted to general availability. What this does is you are iteratively promoting your APIs through a maturity-based distribution model, so that customers at all times are very aware of what to expect from the APIs. The reason that I recommend you go through this framework of promotion in terms of a distribution model is because APIs are inherently designed for your consumers to build their applications on. This process builds the opportunity for you to learn from the customer feedback, iterate on your APIs before they have built dependency on your APIs. Because once your customers have developed with your APIs, you can’t really change them. It’s very important to build that time that allows you to iterate without impacting customer applications, so that they have transparency into understanding how mature your APIs are, and if they’re ready to be integrated into production applications or not, if they’re experimental. Because general availability is something they would expect that your APIs are going to be stable, meet a certain SLA, and not going to change. It’s very important that you be transparent with your customers, whoever, be it internal, partner, or general audience.

The other thing, also in terms of your stakeholders, like the sales, marketing, developer relations team, they should also be aware of how mature an API is, so that they can prioritize what efforts they are pulling in to actually improve customer discoverability of those APIs. For example, if it is a really awaited feature that customers are waiting for and really excited about, and it’s launched in beta, that might have a much higher set of maybe code samples, or blogs, or something that you publish, and get people on the beta experience. Maybe you want to put in more of your efforts on general availability products, trying to get more content out there for them to help customers who are already evaluating your APIs and integrating with them.

Components of API Experience

Once you have your producer lifecycle, we’ve looked at distribution. The other end is to think about customer experience, which is delivered through things like documentation, API reference, code samples, blogs, videos, and so on. All of these that you see on the left, the first orange box in this diagram should be mapped to maturity. Customers know that if it’s a beta product, it might have, for example, a little less extensive documentation, maybe fewer video blogs, and things like that. If it’s a general availability product, they can expect a certain level of documentation, and stability, and scalability of the product. Making sure that your developer experience components are mapped to an API maturity lifecycle, so that your support teams can prioritize the support tickets coming in from all these different API experience components, and establish SLAs for support, is very important in terms of effective API related operations. Your support operations need to be able to prioritize, establish, and meet those SLAs. Your product team should be part of that process in a way that you’re aware of any support escalations that they have reviewed on a regular basis, so that the role of the product team, or the product manager would be to identify when escalations happen. That, is this a feature gap, or is this a bug, and provide the necessary solution. Let’s say if it’s a feature gap, then is it something that’s on the roadmap? Is it not on the roadmap? Or if it’s a bug, when can it be fixed? Then making sure that in your roadmap, and as it happens, it again goes through the iteration of releases and distribution that we just talked about. All the developer experience components are updated with the new functionality and the new behavior of your APIs, so that that feedback loop of getting information from support back into the product and improve your product iteratively continues.

Creating Effective Feedback Loops

As you think about feedback loops, support team is a product manager’s best friend. It’s very important that we think about how our APIs are serving our customers best. Support teams, since they spend so much time talking to customers are our best way to get that insight. We can get both quantitative and qualitative data from this. You can run customer satisfaction surveys, and get CSAT surveys to give you insights in terms of how customers are feeling. You can run CSAT scores to understand maybe every quarter, or you can segment your population of customers depending on how big your audience is, to have more insight into overall customer experience and how it’s impacting your customers. You can also set metrics like ticket volume to active customer count, because, for example, if your APIs are being used by 100 customers, actively 100 active customers, but then you’re getting 60 tickets, then you’re probably doing something wrong. It’s very important to understand that that ratio is very important. You want to reduce that over time. You can also segment tickets to understand what kind of customers are generally getting issues. Is it like usually small teams or large teams or certain regions, perhaps. There might be some segmentation to understand what is the segment of population that generally runs into issues. You can also segment tickets to understand what issues your customers generally run into so that you can design a better roadmap from product perspective to help prioritize those particular type of issues.

Continuous Iteration Using Data

Ultimately, it’s very important from a product perspective to have metrics to measure every initiative that you have. For APIs, I like to think about metrics in terms of infrastructure metrics, such as uptime, requests per minute, average latency, errors per minute, because that’s your fundamental set of metrics that define how your API works. If your API doesn’t work, then there is no point of measuring anything else in the experience of product metrics. The founding block of your metric should always be the API in terms of the infrastructure itself. Then you have the experience layer on top, which is where you start to measure things like views, session duration, support volume by channel. This is just a small subset of examples. You can build a lot of different metrics across the user journey, do think about acquisition, retention, engagement, and overall experience. You start to measure different aspects of the customer journey to establish how well your API experience is, and also to identify opportunities for improving your experience.

In terms of product metrics, which is built on top of the infrastructure and API experience metrics, you start to think about things like API usage growth, your unique API consumers. It’s very common, where you have a handful of customers driving majority of the usage. In terms of APIs and API products, it’s not necessarily that you have a lot of scale to be successful using a set of APIs. A customer can potentially have just 50 API calls a week and still be very successful and very satisfied with your APIs. Versus a customer can have 5 million weekly API calls and still be very dissatisfied with your APIs. The spectrum is fairly large, and is not necessarily correlated with scale. It’s very important to understand who are the API users, who are the consumers who are driving majority of your usage. Another very important metric is Time to First Hello World, where how quickly can a developer discover and start using your APIs because that tends to be the first step. If that one is full of friction, and customers can never get started with your API, so it’s very important that you make that process the most frictionless so that customers can really check out your APIs quickly, and get started. That’s where it would really help them to get into evaluation and start building with your APIs.

Conclusion

I hope that the frameworks I provided help you build successful APIs and become successful as API product managers. A lot of this information is also available in my upcoming book called, “API Analytics for Product Managers” where I go a little deeper into the analytics and how to measure the success of your APIs.

See more presentations with transcripts