How Google built generative AI tools for the Chrome browser
From an engineering perspective, implementing LLM technology into Chrome was a challenge. “It’s a new skill set,” Adriana says. “We had to learn not only how this technology works but also how to turn it into a product people can use. Traditional browser features work the same way every time you run them. If a feature has the same input, it will give the same output.” When Adriana and her team write code for a new Chrome feature, they also write tests to check it works as expected. “If it passes the tests, you have confidence it works,” she says.
With features that use generative AI, it’s not so simple. Large language models recognize and generate text or images, and they need to be able to adapt to many kinds of user input. “We take the foundation model and we teach it what we want it to do for our example use cases, and then we evaluate how it works against many different types of user scenarios,” Adriana says. Determining whether it’s working requires deeper human evaluation. “It’s not a simple binary of ‘it runs’ or ‘it doesn’t run,'” Adriana says. “We’re looking at it and thinking, ‘Is the tone right? Is this length OK? Is this the level of specificity we’re looking for?’ It’s a very different process.”
One training scenario Adriana thought was particularly interesting was how the AI tab organizer uses emoji. “I really love how people use emoji to label tab groups,” she says. “Seeing the emoji makes it easier to know the topic of that tab group when you’re scanning your tabs.” The Chrome team wanted the new auto-tab organizer to have an emoji option for users, but they also didn’t want it to potentially pick inappropriate options. For example, If you’re planning a celebration of life, Adriana explains, they don’t want Chrome to show you a skull and crossbones. So, with help from Google’s emoji team, they decided to map out what kinds of tab group categories were safe for broad use. “Travel, animals, places, nature — these kinds of things are great use cases for emoji, so we know the auto-tab organizer has a good chance of getting it right by only drawing from those categories,” she says.
The Chrome team also wanted to make sure that people could use the new AI features without needing to understand how the underlying technology works. So they designed Help me write to gather context from the webpage you’re on and take it from there. “It can see you want to write a restaurant review and adjust for that versus helping you fill out a form or sell something,” Adriana says. Similarly, when creating the AI themes tool, they originally thought users could write their own prompts to populate the visual themes. “We realized it was actually kind of difficult to come up with a prompt for this,” Adriana says. Instead, they went with a drop-down approach where you choose a subject — like the Aurora borealis or rainbows — and then can use other drop downs to add styling details and select a color scheme. “We want people to be able to customize it but also give narrower options that get good results,” Adriana says.
