Booking.com and Flight Centre on measuring generative AI’s efforts to make travel better

From the outset of Tuesday’s opening Center Stage presentation at the 2024 Phocuswright Europe conference, it became clear that this year’s discussions about generative artificial intelligence would be far different from what conference attendees heard last year.

Instead of speculating on what generative AI could or might do, the panelists spoke of key performance indicators. Instead of reassuring audience members AI wasn’t coming for their jobs, they spoke of demonstrated productivity gains that came from employing the technology.

While it’s something of an educational process to get a company to buy into the idea of embracing generative AI, it’s ultimately worthwhile, agreed panelists Adrian Lopez, the corporate head of AI at Flight Centre Travel Group, and Rob Francis, senior vice president and chief technology officer at Booking.com.

Lopez shared a story of running workshops with different teams within the company.

“We say, ‘OK, tell us about your day-to-day problems.’ And they end up so happy because at the beginning, they’re like, ‘I thought you were coming here to kill my job and replace me with AI. This is actually going to make my day much better because I’m not going to waste two hours doing these really low-value things I’ve done and I can focus on doing the better part of it,’” Lopez said.

Francis said his company has focused on training employees how to think about where generative AI could be helpful in their space.

“We have found over time that the closer you put the problem into the hands of people who deal with the problem and democratize access to GenAI, the more likely something will percolate,” he said, later adding, “It’s a mind shift that you have to really incentivize and encourage.”

The incentives and encouragement can be self-fulfilling when employees recognize the time savings, the panelists said. When moderator and PhocusWire editor in chief Mitra Sorrells asked about KPIs and other metrics of AI’s success, Lopez said internal tools are easier to measure.

“Because we’re starting to give these tools to different teams, we can actually measure by doing this task with AI we’re saving 20 minutes per day, and that’s an easy number to keep track of savings,” he said.

Francis agreed. “Code generation is one that lots of people are using in the software development world,” he said. “I’m going after, literally, the number of engineering hours saved through that. So there are some cases where you can do that very explicitly.”

It’s not so simple with external-facing tools, he conceded.

Because we’re starting to give these tools to different teams, we can actually measure by doing this task with AI we’re saving 20 minutes per day, and that’s an easy number to keep track of savings.

Adrian Lopez – Flight Centre Travel Group

“When it comes to customer experiences and what we can do for our partners and our travelers, we’re more in a learning mode than trying to hold ourselves and constrain ourselves to bottom-line contributions.”

A good example is the AI trip planner that Booking.com launched last year in the United States. The results have been encouraging enough that Francis said plans are in place to expand it to other countries, though he preferred not to specify where or provide a timeline.

“We’re still very much in a test and learning mode with it,” he said. “We spend a lot of time on what we call intent optimization. The better we understand the actual intent of our customer, the better we can get them straight to what we’re looking for in terms of their desire to travel.”

That’s where large language models do really well, he added.

“We’re having a conversation, a chat … and you can do things that you cannot do in a traditional user experience — you know, filter, filter, filter,” he said. “This lets you get much more specific about what someone is looking for.”

He offered an example — presumably not firsthand experience — of a romantic getaway to Las Vegas that’s “cheesy.”

“There’s not a ‘cheesy’ filter,” he reassured the amused audience. But generative AI can get the idea from its chat with the user more quickly than a traditional search.

“If you’re looking for cheesy, maybe it’s a heart-shaped bed. Maybe it’s staying at the Elvis Chapel, those things,” Francis said. “Now you have an understanding that can be fed [into the system].”

The bottom line for tools like an AI trip planner is learning what customers enjoy, Francis said.    

“For us, success is: Does the customer rate the quality of the conversation and feel like they got what they’re look for?” he said. “Right now, we’re still just trying to optimize that and make that the richest experience. It’s a long game for that.”

A long game, the panelists agreed, filled with lots of intermediate changes — sometimes coming so fast they can be hard to track. Near the session’s close, Sorrells asked the panelists if the pace of change in their field was exhilarating — or exhausting. Both leaned toward the former.

“I really like this. It’s not exhausting,” Lopez said. “There’s a lot of news. … But at the same time we are living now in the moment that I’m watching, like a couple of weeks ago, the keynote from OpenAI. Ten minutes later I’m using the new model.”

And while that compels some additional homework and a new round of testing, it’s ultimately worth it, he said.

“It’s not like we’re done now [and] I need to throw away everything and start again from scratch. It’s the opposite,” he said. “It’s like, ‘OK, my system now is better. Faster. Smarter. Or it’s cheaper.’ So it’s really exciting, but yeah, you need to spend a lot of hours just keeping up with everything.”

Watch the full discussion in the video below.