Lessons from Gmail’s Failed Integration with Google+
We are happy to share this excerpt from ‘Evidence-Guided: Creating High-Impact Products in the Face of Uncertainty,’ a book that provides invaluable insights into the world of product management and development.
This chapter is particularly relevant to the forex and crypto industry, as it offers a deep dive into the challenges faced by one of the tech giants, Google, in its quest to innovate and stay ahead in a rapidly changing market. The experiences shared in this chapter resonate with the volatile and dynamic nature of the forex and crypto markets, where evidence-based decision-making is crucial.
By exploring the integration of Gmail with Google+, the chapter sheds light on the importance of adapting to market trends, understanding customer needs, and avoiding common cognitive biases in product development. We believe this excerpt will provide valuable lessons and strategies that can be applied to navigate the complexities of our industry.
Itamar Gilad
Itamar Gilad is an internationally acclaimed author, speaker, and coach. He held senior product management and engineering roles for two decades at Google, Microsoft, and a number of startups. Since 2017, Itamar has been teaching and coaching evidence-guided product development to hundreds of companies and thousands of product people. He is the creator of the GIST model, the Confidence Meter, and other widely-used frameworks and tools. Itamar is a regular speaker at industry events, and he publishes a popular product management newsletter.
It was August 2011, and I had just joined Gmail as a product
manager. As I reported for duty on my first day, my head was buzzing with ideas
for how I might make my mark on Google’s flagship email service, and, who
knows, maybe even reinvent email. But,
of course, it was not to be. Google had plans of its own for Gmail, and there
was already a big, strategic project waiting for me.
You’ve probably heard of Google+. Maybe you even used it at
one point. In the early 2010s, with the threat of Facebook looming over its
advertising business, Google decided to go all-in on Social. Google+ wasn’t
just a new social network, built from the ground up with all the bells and
whistles, it was an entirely new division within Google and the cornerstone of
its strategy.
Bradley Horowitz, the Vice President of products for Google+,
explained this strategy in a 2011 interview: “Google+ is Google itself. We’re
extending it across all that we do—search, ads, Chrome, Android, Maps,
YouTube—so that each of those services contributes to our understanding of who
you are.” (“Inside Google Plus | WIRED.” Sep. 27, 2011, Accessed Aug. 21, 2020).
And that’s where we, Gmail, came in. Our mission was to
connect Gmail and Google+ and make them feel as a part of one whole. This
seemed like a reasonable and important thing to do. Social networks were
growing at a pace never seen before, and people were spending hours each day
using them. Early Google+ was growing fast too.
By October 2011, just months
after its launch, it already had 40 million users. Three months later, it
officially had 90 million. These were small numbers compared to Facebook’s 800
million active users, but it was a promising start that made us quietly
optimistic.
So, off we went. We brainstormed, designed, and built
features that brought elements of Google+ into Gmail: you could share photos
from Gmail to Google+, see the latest posts from your friends in a side panel
next to your inbox, and filter messages by Circles (Google+ friend groups).
These were not necessarily cheap or easy projects to pull off. Some carried on
until late 2013. Other product ideas were put on hold to allow the Google+
projects to jump to the head of the queue.
This story ends in a sadly familiar way. A small group of
enthusiasts adopted the Google+ features in Gmail and loved them, but most
other users just disregarded our hard work, and some clearly disliked the mix
of social and personal. We persevered and continued to launch according to
plan, but the results we hoped for never came. Eventually, after years of low
usage, we discontinued all the Google+ features. Today, you won’t find them in
Gmail.
Google+ itself didn’t fare much better. While user numbers
grew strongly on paper, only a small subset of users remained truly active and
engaged. It seemed that most people just didn’t need another social network.
The Google+ team iterated and launched new features and redesigns at a fast
pace, but to no avail.
Google+ didn’t become an important part of most Google
users’ lives, Facebook continued to grow as before, as did Google’s ad
business. Eventually, after years of hard work, it became clear that Google’s
social strategy had failed and the project lost momentum. Parts of Google+ were
spun off into separate products, and the core social network was shut down in
2019.
But, the bad news didn’t end there. With the benefit of
hindsight we now know that by placing all its bets on Google+, Google had
missed a much larger opportunity presented by social mobile apps, such as
WhatsApp, Instagram, and Snapchat. These apps eventually became the threat to
Facebook that Google+ aspired to be, and they did it at a fraction of the cost
and effort.
With
Google+, Google followed a classic product development pattern, we picked a
promising idea, turned it into a plan, then switched to execution. Google+ was
a big, strategic idea (which made its failure all the more obvious), but I see
companies use this same plan-and-execute
approach for anything from minor product enhancements to major new features and
products. With plan-and-execute we have to start by picking which ideas to
invest in.
The decision usually involves some data, a customer request, what the
leading competitor is doing, the latest trend in the market, but is mostly based
on our own judgment. We rely on our
experience and logic to form opinions, and on consensus and rank to arrive at
decisions. Many companies repeat the plan-and-execute process at multiple
nested levels: strategy, roadmap, product, and feature, creating what I call a
Planning Waterfall.
Figure 1.1 Planning Waterfall
Human
judgment is extremely powerful, but it can also be very flawed. While for most
day-to-day decisions experience, logic, and consensus are perfectly adequate,
psychologists have found that we struggle with questions that entail
complexity, uncertainty, and too little or too much information (The
book Thinking Fast and Slow by
Nobel-winning psychologist Daniel Kahneman explains this phenomenon in detail).
For
example, when we debate a question like “which of these 12 ideas is the most
likely to achieve the goal?” we have to consider the users, the market, the
product, the technology, as well as the internal dynamics of our company. These
are all complex things, full of moving parts, and in constant change. Many
ideas, including ones that seem like sure bets, will fail to make an impact, or
produce unexpected side effects. Our brains, powerful as they are, are simply
not able to compute all the probabilities and arrive at a correct decision.
But
instead of telling us “I don’t know” our brains use a trick: they fall back to cognitive biases such as confirmation bias, risk aversion, the sunk cost fallacy, and hundreds more (For a full list of cognitive biases see https://en.wikipedia.org/wiki/List_of_cognitive_biases).
Our
cognitive biases can not only lead us to the wrong decision, but they can also
make us feel overly confident in this decision (and perhaps make us believe
it’s the only one possible). Debating ideas in groups or committees doesn’t fix
the problem, as groups introduce their own biases such as groupthink and
politics.
Trusting experienced and senior people to choose has been shown time
and again to be unreliable (the leadership team of Google had some of the
smartest, most successful executives in the industry). The result is plans that
are full of questionable decisions and bad ideas.
All of this may sound overly pessimistic to you. Product
companies are obviously capable of making good decisions; the results are all
around us. But, that may just be survivorship bias. We see the successes, but
below the surface there’s an awful lot of hidden failure.
If you analyze usage
in your product you’re likely to see that most features get little or no
traction with users (a 2019
research conducted by Pendo.io suggests that 80% of features are rarely or
never used). You
can remove them today and almost no one will notice. Product portfolios have a
similar power law: a few products generate all the traction and revenue, while
others contribute very little.
Research of thousands of A/B experiments across
multiple companies shows that at most one in three product ideas show
measurable improvement, (Kohavi,
Ron, Diane Tang, and Ya Xu. 2020. Trustworthy
Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge
University Press. Chapter 1) and
the average in the industry may be far lower. If these numbers are true for
you, then the majority of what’s currently in your roadmap and product backlog
isn’t worth doing; it will just create waste at high cost.
Some companies seem to defy the odds and produce hit after
hit over the course of years and decades. Google is one of those companies. For
me that was a key reason to join; frustrated with my own track record of
success I wanted to understand what made Google different. What I discovered
was that Google didn’t find a way to predict the future, but rather created a
system that acknowledged uncertainty and worked to improve the ratio of success
versus failure.
With Google+ Google obviously used a different playbook
(perhaps because of the existential threat that Facebook presented), but
historically the company always started with customer-focused goals (internal
principle: Start with the user and all
else will follow); it was willing to try out many ideas to address these
goals (Let a thousand flowers bloom),
and wasn’t shy about testing minimal, unpolished products with users (Think big, but start small).
Perhaps
most important, Google expected its product teams to act on the results, even
if that meant pivoting or shutting down the project (Fail fast). Underlying all of this was a strong emphasis on data,
both quantitative and qualitative.
Google is not unique in this approach. One of Amazon’s
principles is “We go where the data leads us.” Airbnb, Netflix, Microsoft,
Slack, Booking.com, Facebook/Meta, Apple, and many successful but lesser-known
companies, all have their own variation of this same theme. The processes these
companies employ are very different, but they have one thing in common: they
supercharge human judgment with evidence.
Evidence is facts and information that confirm or refute our
assumptions and theories (different from data,
which is simply facts and information, but not necessarily with any meaning).
Using evidence helps us break away from our internal perceptions and biases. It
forces us to confront reality and to update our picture of the world.
For this
reason evidence is a foundational element of science, medicine, and law;
disciplines where it’s recognized that decisions should not be made solely on
the basis of human judgment. Evidence-guided
development attempts to bring these same concepts to the creation of
products. It’s a key part of all modern development methodologies including
Lean Startup, Design Thinking, Product Discovery, and Growth.
While at Google I had a chance to experience evidence-guided
development firsthand (a full example is coming shortly). It was a real
revelation, a completely different way to develop products. In 2017, I left
Google and started consulting product organizations, eager to share what I
learned. But, I then realized there was yet a bigger problem.
Most of the people
I talked with already knew the concepts, but few were able to implement them in
their companies. Evidence-guided development goes against the grain of
long-held “best practices” and requires rethinking company power dynamics. It’s
hard to get managers and stakeholders to give up on release roadmaps, and it’s
equally hard to convince product teams that their job is to discover as well as
deliver.
At Google I observed that it’s useful to break
evidence-guided development into four areas of change: goal-setting, idea
selection, experimentation, and execution. To make it easier for my clients to
tackle these changes one at a time I created a four-layer model I called GIST: Goals, Ideas, Steps, and Tasks.
Figure 1.2: GIST model
Figure 1.2: An example of the GIST model at work. The team
is considering a total of seven ideas
across two goals. It is using steps to develop and test the ideas, and
each step breaks into tasks. Some
ideas fail during testing (marked with x in the diagram) and are dropped,
making room for work on other ideas.
Goals
define the outcomes we wish to achieve, measurable benefits for customers and
for the business. Ideas are
hypothetical ways to achieve the goals. The key word here is hypothetical. As
most ideas don’t work, we have to test multiple ideas and invest only in those
that yield supporting evidence. That’s the job of steps: short projects or activities that develop an idea somewhat
(often with no coding) and test it.
A
step may be as simple as creating a model on paper, or as complex as a beta
launch. Tasks are the work items that
go into doing the work, the things we manage using Scrum, Kanban, or other
methods. GIST doesn’t dictate how you manage tasks, but it strives to connect
the work to steps, ideas, and goals, allowing the team to work with full context
and deep focus on the customers and the business. The GIST framework applies to
small, medium, and large product ideas. It can be used in a ten-person startup
as well as in a 10,000-person enterprise.
GIST is not a radically new concept. I consider it a
meta-framework that combines tried-and-tested product methodologies with models
and tools I created along the way. Still, when I started sharing GIST the
reaction from the product community and from my clients was overwhelmingly
positive.
There seemed to be a strong need for a practical framework that “puts
it all together.” Many people were willing to give GIST a go, and the
information they shared helped me iterate, improve, and create the framework
described in this book. Still, it’s important not to consider GIST a fixed
process. It’s a model that helps bring evidence-guided thinking into your
development, but you should test it and adapt it to your context.
To make things less theoretical, let me tell you about the
first project in which I experienced the power of evidence-guided
development, my proto-GIST project at Gmail.
In late 2011, not long after we shipped the first wave of
Google+ integrations, I had an idea. I noticed that my personal inbox (Gmail,
of course) was full of promotions, social updates, and notifications that I
didn’t care to read, but had too little time or motivation to delete. In Gmail,
we called this situation inbox clutter.
Wouldn’t it be cool, I thought, if Gmail just magically organized my inbox,
without me having to do any work? I imagined the non-important email showing as
a digest in a side panel: “You also received 5 promotions, 12 social messages,
and 3 receipts.” Users would be able to click on the digest and read these less
important messages when the mood strikes them.
As is often the case, I fell in love with my own idea and
immediately wanted to push it forward for funding, planning, and execution. But, my managers and colleagues were not convinced, and they had their reasons. It
wasn’t clear that inbox clutter was indeed a widespread and severe problem
among casual users who on average received very little mail.
Gmail already had
multiple features to help manage your inbox, but none got a lot of usage.
Worse, my idea to push low-priority mail to the side could easily confuse and
annoy the users who were used to having their inbox work in a certain way ever
since Gmail launched in 2004.
My colleagues and managers were pushing back on my idea, but
implicitly they were asking me some very Googly questions: What are you trying
to achieve? Who is this for? What would be considered a success? In other
words, what is the goal?
At that time Gmail had a product-wide goal to become more
relevant and engaging in a world increasingly dominated by social networks and
instant messaging. I argued that solving inbox clutter is how we, the Gmail
Inbox team, could contribute to this broader goal. My team, skeptical as it
was, was willing to explore.
We started with research to understand inbox
clutter better. One of our backend developers ran a large data analysis to see
how consumer users process their inboxes. The data he brought back took us by
surprise. A huge portion of Gmail’s users did not manage their inbox in any
way. They just went through their emails and read selected messages, otherwise
leaving everything they received untouched.
What does such an inbox look like? We got the answer when we
interviewed casual Gmail users and asked them to show us their inboxes. We
realized that average statistics didn’t tell the whole story.
With the
proliferation of social networks and eCommerce, these people got substantial
amounts of promotion, social notifications, and transactional emails. Some had
tens of thousands of unread messages in their inbox. They had to work hard to
spot important new messages, and even harder to get back to those messages
later. Inbox clutter was a much bigger problem than we thought.
Armed with this information, we were able to formulate an
objective: help casual users interact only with the messages they want to
interact with. This entailed a few success metrics: being able to accurately
predict which messages users wish to read; ensuring that they would never miss
out on an important message, and most importantly, high levels of user
engagement and satisfaction with the new experience as measured by usage,
retention, and surveys. This goal served us in the months to come: we knew who we
were focusing on and what success looks like.
Now it was time to discuss ideas. We had a few: teaching
users how to clean their inbox themselves, grouping or reordering inbox
messages in various ways, and creating digests of the least important messages
(my initial idea). Each idea had its pros and cons, and different people had
their favorite. However, debating the ideas didn’t lead to any useful
conclusion. We knew we had to test, and we had to start with the riskiest part:
the user experience.
So we brought people into our usability lab to test some of
these ideas. In one of those tests, we asked 12 participants to use Gmail, but
with a difference, the inbox was split into tabs. The main tab contained mail
from friends and family, but there were other tabs for social notifications,
promotions, and transactions. When opening each tab, the participants found
their very own messages, taken from their inbox, sorted into the right place.
As if by magic, Gmail organized their inbox by mail category.
This part of the study was a complete sham of course. The
thing the participants saw wasn’t Gmail, but a thin facade of HTML that our
user experience designers cooked up. You could do little with it except look at
your inbox and click into the tabs. Message categorization was another bit of
sleight of hand.
With the participants’ permission, and while they were being
interviewed, a few of us sat in a separate room and copied the first 50
messages from their inbox, sender and subject only, into the appropriate tab,
based on our own best guesses. It was a bit of a race against time, but it
worked. The participants experienced the Tabbed Inbox (almost) as if it was a
real thing.
The results? All the participants immediately understood
what was going on and could explain why the messages were placed where they
were. Using the tabs was natural and easy. Best of all, 10 of the 12
participants absolutely loved this new inbox. In fact, many of them asked how
soon they could have it. The smiles on their faces told a clear story, this was
something they needed, but never knew they could have.
The other two
participants already had their inbox under control using other Gmail functions,
so they didn’t find the new inbox appealing. Interestingly this same ratio of
10–15 percent of people who already managed their personal inboxes well and
thought that the Tabbed Inbox was unnecessary, repeated throughout our research
over the next 14 months. Unfortunately, this minority included many of my
colleagues at Gmail, and as we discovered later, most of the tech press.
Given these strong results we decided to double-down on the
Tabbed Inbox idea and leave other ideas as fallbacks. Still, we didn’t go
all-in. Our initial research gave us sufficient evidence just to fund a small
project, a rudimentary, functioning version of the inbox with a bare-bones user
interface and simplistic classification. We used this early version to
self-test on our own personal inboxes (in Gmail this is called Fishfood, team testing).
I remember
having to convince my team members to stop diligently organizing their inboxes
and to let email pile up, just like ordinary users. It was an adjustment for
some, but we quickly realized that even with this basic version of the inbox,
the experience was quite transformative.
Throughout the
project, we kept progressing in stepwise increments, build a more-evolved
version of the inbox, then test it. Along the way, we enlisted the help of
thousands of Googlers who valiantly agreed to use an interim version of the
inbox in their own personal accounts (this process is commonly called Dogfooding).
We exposed our evolving
email classification system to a small subset of Gmail users through a lab (an
optional feature you can enable in Settings) where they could report
classification mistakes. This gave us much-needed data to improve our
algorithms. We continued to perform regular external user studies as well, some
involving hundreds of participants using the new inbox over periods of weeks.
The project definitely didn’t go according to any plan. We
had to redesign the user interface multiple times, and getting the
classification right required assembling a small team of data and
machine-learning specialists.
We also learned that launching the new inbox on
the desktop version of Gmail first, per the plan, was not enough, inbox clutter
was an even bigger problem on mobile where it was causing excessive
notifications. So we expanded the project to include the Android and iOS Gmail
apps. I didn’t have to pitch very hard to get these other teams to join the
project; the evidence we collected did the talking for me.
The new inbox was launched in May 2013 to generally lukewarm
reviews from the tech press, but we didn’t really care. By then, we had tested
the new feature with thousands of users and were fairly certain we had found a
winner. Still, the testing continued. We gradually switched more and more
accounts into the new experience (programatically skipping those 10–15 percent
who looked like they had things under control) and tracked their behavior,
satisfaction levels, and classification error rates.
The rollout was uneventful, with very little backlash and
tons of positive feedback. People loved the fact that social and marketing
emails no longer competed with emails from friends and family, and never buzzed
their phones. It was easy to switch off the new feature, but few users did, and
our data confirmed they were using the inbox as we expected.
The Tabbed Inbox
won a number of innovation awards inside Google, and was generally recognized
as one of the most important improvements to the product. It was a fun project
to work on, and, more importantly, one that created high impact. Today, the
Tabbed Inbox is the first thing the majority of Gmail’s users see, and we have
strong evidence to show it’s giving them lots of value.
The story of the Tabbed Inbox brings together many of the
elements of the GIST model. Let’s look at it again from the perspective of
goals, ideas, steps, and tasks.
Goals
I tried to kick off the project around an idea. Had I
succeeded, the implicit goal would have been “let’s launch Itamar’s new inbox,”
which would have been great for my ego, but not so much for the end results.
Such goals, often called output goals, are very common in the industry
and very harmful. When most ideas yield little or no value, betting on an
unproven idea is likely to produce waste.
Luckily, in this case, I was forced to
backtrack and form an outcome goal centered on measurably improving the inbox experience of casual Gmail users.
This goal helped focus us on the destination rather than on the way.
In Chapter 2, Goals,
I’ll show you how to lead your area of responsibility, whether it’s a company,
business unit, group, or team, with a concise set of outcome goals. We will see
how to pick goals using the North Star Metric and metrics trees and express
them in the popular format of Objectives and Key Results (OKR). Then we’ll talk
about how leaders can steer without deciding what to work on, and how to tackle
the tough challenge of aligning the goals top-down, bottom-up, and across the
organization.
Ideas
Once we, the Gmail Inbox team, had a goal in place, we were
ready to discuss ideas. But, instead of trying to pick the winning idea through
brainstorms, debate, and reviews, we did three common-sense things that are not
at all common: 1) we based our ideas on research rather than opinions, 2) we
evaluated which ideas seem most promising given the data we had, and 3) we went
on to test multiple ideas.
This process is at the heart of GIST. In Chapter 3, Ideas,
I’ll show you how to collect ideas in idea banks, and how to evaluate and
compare them in a fast, objective, and unbiased way. A key component of this
evaluation is Confidence, which
measures the strength of the evidence in support of an idea.
I’ll teach you how
to use the Confidence Meter (see Figure 1.3), a tool I created specifically for
this purpose and that today is used by numerous companies, including Google.
We’ll see how to cut prioritization debates short and how to leave politics and
pressure tactics out of the discussion.
Figure 1.3: The Confidence Meter
What about Opportunities?
Many people find it
helpful to connect ideas to what is now known as opportunities (formerly called customer
problems, or underserved user needs).
If that describes your approach, you can certainly use it with GIST.
If you
uncover an important user need you can turn addressing it into a goal (as we
did for Inbox Clutter in the Tabbed Inbox project) or you can use it as a
filter for the type of ideas you’d like to pursue. You can also keep a list or
a tree (as
described by Teresa Torres in her book Continuous
Discovery Habits) of opportunities alongside the artifacts of GIST.
In Chapter 6 we will talk about strategic
opportunities, which is a related but different concept to do with strategy
formation.
Steps
In the Tabbed Inbox project we were lucky to land on a
strong idea relatively early in our product discovery. But, many initially
promising ideas turn out to be duds, and even the good ones need iteration and
refinement. For this reason we built the Tabbed Inbox through a series of
build-measure-learn loops, which are called Steps
in GIST. There were usability tests, longitudinal studies, fishfood and
dogfood, labs, and partial launches.
Each step generated a more complete
version of the product, but also gave us new evidence, which helped build our
confidence in the idea and get more funding. We used what we learned in each
step to course-correct and improve the idea. The feature we eventually launched
was profoundly better than the one we started with.
Many product organizations I meet are familiar with this
important concept, which Product Management guru Marty Cagan has aptly named, Product Discovery, but struggle to
implement it, they build too much and test too little; even when they do test
they fail to learn and take meaningful action.
In Chapter 4, Steps, I’ll walk
you through a full example of how to take a high-potential idea, and get it
through a series of steps to a clear Yes/No decision that no person, no matter
how opinionated, will dispute. This process isn’t meant just for big ideas;
cheap ideas entail risks too and should similarly be validated, although faster
and with fewer steps. In Chapter 4, we will review the wide gamut of validation
techniques at our disposal, from data analysis through smoke tests to A/B
experiments, and how to combine them into a development project.
Tasks
You might have noticed in my story how involved my product
team was in shaping the product. That is a norm in many good product companies,
but not a very common practice in the industry. There’s a double problem: the
managers don’t trust the team to make good decisions, and team members are
increasingly focused on delivery and seem disinterested in business and user
goals. Even in companies where these are not major issues, teams struggle to
reconcile product discovery with their agile development practices.
Working with strong people is one of the benefits of joining
Google, but that’s not the only reason my team was so engaged. Google was able
to create an environment that allowed rank-and-file employees to be in the
know, to contribute, and to make decisions. In Chapter 5, Tasks, we’ll talk
about how to do the same with your team, by involving them in all four layers
of the GIST stack, and by having them own discovery as well as delivery. I’ll
also share a tool, the GIST board, to help jointly manage the work of the team
across goals, ideas, and steps.
There’s nothing particularly revolutionary about GIST, and
it’s not the only evidence-guided model out there. Many good product companies
have been working in similar ways for years, although they probably don’t call
what they do GIST.
Still, I see far more companies that try to incorporate
evidence into their work but get stuck halfway between the old model and the
new. These companies adopt new processes, but get few benefits. Part of the
problem is overcoming entrenched mindsets and practices. In order to move
forward you need to change the way your managers, stakeholders, and team think
and behave.
The second half of the book is devoted to helping you adopt
evidence-guided development in your organization. In Chapter 6, The
Evidence-Guided Company, we’ll see how evidence-guided thinking can be deployed
across an entire company, and what role managers, stakeholders, and product
teams play in this brave new world.
We’ll also talk about the often-sticky
topics of product strategy, big projects, cross-team dependencies, and
roadmaps. In Chapter 7, Scaling GIST, I’ll show you how evidence-guided development
works in startups, in midsize companies, and in enterprises. Chapter 8, GIST
Patterns, talks about GIST in common types of products and business models:
B2B, B2C, physical products, marketplaces, and internal platforms and services.
Chapter 9, Adopting GIST, discusses ways to smooth the adoption curve and to
deal with common objections.
How big of a transformation you’re facing depends on where
you are today. GIST may not work in a very traditional organization where
decisions strictly flow top-down, and where the function of the product org is
merely “delivery.” However, if your company isn’t that extreme (most aren’t)
there’s a good chance you can adopt GIST, and it’s even likely that your
colleagues will be in support.
I regularly teach GIST to product leads,
engineers, executives, and business folk, and once the model is clear, the response
is almost always “we need this” and “let’s do it!”
At the end of the day, meaningful change is never easy or
comfortable; it takes visionary, motivated, and persistent people to push it
forward. If you’re a product manager tired of building things that nobody
needs, a manager frustrated with the rate of innovation in your organization,
or an engineer or designer that feels that most of what you create is a waste,
GIST was created to help you drive
the change in your own organization. It’s not going to be an overnight switch,
but I can guarantee you that it’ll be a worthwhile journey.
Let’s get started.
●
The classic approach of
plan-and-execute forces us to commit to a plan while facing high levels of
uncertainty.
●
Our brains and our decision
processes are not well suited to making decisions when confronted with
uncertainty. We rely on opinions, sparse data, consensus, and rank, backed by
cognitive biases, group dynamics, and politics.
●
The track record of success in the
industry is abysmal. Research indicates that only a minority of ideas create
measurable impact, and a high ratio of launched features and products get
little or no use.
●
Evidence-guided companies improve
the odds of success by supercharging human judgment with evidence. They
continuously evaluate and test ideas and update the plans based on what they
learn.
●
The principles of guiding our
decisions by evidence are not new, but they are hard for companies to adopt as
they go against traditional best practices and corporate power dynamics.
●
I created the GIST model, Goals,
Ideas, Steps, Tasks, to help break the change into four distinct areas:
goal-setting, idea evaluation, experimentation, and execution.
●
The Tabbed Inbox story, while not
perfect (no project is), is an example of the GIST model in action. Starting
with user/business-centered goals. Considering multiple ideas, iterating by
running the ideas through various steps, each producing a more complete version
of the feature and more supporting evidence. Optimizing the work for both
discovery and delivery.
●
The first chapters of the book are
devoted to explaining the GIST model in detail. Chapter 6 covers a full-company
example. Chapters 7 and 8 describe how to adapt the model to the size and type
of company you work in. And Chapter 9 explains how to tackle objections, and
smooth the adoption curve.
This
is an excerpt from the book. Available on Amazon, Apple Books, Kobo and other platforms. For details see: EvidenceGuided.com
Copyright © 2023 by Itamar Gilad
This chapter explored the lessons learned from the failed integration of Gmail with Google+, a project that aimed to create a unified social experience for Google users.
It shares how to apply the principles and practices of evidence-guided product development, such as:
– Defining and communicating a clear and compelling vision
– Aligning the product strategy with the company’s strategy and values
– Balancing innovation and iteration
– Using evidence and data to validate assumptions and decisions
– Avoiding common pitfalls and biases.
These lessons are relevant for any product that operates in a dynamic and competitive market, such as forex and CFDs brokers. By following these guidelines, forex and CFDs brokers can create high-impact products that solve real problems for real users, and avoid wasting time and money on features and projects that don’t deliver value or meet the goals.
We are happy to share this excerpt from ‘Evidence-Guided: Creating High-Impact Products in the Face of Uncertainty,’ a book that provides invaluable insights into the world of product management and development.
This chapter is particularly relevant to the forex and crypto industry, as it offers a deep dive into the challenges faced by one of the tech giants, Google, in its quest to innovate and stay ahead in a rapidly changing market. The experiences shared in this chapter resonate with the volatile and dynamic nature of the forex and crypto markets, where evidence-based decision-making is crucial.
By exploring the integration of Gmail with Google+, the chapter sheds light on the importance of adapting to market trends, understanding customer needs, and avoiding common cognitive biases in product development. We believe this excerpt will provide valuable lessons and strategies that can be applied to navigate the complexities of our industry.
Itamar Gilad
Itamar Gilad is an internationally acclaimed author, speaker, and coach. He held senior product management and engineering roles for two decades at Google, Microsoft, and a number of startups. Since 2017, Itamar has been teaching and coaching evidence-guided product development to hundreds of companies and thousands of product people. He is the creator of the GIST model, the Confidence Meter, and other widely-used frameworks and tools. Itamar is a regular speaker at industry events, and he publishes a popular product management newsletter.
It was August 2011, and I had just joined Gmail as a product
manager. As I reported for duty on my first day, my head was buzzing with ideas
for how I might make my mark on Google’s flagship email service, and, who
knows, maybe even reinvent email. But,
of course, it was not to be. Google had plans of its own for Gmail, and there
was already a big, strategic project waiting for me.
You’ve probably heard of Google+. Maybe you even used it at
one point. In the early 2010s, with the threat of Facebook looming over its
advertising business, Google decided to go all-in on Social. Google+ wasn’t
just a new social network, built from the ground up with all the bells and
whistles, it was an entirely new division within Google and the cornerstone of
its strategy.
Bradley Horowitz, the Vice President of products for Google+,
explained this strategy in a 2011 interview: “Google+ is Google itself. We’re
extending it across all that we do—search, ads, Chrome, Android, Maps,
YouTube—so that each of those services contributes to our understanding of who
you are.” (“Inside Google Plus | WIRED.” Sep. 27, 2011, Accessed Aug. 21, 2020).
And that’s where we, Gmail, came in. Our mission was to
connect Gmail and Google+ and make them feel as a part of one whole. This
seemed like a reasonable and important thing to do. Social networks were
growing at a pace never seen before, and people were spending hours each day
using them. Early Google+ was growing fast too.
By October 2011, just months
after its launch, it already had 40 million users. Three months later, it
officially had 90 million. These were small numbers compared to Facebook’s 800
million active users, but it was a promising start that made us quietly
optimistic.
So, off we went. We brainstormed, designed, and built
features that brought elements of Google+ into Gmail: you could share photos
from Gmail to Google+, see the latest posts from your friends in a side panel
next to your inbox, and filter messages by Circles (Google+ friend groups).
These were not necessarily cheap or easy projects to pull off. Some carried on
until late 2013. Other product ideas were put on hold to allow the Google+
projects to jump to the head of the queue.
This story ends in a sadly familiar way. A small group of
enthusiasts adopted the Google+ features in Gmail and loved them, but most
other users just disregarded our hard work, and some clearly disliked the mix
of social and personal. We persevered and continued to launch according to
plan, but the results we hoped for never came. Eventually, after years of low
usage, we discontinued all the Google+ features. Today, you won’t find them in
Gmail.
Google+ itself didn’t fare much better. While user numbers
grew strongly on paper, only a small subset of users remained truly active and
engaged. It seemed that most people just didn’t need another social network.
The Google+ team iterated and launched new features and redesigns at a fast
pace, but to no avail.
Google+ didn’t become an important part of most Google
users’ lives, Facebook continued to grow as before, as did Google’s ad
business. Eventually, after years of hard work, it became clear that Google’s
social strategy had failed and the project lost momentum. Parts of Google+ were
spun off into separate products, and the core social network was shut down in
2019.
But, the bad news didn’t end there. With the benefit of
hindsight we now know that by placing all its bets on Google+, Google had
missed a much larger opportunity presented by social mobile apps, such as
WhatsApp, Instagram, and Snapchat. These apps eventually became the threat to
Facebook that Google+ aspired to be, and they did it at a fraction of the cost
and effort.
With
Google+, Google followed a classic product development pattern, we picked a
promising idea, turned it into a plan, then switched to execution. Google+ was
a big, strategic idea (which made its failure all the more obvious), but I see
companies use this same plan-and-execute
approach for anything from minor product enhancements to major new features and
products. With plan-and-execute we have to start by picking which ideas to
invest in.
The decision usually involves some data, a customer request, what the
leading competitor is doing, the latest trend in the market, but is mostly based
on our own judgment. We rely on our
experience and logic to form opinions, and on consensus and rank to arrive at
decisions. Many companies repeat the plan-and-execute process at multiple
nested levels: strategy, roadmap, product, and feature, creating what I call a
Planning Waterfall.
Figure 1.1 Planning Waterfall
Human
judgment is extremely powerful, but it can also be very flawed. While for most
day-to-day decisions experience, logic, and consensus are perfectly adequate,
psychologists have found that we struggle with questions that entail
complexity, uncertainty, and too little or too much information (The
book Thinking Fast and Slow by
Nobel-winning psychologist Daniel Kahneman explains this phenomenon in detail).
For
example, when we debate a question like “which of these 12 ideas is the most
likely to achieve the goal?” we have to consider the users, the market, the
product, the technology, as well as the internal dynamics of our company. These
are all complex things, full of moving parts, and in constant change. Many
ideas, including ones that seem like sure bets, will fail to make an impact, or
produce unexpected side effects. Our brains, powerful as they are, are simply
not able to compute all the probabilities and arrive at a correct decision.
But
instead of telling us “I don’t know” our brains use a trick: they fall back to cognitive biases such as confirmation bias, risk aversion, the sunk cost fallacy, and hundreds more (For a full list of cognitive biases see https://en.wikipedia.org/wiki/List_of_cognitive_biases).
Our
cognitive biases can not only lead us to the wrong decision, but they can also
make us feel overly confident in this decision (and perhaps make us believe
it’s the only one possible). Debating ideas in groups or committees doesn’t fix
the problem, as groups introduce their own biases such as groupthink and
politics.
Trusting experienced and senior people to choose has been shown time
and again to be unreliable (the leadership team of Google had some of the
smartest, most successful executives in the industry). The result is plans that
are full of questionable decisions and bad ideas.
All of this may sound overly pessimistic to you. Product
companies are obviously capable of making good decisions; the results are all
around us. But, that may just be survivorship bias. We see the successes, but
below the surface there’s an awful lot of hidden failure.
If you analyze usage
in your product you’re likely to see that most features get little or no
traction with users (a 2019
research conducted by Pendo.io suggests that 80% of features are rarely or
never used). You
can remove them today and almost no one will notice. Product portfolios have a
similar power law: a few products generate all the traction and revenue, while
others contribute very little.
Research of thousands of A/B experiments across
multiple companies shows that at most one in three product ideas show
measurable improvement, (Kohavi,
Ron, Diane Tang, and Ya Xu. 2020. Trustworthy
Online Controlled Experiments: A Practical Guide to A/B Testing. Cambridge
University Press. Chapter 1) and
the average in the industry may be far lower. If these numbers are true for
you, then the majority of what’s currently in your roadmap and product backlog
isn’t worth doing; it will just create waste at high cost.
Some companies seem to defy the odds and produce hit after
hit over the course of years and decades. Google is one of those companies. For
me that was a key reason to join; frustrated with my own track record of
success I wanted to understand what made Google different. What I discovered
was that Google didn’t find a way to predict the future, but rather created a
system that acknowledged uncertainty and worked to improve the ratio of success
versus failure.
With Google+ Google obviously used a different playbook
(perhaps because of the existential threat that Facebook presented), but
historically the company always started with customer-focused goals (internal
principle: Start with the user and all
else will follow); it was willing to try out many ideas to address these
goals (Let a thousand flowers bloom),
and wasn’t shy about testing minimal, unpolished products with users (Think big, but start small).
Perhaps
most important, Google expected its product teams to act on the results, even
if that meant pivoting or shutting down the project (Fail fast). Underlying all of this was a strong emphasis on data,
both quantitative and qualitative.
Google is not unique in this approach. One of Amazon’s
principles is “We go where the data leads us.” Airbnb, Netflix, Microsoft,
Slack, Booking.com, Facebook/Meta, Apple, and many successful but lesser-known
companies, all have their own variation of this same theme. The processes these
companies employ are very different, but they have one thing in common: they
supercharge human judgment with evidence.
Evidence is facts and information that confirm or refute our
assumptions and theories (different from data,
which is simply facts and information, but not necessarily with any meaning).
Using evidence helps us break away from our internal perceptions and biases. It
forces us to confront reality and to update our picture of the world.
For this
reason evidence is a foundational element of science, medicine, and law;
disciplines where it’s recognized that decisions should not be made solely on
the basis of human judgment. Evidence-guided
development attempts to bring these same concepts to the creation of
products. It’s a key part of all modern development methodologies including
Lean Startup, Design Thinking, Product Discovery, and Growth.
While at Google I had a chance to experience evidence-guided
development firsthand (a full example is coming shortly). It was a real
revelation, a completely different way to develop products. In 2017, I left
Google and started consulting product organizations, eager to share what I
learned. But, I then realized there was yet a bigger problem.
Most of the people
I talked with already knew the concepts, but few were able to implement them in
their companies. Evidence-guided development goes against the grain of
long-held “best practices” and requires rethinking company power dynamics. It’s
hard to get managers and stakeholders to give up on release roadmaps, and it’s
equally hard to convince product teams that their job is to discover as well as
deliver.
At Google I observed that it’s useful to break
evidence-guided development into four areas of change: goal-setting, idea
selection, experimentation, and execution. To make it easier for my clients to
tackle these changes one at a time I created a four-layer model I called GIST: Goals, Ideas, Steps, and Tasks.
Figure 1.2: GIST model
Figure 1.2: An example of the GIST model at work. The team
is considering a total of seven ideas
across two goals. It is using steps to develop and test the ideas, and
each step breaks into tasks. Some
ideas fail during testing (marked with x in the diagram) and are dropped,
making room for work on other ideas.
Goals
define the outcomes we wish to achieve, measurable benefits for customers and
for the business. Ideas are
hypothetical ways to achieve the goals. The key word here is hypothetical. As
most ideas don’t work, we have to test multiple ideas and invest only in those
that yield supporting evidence. That’s the job of steps: short projects or activities that develop an idea somewhat
(often with no coding) and test it.
A
step may be as simple as creating a model on paper, or as complex as a beta
launch. Tasks are the work items that
go into doing the work, the things we manage using Scrum, Kanban, or other
methods. GIST doesn’t dictate how you manage tasks, but it strives to connect
the work to steps, ideas, and goals, allowing the team to work with full context
and deep focus on the customers and the business. The GIST framework applies to
small, medium, and large product ideas. It can be used in a ten-person startup
as well as in a 10,000-person enterprise.
GIST is not a radically new concept. I consider it a
meta-framework that combines tried-and-tested product methodologies with models
and tools I created along the way. Still, when I started sharing GIST the
reaction from the product community and from my clients was overwhelmingly
positive.
There seemed to be a strong need for a practical framework that “puts
it all together.” Many people were willing to give GIST a go, and the
information they shared helped me iterate, improve, and create the framework
described in this book. Still, it’s important not to consider GIST a fixed
process. It’s a model that helps bring evidence-guided thinking into your
development, but you should test it and adapt it to your context.
To make things less theoretical, let me tell you about the
first project in which I experienced the power of evidence-guided
development, my proto-GIST project at Gmail.
In late 2011, not long after we shipped the first wave of
Google+ integrations, I had an idea. I noticed that my personal inbox (Gmail,
of course) was full of promotions, social updates, and notifications that I
didn’t care to read, but had too little time or motivation to delete. In Gmail,
we called this situation inbox clutter.
Wouldn’t it be cool, I thought, if Gmail just magically organized my inbox,
without me having to do any work? I imagined the non-important email showing as
a digest in a side panel: “You also received 5 promotions, 12 social messages,
and 3 receipts.” Users would be able to click on the digest and read these less
important messages when the mood strikes them.
As is often the case, I fell in love with my own idea and
immediately wanted to push it forward for funding, planning, and execution. But, my managers and colleagues were not convinced, and they had their reasons. It
wasn’t clear that inbox clutter was indeed a widespread and severe problem
among casual users who on average received very little mail.
Gmail already had
multiple features to help manage your inbox, but none got a lot of usage.
Worse, my idea to push low-priority mail to the side could easily confuse and
annoy the users who were used to having their inbox work in a certain way ever
since Gmail launched in 2004.
My colleagues and managers were pushing back on my idea, but
implicitly they were asking me some very Googly questions: What are you trying
to achieve? Who is this for? What would be considered a success? In other
words, what is the goal?
At that time Gmail had a product-wide goal to become more
relevant and engaging in a world increasingly dominated by social networks and
instant messaging. I argued that solving inbox clutter is how we, the Gmail
Inbox team, could contribute to this broader goal. My team, skeptical as it
was, was willing to explore.
We started with research to understand inbox
clutter better. One of our backend developers ran a large data analysis to see
how consumer users process their inboxes. The data he brought back took us by
surprise. A huge portion of Gmail’s users did not manage their inbox in any
way. They just went through their emails and read selected messages, otherwise
leaving everything they received untouched.
What does such an inbox look like? We got the answer when we
interviewed casual Gmail users and asked them to show us their inboxes. We
realized that average statistics didn’t tell the whole story.
With the
proliferation of social networks and eCommerce, these people got substantial
amounts of promotion, social notifications, and transactional emails. Some had
tens of thousands of unread messages in their inbox. They had to work hard to
spot important new messages, and even harder to get back to those messages
later. Inbox clutter was a much bigger problem than we thought.
Armed with this information, we were able to formulate an
objective: help casual users interact only with the messages they want to
interact with. This entailed a few success metrics: being able to accurately
predict which messages users wish to read; ensuring that they would never miss
out on an important message, and most importantly, high levels of user
engagement and satisfaction with the new experience as measured by usage,
retention, and surveys. This goal served us in the months to come: we knew who we
were focusing on and what success looks like.
Now it was time to discuss ideas. We had a few: teaching
users how to clean their inbox themselves, grouping or reordering inbox
messages in various ways, and creating digests of the least important messages
(my initial idea). Each idea had its pros and cons, and different people had
their favorite. However, debating the ideas didn’t lead to any useful
conclusion. We knew we had to test, and we had to start with the riskiest part:
the user experience.
So we brought people into our usability lab to test some of
these ideas. In one of those tests, we asked 12 participants to use Gmail, but
with a difference, the inbox was split into tabs. The main tab contained mail
from friends and family, but there were other tabs for social notifications,
promotions, and transactions. When opening each tab, the participants found
their very own messages, taken from their inbox, sorted into the right place.
As if by magic, Gmail organized their inbox by mail category.
This part of the study was a complete sham of course. The
thing the participants saw wasn’t Gmail, but a thin facade of HTML that our
user experience designers cooked up. You could do little with it except look at
your inbox and click into the tabs. Message categorization was another bit of
sleight of hand.
With the participants’ permission, and while they were being
interviewed, a few of us sat in a separate room and copied the first 50
messages from their inbox, sender and subject only, into the appropriate tab,
based on our own best guesses. It was a bit of a race against time, but it
worked. The participants experienced the Tabbed Inbox (almost) as if it was a
real thing.
The results? All the participants immediately understood
what was going on and could explain why the messages were placed where they
were. Using the tabs was natural and easy. Best of all, 10 of the 12
participants absolutely loved this new inbox. In fact, many of them asked how
soon they could have it. The smiles on their faces told a clear story, this was
something they needed, but never knew they could have.
The other two
participants already had their inbox under control using other Gmail functions,
so they didn’t find the new inbox appealing. Interestingly this same ratio of
10–15 percent of people who already managed their personal inboxes well and
thought that the Tabbed Inbox was unnecessary, repeated throughout our research
over the next 14 months. Unfortunately, this minority included many of my
colleagues at Gmail, and as we discovered later, most of the tech press.
Given these strong results we decided to double-down on the
Tabbed Inbox idea and leave other ideas as fallbacks. Still, we didn’t go
all-in. Our initial research gave us sufficient evidence just to fund a small
project, a rudimentary, functioning version of the inbox with a bare-bones user
interface and simplistic classification. We used this early version to
self-test on our own personal inboxes (in Gmail this is called Fishfood, team testing).
I remember
having to convince my team members to stop diligently organizing their inboxes
and to let email pile up, just like ordinary users. It was an adjustment for
some, but we quickly realized that even with this basic version of the inbox,
the experience was quite transformative.
Throughout the
project, we kept progressing in stepwise increments, build a more-evolved
version of the inbox, then test it. Along the way, we enlisted the help of
thousands of Googlers who valiantly agreed to use an interim version of the
inbox in their own personal accounts (this process is commonly called Dogfooding).
We exposed our evolving
email classification system to a small subset of Gmail users through a lab (an
optional feature you can enable in Settings) where they could report
classification mistakes. This gave us much-needed data to improve our
algorithms. We continued to perform regular external user studies as well, some
involving hundreds of participants using the new inbox over periods of weeks.
The project definitely didn’t go according to any plan. We
had to redesign the user interface multiple times, and getting the
classification right required assembling a small team of data and
machine-learning specialists.
We also learned that launching the new inbox on
the desktop version of Gmail first, per the plan, was not enough, inbox clutter
was an even bigger problem on mobile where it was causing excessive
notifications. So we expanded the project to include the Android and iOS Gmail
apps. I didn’t have to pitch very hard to get these other teams to join the
project; the evidence we collected did the talking for me.
The new inbox was launched in May 2013 to generally lukewarm
reviews from the tech press, but we didn’t really care. By then, we had tested
the new feature with thousands of users and were fairly certain we had found a
winner. Still, the testing continued. We gradually switched more and more
accounts into the new experience (programatically skipping those 10–15 percent
who looked like they had things under control) and tracked their behavior,
satisfaction levels, and classification error rates.
The rollout was uneventful, with very little backlash and
tons of positive feedback. People loved the fact that social and marketing
emails no longer competed with emails from friends and family, and never buzzed
their phones. It was easy to switch off the new feature, but few users did, and
our data confirmed they were using the inbox as we expected.
The Tabbed Inbox
won a number of innovation awards inside Google, and was generally recognized
as one of the most important improvements to the product. It was a fun project
to work on, and, more importantly, one that created high impact. Today, the
Tabbed Inbox is the first thing the majority of Gmail’s users see, and we have
strong evidence to show it’s giving them lots of value.
The story of the Tabbed Inbox brings together many of the
elements of the GIST model. Let’s look at it again from the perspective of
goals, ideas, steps, and tasks.
Goals
I tried to kick off the project around an idea. Had I
succeeded, the implicit goal would have been “let’s launch Itamar’s new inbox,”
which would have been great for my ego, but not so much for the end results.
Such goals, often called output goals, are very common in the industry
and very harmful. When most ideas yield little or no value, betting on an
unproven idea is likely to produce waste.
Luckily, in this case, I was forced to
backtrack and form an outcome goal centered on measurably improving the inbox experience of casual Gmail users.
This goal helped focus us on the destination rather than on the way.
In Chapter 2, Goals,
I’ll show you how to lead your area of responsibility, whether it’s a company,
business unit, group, or team, with a concise set of outcome goals. We will see
how to pick goals using the North Star Metric and metrics trees and express
them in the popular format of Objectives and Key Results (OKR). Then we’ll talk
about how leaders can steer without deciding what to work on, and how to tackle
the tough challenge of aligning the goals top-down, bottom-up, and across the
organization.
Ideas
Once we, the Gmail Inbox team, had a goal in place, we were
ready to discuss ideas. But, instead of trying to pick the winning idea through
brainstorms, debate, and reviews, we did three common-sense things that are not
at all common: 1) we based our ideas on research rather than opinions, 2) we
evaluated which ideas seem most promising given the data we had, and 3) we went
on to test multiple ideas.
This process is at the heart of GIST. In Chapter 3, Ideas,
I’ll show you how to collect ideas in idea banks, and how to evaluate and
compare them in a fast, objective, and unbiased way. A key component of this
evaluation is Confidence, which
measures the strength of the evidence in support of an idea.
I’ll teach you how
to use the Confidence Meter (see Figure 1.3), a tool I created specifically for
this purpose and that today is used by numerous companies, including Google.
We’ll see how to cut prioritization debates short and how to leave politics and
pressure tactics out of the discussion.
Figure 1.3: The Confidence Meter
What about Opportunities?
Many people find it
helpful to connect ideas to what is now known as opportunities (formerly called customer
problems, or underserved user needs).
If that describes your approach, you can certainly use it with GIST.
If you
uncover an important user need you can turn addressing it into a goal (as we
did for Inbox Clutter in the Tabbed Inbox project) or you can use it as a
filter for the type of ideas you’d like to pursue. You can also keep a list or
a tree (as
described by Teresa Torres in her book Continuous
Discovery Habits) of opportunities alongside the artifacts of GIST.
In Chapter 6 we will talk about strategic
opportunities, which is a related but different concept to do with strategy
formation.
Steps
In the Tabbed Inbox project we were lucky to land on a
strong idea relatively early in our product discovery. But, many initially
promising ideas turn out to be duds, and even the good ones need iteration and
refinement. For this reason we built the Tabbed Inbox through a series of
build-measure-learn loops, which are called Steps
in GIST. There were usability tests, longitudinal studies, fishfood and
dogfood, labs, and partial launches.
Each step generated a more complete
version of the product, but also gave us new evidence, which helped build our
confidence in the idea and get more funding. We used what we learned in each
step to course-correct and improve the idea. The feature we eventually launched
was profoundly better than the one we started with.
Many product organizations I meet are familiar with this
important concept, which Product Management guru Marty Cagan has aptly named, Product Discovery, but struggle to
implement it, they build too much and test too little; even when they do test
they fail to learn and take meaningful action.
In Chapter 4, Steps, I’ll walk
you through a full example of how to take a high-potential idea, and get it
through a series of steps to a clear Yes/No decision that no person, no matter
how opinionated, will dispute. This process isn’t meant just for big ideas;
cheap ideas entail risks too and should similarly be validated, although faster
and with fewer steps. In Chapter 4, we will review the wide gamut of validation
techniques at our disposal, from data analysis through smoke tests to A/B
experiments, and how to combine them into a development project.
Tasks
You might have noticed in my story how involved my product
team was in shaping the product. That is a norm in many good product companies,
but not a very common practice in the industry. There’s a double problem: the
managers don’t trust the team to make good decisions, and team members are
increasingly focused on delivery and seem disinterested in business and user
goals. Even in companies where these are not major issues, teams struggle to
reconcile product discovery with their agile development practices.
Working with strong people is one of the benefits of joining
Google, but that’s not the only reason my team was so engaged. Google was able
to create an environment that allowed rank-and-file employees to be in the
know, to contribute, and to make decisions. In Chapter 5, Tasks, we’ll talk
about how to do the same with your team, by involving them in all four layers
of the GIST stack, and by having them own discovery as well as delivery. I’ll
also share a tool, the GIST board, to help jointly manage the work of the team
across goals, ideas, and steps.
There’s nothing particularly revolutionary about GIST, and
it’s not the only evidence-guided model out there. Many good product companies
have been working in similar ways for years, although they probably don’t call
what they do GIST.
Still, I see far more companies that try to incorporate
evidence into their work but get stuck halfway between the old model and the
new. These companies adopt new processes, but get few benefits. Part of the
problem is overcoming entrenched mindsets and practices. In order to move
forward you need to change the way your managers, stakeholders, and team think
and behave.
The second half of the book is devoted to helping you adopt
evidence-guided development in your organization. In Chapter 6, The
Evidence-Guided Company, we’ll see how evidence-guided thinking can be deployed
across an entire company, and what role managers, stakeholders, and product
teams play in this brave new world.
We’ll also talk about the often-sticky
topics of product strategy, big projects, cross-team dependencies, and
roadmaps. In Chapter 7, Scaling GIST, I’ll show you how evidence-guided development
works in startups, in midsize companies, and in enterprises. Chapter 8, GIST
Patterns, talks about GIST in common types of products and business models:
B2B, B2C, physical products, marketplaces, and internal platforms and services.
Chapter 9, Adopting GIST, discusses ways to smooth the adoption curve and to
deal with common objections.
How big of a transformation you’re facing depends on where
you are today. GIST may not work in a very traditional organization where
decisions strictly flow top-down, and where the function of the product org is
merely “delivery.” However, if your company isn’t that extreme (most aren’t)
there’s a good chance you can adopt GIST, and it’s even likely that your
colleagues will be in support.
I regularly teach GIST to product leads,
engineers, executives, and business folk, and once the model is clear, the response
is almost always “we need this” and “let’s do it!”
At the end of the day, meaningful change is never easy or
comfortable; it takes visionary, motivated, and persistent people to push it
forward. If you’re a product manager tired of building things that nobody
needs, a manager frustrated with the rate of innovation in your organization,
or an engineer or designer that feels that most of what you create is a waste,
GIST was created to help you drive
the change in your own organization. It’s not going to be an overnight switch,
but I can guarantee you that it’ll be a worthwhile journey.
Let’s get started.
●
The classic approach of
plan-and-execute forces us to commit to a plan while facing high levels of
uncertainty.
●
Our brains and our decision
processes are not well suited to making decisions when confronted with
uncertainty. We rely on opinions, sparse data, consensus, and rank, backed by
cognitive biases, group dynamics, and politics.
●
The track record of success in the
industry is abysmal. Research indicates that only a minority of ideas create
measurable impact, and a high ratio of launched features and products get
little or no use.
●
Evidence-guided companies improve
the odds of success by supercharging human judgment with evidence. They
continuously evaluate and test ideas and update the plans based on what they
learn.
●
The principles of guiding our
decisions by evidence are not new, but they are hard for companies to adopt as
they go against traditional best practices and corporate power dynamics.
●
I created the GIST model, Goals,
Ideas, Steps, Tasks, to help break the change into four distinct areas:
goal-setting, idea evaluation, experimentation, and execution.
●
The Tabbed Inbox story, while not
perfect (no project is), is an example of the GIST model in action. Starting
with user/business-centered goals. Considering multiple ideas, iterating by
running the ideas through various steps, each producing a more complete version
of the feature and more supporting evidence. Optimizing the work for both
discovery and delivery.
●
The first chapters of the book are
devoted to explaining the GIST model in detail. Chapter 6 covers a full-company
example. Chapters 7 and 8 describe how to adapt the model to the size and type
of company you work in. And Chapter 9 explains how to tackle objections, and
smooth the adoption curve.
This
is an excerpt from the book. Available on Amazon, Apple Books, Kobo and other platforms. For details see: EvidenceGuided.com
Copyright © 2023 by Itamar Gilad
This chapter explored the lessons learned from the failed integration of Gmail with Google+, a project that aimed to create a unified social experience for Google users.
It shares how to apply the principles and practices of evidence-guided product development, such as:
– Defining and communicating a clear and compelling vision
– Aligning the product strategy with the company’s strategy and values
– Balancing innovation and iteration
– Using evidence and data to validate assumptions and decisions
– Avoiding common pitfalls and biases.
These lessons are relevant for any product that operates in a dynamic and competitive market, such as forex and CFDs brokers. By following these guidelines, forex and CFDs brokers can create high-impact products that solve real problems for real users, and avoid wasting time and money on features and projects that don’t deliver value or meet the goals.
