and where to find them
Where I’m from, gotta be a quick decision-maker
E-40, The Barbershop
How organizations make product decisions is the crux of a lot of my writing about Product Management. Much of this is a reaction to the current dominant Product Management praxis, which assumes that uncertainty can be squeezed out through the decision making process. But so much of product intuition, specifically product feel—and product feel/execution alignment relies on choosing options that have the highest probability of achieving your goals by default.
Notes from Apple’s Golden Age
“We only need one of these, right?”
Not what I was expecting. I think I may have swallowed hard. [Steve Jobs] was still looking at me, and so, with a half shrug, I said, “Yeah . . . uh . . . I guess so.”
Steve sized me up a little and then asked, “Which one do you think we should use?”
“Well, I’ve been using these demos for the past few days, and I’ve started to like the keyboard layout with the bigger keys. I think I could learn to touch type on it, and I think other people could too. Autocorrection has been a big help.”
“OK. We’ll go with the bigger keys.”
Kocienda, Ken. Creative Selection: Inside Apple’s Design Process During the Golden Age of Steve Jobs (p. 30). St. Martin’s Press. Kindle Edition.
Should we integrate 3rd party storage options? This API call is 300ms, is that fast enough?
We make these decisions every day, often in real-time, and each can have a critical impact on business and customer outcomes. And sometimes, those decisions are complex and urgent: We need to design a sign-up flow, we have no time to A/B test it and no time for usability research. Designing this flow is going to involve hundreds of micro-decisions. A team with product intuition will arrive at a great outcome by default, a team that lacks product intuition will often redesign the flow time and time again and still ship a subpar experience. How do we get the product feel right by default?
Well, you need a useful decision framework for making decisions that will also have a high likelihood of success—or you’re going to ship a bad product by default. This is product feel/execution alignment. Making product decisions fast and “right”—that other teams would make slow and “wrong.”
Whoever makes the “right” decision the fastest wins.
Premise 1: Acquiring information has a cost
Implication: Decision making can consume an organization’s resources of time and money.
Impact: Assuming all things are equal, whoever arrives at the most likely to succeed decision with the least information wins because they have more time and dollars available to make the decision successful.
Example: The tools and services to translate just about any text is available to everyone with an internet accessible computing device. But the “cost” of translating this information is at least an order of magnitude slower than our innate ability to process information when we’re fluent in a language. We don’t say we “know” a language when we have access to translators—we know a language when we can use a language fluently, in real-time.
Information has a cost.
Let’s assume it takes even 20 minutes to answer a question. Now answering this question has to compete for a finite resource, time. If you spent all day answering questions and gathering information, you can only 24 questions in an eight hour day (8 hours * 3 questions per hour), and you have to choose which questions to answer. (Ignore the actual number of questions you can answer, and just follow the premise–even when you can get information “fast” it still has a cost)
So let’s play this out. I’m a middle manager accountable for selecting the city for our next marketing campaign. My organization is “data-driven,” meaning the only appropriate way to answer a question is with data. I’m rushing between meetings, and I believe I have the right idea—but without confirmatory data—we’re not going to do it.
Luckily I’m in an organization that has data democracy, so at least I can answer the question myself. I find a moment between meetings to pull the data—how many of our existing customers are in the city. I share it with a colleague to get feedback, they say that’s good, but you also need to answer what the conversion rate is for those customers. My next window to answer that question is tomorrow, and I suspect once I review with my manager, she’ll have more feedback that I need to take into account.
Now let’s say for a moment, I had all that information in my head. And I used it to arrive at the right answer without that work. I’ve saved myself at least half of a workweek to arrive at the same decision. (Keep in mind we only have ~48 of these workweeks in a year)
Treating Decisions as Right or Wrong Is Hazardous
A complementary approach for arriving at the “right” answer faster feedback loops. So if a decision has a 60% probability of being “right,” and you can learn that it’s the wrong decision within a month of the decision, you can make a wrong decision now—and know with 100% certainty in a month what the right decision is. Alternatively, an organization can take three months to increase the odds from 60% to 90%, taking three months and still having lower odds than if the organization had made a “wrong decision” fast.
For simplicity, I used the terms “right” and “wrong,” but even this is wrong. It assumes that there is a right or a wrong decision and that it’s knowable whether a decision was right or wrong after the fact (it’s often not).
When we’re framing things as odds of right or wrong decisions—we don’t leave room for a good decision with a bad outcome or a bad decision with a good outcome. Joining a fast-growing company in a great role could be a good decision. If the company later fails, that’s a bad outcome; however, it doesn’t, by itself, make the decision a wrong one.
Focus on the Likelihood of Producing Desired Outcomes
To improve our decision making, we need to think about these things as not the odds of right or wrong decisions—but based on their likelihood of producing our desired outcome.
The nature of the world there are myriad ways to arrive at a destination; bats and birds evolved to fly independently. By this extension, there are often many options that have the same likelihood of success.
I encourage you to challenge whether you really agree with this premise, as most organizations don’t operate this way:
For any decision, many options have the same likelihood of producing the desired outcome.
Let’s take a personal example. Should I move to a better school district or not? Parents, including myself have fretted over this decision and will continue to do so. One way to think about this is to worry about the right or wrong decisions—better school districts correlate with better outcomes. Another way to think about it is what options that involve moving and what options that don’t include moving have the same probability of helping your child grow up to become an effective citizen?
- Moving to a better school district
- Enrolling your child in extracurricular activities that develop confidence
- Hiring a personal tutor
- Choosing an after-school program to supplement learning
- Increasing active parenting time
- Building autodidactic habits and grit
- Increasing empathy through exposure to different cultures and socioeconomic status through classmates, extracurricular activities, and family friends
We’ve taken a binary decision, move or not move, and turned it into a decision with multiple right answers.
Alright now for the more heretical premise.
In a world where there are multiple right answers and numerous options that have the same probability of leading to the desired outcome, you gain nothing by analyzing the decision any further (caveats discussed further below).
It’s a coin flip decision.
Coin flip decisions
A coin flip decision is a decision in which the probability of a successful outcome does not improve with more information about the options.
It’s a decision where more analysis is only decision theatre; it doesn’t result in a higher-quality decision. In fact, it often steals time away from making the eventual decision successful.
Back to the example of “what city to rollout to next”? You can probably narrow the options to 10 or fewer just from your experience.
It’s getting from the 10 to the one or two where organizations spend a lot of decision time. As an example, New York and DC metro were at the top of the betting pools for Amazon HQ2 locations. Yet Amazon spent a year selecting an option that they reversed within three months. If they made a “bad” decision sooner, they could have spent more time ensuring that the decision was successful.
We’re so accustomed to looking for the best option, or the right option. We don’t even have language for when there are many good options. Imagine Bezos going to the board and saying—we’re going to the DC area as our HQ2.
Our discomfort with uncertainty is such that we spend time and energy trying to squeeze information required to make the “best” decision where there are a lot of “best” decisions.
Now I’m not proposing that Amazon should have chosen its headquarters based on a coin flip. But they spent two years making a decision they reversed in two months. Clearly, the decision theater did not properly value the information gathering “post-decision” vs. pre-decision. A coin flip decision is a decision in which the probability of a successful outcome does not improve with more information about the options.
How to find them…
A coin flip decision has more than one option (or combination of options) that meet the following criteria:
- Equal probability of success. Each option has, based on the team’s assessment the same probability of leading to the desired outcome.
- Similar cost. The cost of each option is on the same scale.
- Exemplars. People/companies have arrived at different answers to this question but equal success.
- Philosophy. All remaining options align with your core values, philosophy, and strategy.
- Success factors. We have the prerequisites to make either option successful.
- Inheritance experiment. If you inherited the decision—would you look to reverse the decision or move forward?
A decision as meaningful as to whether you should choose Amazon or Azure for a cloud solution is a coin flip decision for many companies. This decision is super important—and yet the team can succeed with both options. Our draw to look for the best choice where there is not a singular “best” is decision theater.
You can tell you are participating in decision theater when the effort to “persuade” decision-makers (or convince yourself) that a decision is reasonable exceeds the effort to arrive at a reasonable decision.
In effect, decision theater harms decision quality by forcing the decision-maker to overstate the likelihood of success and to develop an overstated conviction about the direction. When a coin flip decision becomes decision theater—the advocator has to overstate their conviction in the idea. Otherwise, decision-makers won’t support it.
All of this derives back to our misguided understanding about the likelihood of success.
If you find yourself in an organization that emphasizes decision theater, do two things: participate and advocate for something better.
Participate. Decision theater is frustrating, but it’s often comforting for an organization. Instead of fighting it and preventing good decisions from surfacing, do your part but advocate for something better.
Advocate. Remind everyone at every step that there are multiple decisions that could lead to success. And what you’re presenting is the option you’re recommending but that there are multiple right answers, and the difference-maker will be the steps you all take to make your ultimate decision successful.
What does this mean?
An organization that excels at decision making really has two strengths:
- Making decisions—that are likely to succeed
- Making—decisions likely to succeed
Of the two, the former is a point in time; the latter is continuous.
How to flip a coin
AKA a “coin flip” is not a coin flip
Flipping the coin in these decision making process is not literally choosing heads or tails. It’s first identifying that it’s a coin flip decision based on the above criteria, including reaching an agreement with decision stakeholders to acknowledge that there are likely multiple options with the same probability of success.
Next, step through the heuristics for choosing the best option.
Bias toward the option that most aligns with prior decisions.
Aligned decisions create clarity. Which option aligns most with your organization’s values, strategy, or prior decisions? If there is one that stands out, choose it. For example—on the question of whether I should move to a better school district, I might choose the decision based on what gives me the most time with my family. Another parent might bias towards multicultural exposure.
First, choosing the option that most aligns with your values is a cheap way to operationalize your value system.
Make a default operating decision.
A leader I once knew had a saying:
The plan is the plan until it’s not the plan.
Assuming there are still multiple options that are aligned with your values. You change the default. When you change the default, you choose an option and keep that option until information surfaces that you should choose a different option.
You’re now have selected the operating decision. When you make an operating decision, the team is not allowed to hedge, you go through the act of making the option successful—and only if you surface material information (such as difficulty implementing) do you have a discussion about changing the operating decision.
The key to doing this right is that someone else makes the decision, not you. When decisions reach this stage, you push it to either the closest person to the problem or an alternating choice of stakeholders. Steve Jobs did this when he asked Ken to weigh in which keyboard he would choose. He wasn’t listening for the decision, but the rationale.
The reason someone else makes this decision is to account for your bias. As the decision lead, you may have subconsciously dressed up a low probability of success option to look at similar odds of success as a better option, or you may have discounted a high probability option to have lower odds for one reason or another. You also give a final opportunity for information to surface that both options don’t have the same probability of success.
A risk-taker will bias towards more contrarian options, a risk-averse leader will skew towards the decision that’s less novel. Compounded through a series of decisions, you reduce the probability of success for a given decision by making decisions with the same bias–systematically too risky, or too sober.
By diversifying who makes the decision and moving forward with that decision until new information surfaces, you expose the organization to the upside of decisions that were misidentified as having a similar probability of success—but actually had better odds. If the idea of outsourcing the decision at this stage is discomforting to you—you should reflect on why. By the time a decision makes it to this stage, you’ve already decided that each option has similar odds of success–so discomfort may be due to a bias against one option, or you don’t believe they have the same probability for success.
Living it
In addition to using this decision framework when I have more than one good option in my personal life, such as competing for job offers with different pros and cons, I use it as a product leader as well.
I once lead a product that needed to present location data. There were successful apps in the marketplace that had a map view and successful apps that did not—for the same scenario. Our understanding of the job to be done found the map view had the potential to help and harm the customer goals in different ways.
We chose not to include one.
We were asked why we chose to move forward without a map view. I shared, there were successful apps with both approaches. We felt like it was a coin flip decision with both options having a similar likelihood of success.
“The best way to deal with coin flip decisions is to identify that it’s a coin flip decision—then flip a coin fast and move forward.”
about the author
Mikal is a reformed startup CEO and experienced Product Executive based in Austin, TX. After years leading product teams at Microsoft, Nordstrom and most recently VP of Product at RetailMeNot, he now serves as a product coach helping teams in growing tech markets work their way up The Product Team Ladder.
subscribe (it’s free)
I’ve been writing business notes since 2005 (as notsocommoncents) – join my newsletter, sent every Tuesday, to read notes for the week on topics that go beyond the daily news cycle, plus any new blog posts.
You should sign up – and hear about an awesome thing each week.
mikalfm 