"We loosely define an iteration in discovery as trying out at least one new idea or approach......To set your expectations, teams competent in modern discovery techniques can generally test on the order of 10-20 iterations per week" - Marty Cagan

‍

‍

"The way most teams test ideas isn't feasible when working with a set of ideas. We can't build three ideas for the same target opportunity and A/B test them to see which is the most effective. It would take too long. More often than not, designing and building a testable prototype of each idea will take more time than we have. Instead we need to learn how to quickly test our ideas through fast iterations" - Teressa Torres

‍

‍

The role of every product team, and product manager specifically, is to ensure that what gets built is the right thing to build for users. It's a tough responsibility because there's only one way to know if what you've build is "Right", by shipping it. Until it's in your customer's hands, you just can't know. Marty wants you to reduce risks and increase confidence in your solution before shipping it using 10-20 iterations a week, but Teressa knows this is not feasible for most teams based on how they test today. So, how do you test 10-20 iterations each week and become a high-performing product team and rockstar product manager? 

‍

‍Assumption Testing

The answer to your prayers is here. A way to test your product ideas quickly without  having to build a new prototype. It also is a key technique to help teams focus on the problem and understand how well a solution satisfies or solves that problem relative to other solutions.
‍

If you use this guide with your team, you'll be able to generate your own set of assumptions that can be tested and improve your solution's likelihood for success.

‍

‍

When to use assumption testing?

1. When you want to increase your confidence in a solution
2. When you’re trying to determine which solution is “best”
3. When you need to rapidly test your product ideas without prototypes

‍

Assumption testing is an advanced product technique, so you need to have the following defined and aligned upon by your product team to actually generate valuable assumptions.
‍

1. Business Outcomes
2. Product Outcomes
3. Target Opportunities
4. Specific Solutions
5. The Opportunity Solution Tree
   ‍

When i work with teams, i usually like to run a lunch and learn session prior to assumption testing so that the team can align on the terms and concepts. Here's an example of and how it relates to these different product concepts.

‍

‍

Business Outcomes

Definition: A business outcome measures how well the business is progressing.
‍

Examples:

• Grow Revenue: Increasing overall sales and income.
• Grow Profit: Enhancing the profitability of the business.
• Grow Margin: Improving the difference between sales and production costs.
• Grow Market Share: Expanding the company's portion of the market.
• Reduce Operating Costs: Decreasing the costs associated with running the business.
• Reduce Churn/Increase Retention: Lowering the rate at which customers stop doing business with the company and increasing customer loyalty.

‍

‍

Product Outcomes

Definition: A product outcome measures the customer behaviors that will drive these business results.
‍

Examples:

• Expansion to New Countries: Attracting customers from new geographic locations (e.g., expanding from the US to the UK, Australia, Canada).
• Appealing to Multilingual Customers: Attracting customers who speak different languages (e.g., expanding from English to Spanish-speaking markets).
• Accelerating Purchase Time: Encouraging customers to make purchases sooner, thus earning revenue faster (e.g., buying this year instead of next year).
• Increasing Purchase Volume: Encouraging customers to buy more (more products, more licenses, more months of service).
• Enhancing Perceived Value: Leading customers to spend more on the same product (e.g., willing to pay more upon renewal or charging for trial access).

‍

‍

Target Opportunities

Definition: Target Opportunities refer to specific customer needs, pain points, or desires identified through research. Below are some examples for a fictional company that sells recycled fish tanks.
‍

Examples:

1. Difficulty in Setting Up a Fish Tank System for the First Time: Many customers face challenges in understanding and implementing the initial setup process for a fish tank, especially when dealing with recycled or unique tank systems.
2. The fish tank doesn't match my furniture or room: Unhappy with the fish tank atheistic because it is using a recycled stand that may not match their home decor.
3. My tank gets very dirty and i don't have time to clean it.: Concerns about the ease of maintaining and cleaning recycled tanks, especially for first-time fish tank owners.
4. Which fish should i buy for my tank? Which ones will go together well: Lack of knowledge or resources about fishkeeping

‍

‍

Specific Solutions

Definition: Specific Solutions have context, meaning that they happen to a specific person at a specific point in time and  directly address a desire or need.
‍

Examples:

1. Solution A: Offer free, personalized consultation sessions for first-time fish tank owners to guide them through the setup process.
2. Solution B: Create and distribute step-by-step setup guides, including videos and written instructions, tailored to different tank types.
3. Solution C: Develop a starter kit for first-time users, including all necessary components and a simple setup manual.

‍
‍

Opportunity Solution Tree

You can take all of the above elements and organize them visually to create an OST(Opportunity Solution Tree). Let's use that fictional company that sells recycled fish tanks as an example.

Product: Recycled Fish Tank Store
‍

Business Outcome: Decrease Cost per Acquisition (CPA)
‍

Product Outcome: Increase number of extremely positive testimonials
‍

Target Opportunity: Difficulty in setting up a fish tank system for the first time
‍

Solutions:

1. Solution 1: Offer free, personalized consultation sessions for first-time fish tank owners to guide them through the setup process.
2. Solution 2: Create and distribute step-by-step setup guides, including videos and written instructions, tailored to different tank types.
3. Solution 3: Develop a starter kit for first-time users, including all necessary components and a simple setup manual.
   ‍

‍

In this example, you can see that we have 3 different specific solutions that may or may not address the target opportunity and help us make progress towards our outcome. Underlying all of these solutions are their built in assumptions that we have about them. An assumption is a belief that may or may not be true. There are assumptions that need to be true for your idea to succeed. As a general rule, the more specific your assumption, the easier it will be to test.

So what assumptions underpin our solutions? 

Broad Assumption: Customers find the initial setup of a fish tank system challenging and seek guidance.
‍

Specific Assumptions:

1. Personalized consultations will provide the right level of support and information for first-time tank owners.
2. Step-by-step guides and videos will simplify the setup process and reduce customer anxiety.
3. A starter kit with everything needed for setup will be appealing to first-time tank owners.
4. Customers who have a positive initial experience are more likely to leave extremely positive testimonials.
5. Providing these resources and support will lead to word-of-mouth referrals, thereby decreasing the CPA.
6. The guidance offered will be clear, comprehensive, and tailored to the needs of beginners.
7. The availability of support and resources will be a key factor in the purchasing decision of potential customers.
8. First-time customers who receive this support will perceive higher value in their purchase, enhancing customer satisfaction.
9. The cost of providing these additional services and resources will be offset by the increased number of sales and reduced marketing expenses due to positive testimonials

Now we can add them to complete our final OST!
‍

Remember, these specific assumptions are not necessarily true, but we need them to be true for our specific solution to be true. So, how do we actually test them? Well, maybe we don't actually need to test them at all. Assumptions need to be prioritized based on their importance and evidence to determine if you should even spend your time on them. Remember we are using this technique because we don't have the time to explicitly prototype every single solution if we want to deliver value to users.

‍

You can use the simple action board below to easily visualize which assumptions should be tested right away. Very important assumptions, or assumptions that are fundamental to the solution(s), which you have weak evidence for or confidence in need to be tested first to mitigate the risk as soon as possible. If you miss testing one of these assumptions you'll likely build the wrong solution.

‍

Finally! You have your assumptions generated, and prioritized, and now you can take those most important assumptions and test them. One way to make it easier for your team to determine how best to test an assumption is to categorize them first by product risk.

‍
Types of assumptions:

1. Desirability assumptions: does anyone want it? Will out customers get value from it?
2. Viability Assumptions: should we build it? There are many ideas that will work for our customers but won't work for our business.
3. Feasibility assumptions: Can we build it?
4. Usability assumptions: Is it usable? Can customers find what they need?

‍

‍
Here’s a few examples of how I would categorize the above assumptions into these different categories of product risk:

Desirability Assumptions

• Personalized consultations will provide the right level of support and information for first-time tank owners.
• A starter kit with everything needed for setup will be appealing to first-time tank owners.
• First-time customers who receive this support will perceive higher value in their purchase, enhancing customer satisfaction.
  ‍

‍Testing Methods

• User Interviews
• User Feedback
• Ethnography
• Ad Testing
  ‍
  ‍

Viability Assumptions

• Providing these resources and support will lead to word-of-mouth referrals, thereby decreasing the CPA.
• Customers who have a positive initial experience are more likely to leave extremely positive testimonials.
• The cost of providing these additional services and resources will be offset by the increased number of sales and reduced marketing expenses due to positive testimonials.
  ‍

‍Testing Methods

• Modeling Analysis
• Internal Stakeholder Discovery
• Ad Testing
• Dry Wallet

Feasibility Assumptions

• The availability of support and resources will be a key factor in the purchasing decision of potential customers.
  ‍

‍Testing Methods

• Proof of Concept or Technical Spike
• SWOT Analysis/Matrix
• Impersonator

‍

Usability Assumptions

• Users will be able to find and follow the step-by-step guides and videos independently will simplify the setup process
• The guidance offered will be clear, comprehensive, and tailored to the needs of beginners.
  ‍

‍Testing Methods

• Prototype
• UAT (User Acceptance Testing) Scores
• Scenario Testing
• A/B Testing

‍

‍

It's critically important to remember that shipping a solution is the only true way to see the results. These techniques are to identify and reduce the assumptions that we have more or less confidence in, but ultimately we can’t get our confidence in a solution to 100% or our solution’s risk to 0%. The goal is to perform these tests quickly so that you can build quickly and deliver value to the user and the business quickly.