Review
This is a legendary startup book that somehow evaded me for a long time. I was still aware of the general ideas, however, reading it sooner would probably help me avoid a basic mistake we made in one of our latests projects. I love the concept of Validated Learning and are already working on implementing it into our processes.
Highly recommended.
tl ;dr
Run experiments to learn if something is working. Build-Measure-Learn feedback loop. Build the MVP (either a product or a single feature), Measure the effect on customers, Learn if it’s value-creating or wasteful. Use cohort analysis for measurements. Always work in small batches.
Summary
1 START
The goal of a startup is to figure out the right thing to build – the thing customers want and will pay for – as quickly as possible.
2 DEFINE
Brad (Intuit) explained to me how they hold themselves accountable for their new innovation efforts by measuring two things: the number of customers using products that didn’t exist three years ago and the percentage of revenue coming from offerings that did not exist three years ago.
3 LEARN
Validated learning is the process of demonstrating empirically that a team has discovered valuable truths about a startup’s present and future business prospects. In other words, which of our efforts are value-creating and which are wasteful?
4 EXPERIMENT
Break It Down
The value hypothesis tests whether a product or service really delivers value to customers once they are using it.
The growth hypothesis, which tests how new customers will discover a product or service.
The point is not to find the average customer but to find early adopters: the customers who feel the need for the product most acutely.
Build-Measure-Learn feedback loop
To apply the scientific method to a startup, we need to identify which hypotheses to test.
The first step is to enter the Build phase as quickly as possible with a minimum viable product (MVP).
If we’re building something that nobody wants, it doesn’t much matter if we’re doing it on time and on budget.
5 LEAP
Analogs and Antilogs
iPod as an example. Sony’s Walkman was the analog (people will listen to music in a public place). Napster was an antilog (people will download music but won’t pay for it).
Genchi gembutsu: “go and see for yourself”
6 TEST
The goal of the MVP is to begin the process of learning, not end it.
If we do not know who the customer is, we do not know what quality is.
7 MEASURE
How do we know that the changes we’ve made are related to the results we’re seeing? More important, how do we know that we are drawing the right lessons from those changes?
This is an important rule: a good design is one that changes customer behavior for the better.
Cohort analysis: Instead of looking at cumulative totals or gross numbers such as total revenue and total number of customers, one looks at the performance of each group of customers that comes into contact with the product independently.
Cohorts and Split-tests
Instead of looking at gross metrics, Grockit switched to cohort-based metrics, and instead of looking for cause-and-effect relationships after the fact, Grockit would launch each new feature as a true split-test experiment.
User stories were not considered complete until they led to validated learning.
Stories could be cataloged as being in one of four states of development: in the product backlog, actively being built, done (feature complete from a technical point of view), or in the process of being validated.
Validated was defined as “knowing whether the story was a good idea to have been done in the first place.”
A solid process lays the foundation for a healthy culture, one where ideas are evaluated by merit and not by job title.
Three A’s of metrics: actionable, accessible, and auditable.
Each cohort analysis says: among the people who used our product in this period, here’s how many of them exhibited each of the behaviors we care about.
8 PIVOT (OR PERSEVERE)
With pivots it is not necessary to throw out everything that came before and start over.
Once you have found success with early adopters, you want to sell to mainstream customers. Mainstream customers have different requirements and are much more demanding.
When efforts at tuning the growth engine are reaching diminishing returns, it’s a classic sign of the need to pivot.
A pivot is a special kind of change designed to test a new fundamental hypothesis about the product, business model, and engine of growth.
Sustainable growth follows one of three engines of growth: paid, viral, or sticky.
9 BATCH
The small-batch approach produces a finished product every few seconds, whereas the large-batch approach must deliver all the products at once, at the end.
The biggest advantage of working in small batches is that quality problems can be identified much sooner.
Instead of working in separate departments, engineers and designers would work together side by side on one feature at a time.
So strong is the instinct to work in large batches, that even when a large-batch system is malfunctioning, we have a tendency to blame ourselves.
10 GROW
Sustainable growth is characterized by one simple rule: New customers come from the actions of past customers.
1. Word of mouth.
2. As a side effect of product usage.
3. Through funded advertising.
4. Through repeat purchase or use.
The Sticky Engine of Growth: companies using the sticky engine of growth track their attrition rate or churn rate very carefully.
The Viral Engine of Growth: many viral products do not charge customers directly but rely on indirect sources of revenue such as advertising.
The Paid Engine of Growth
successful startups usually focus on just one engine of growth, specializing in everything that is required to make it work.
11 ADAPT
Every new engineer would be assigned a mentor, who would help the new employee work through a curriculum of systems, concepts, and techniques he or she would need to become productive at IMVU.
The Wisdom of The Five Whys
Repeating “why” five times, like this, can help uncover the root problem and correct it.
Make a Proportional Investment
When blame inevitably arises, the most senior people in the room should repeat this mantra: if a mistake happens, shame on us for making it so easy to make that mistake.
“If our production process is so fragile that you can break it on your very first day of work, shame on us for making it so easy to do so.”
1. Be tolerant of all mistakes the first time.
2. Never allow the same mistake to be made twice.
Good Five Whys session has two outputs, learning and doing.
Each team works on a new feature for approximately six weeks end to end, testing it with real customers throughout the process.
12 INNOVATE
Startup teams require three structural attributes: scarce but secure resources, independent authority to develop their business, and a personal stake in the outcome.