AltF IdeaLab
MVP experiment planner
The MVP experiment planner helps you choose one of eight lean tests, such as a landing page smoke test or a pre-sale, set a success target before you start, and work out how many people you need. When results come in, it marks the test pass, fail or unclear using a 95% confidence interval. Small samples often come back unclear, which is a real answer, not a bug.
- Free, no sign-up
- Runs in your browser
- No AI guesswork
- Export to Markdown
How to use the mvp experiment planner
- 1
Pick an experiment
Choose from problem interviews, landing page, fake door, pre-sale, concierge MVP, Wizard of Oz, competitor teardown or targeted survey.
- 2
Write the hypothesis and target
State what you expect and the success rate that would count as a pass, before running the test.
- 3
Size the sample
Enter the rate you expect and the margin of error you can accept to get the number of people needed.
- 4
Enter the results
Add the number of successes and the sample size to see the observed rate, its interval and the verdict.
How it works
- Sample size uses n = z^2 x p(1 - p) / e^2 with z = 1.96 (95% confidence), rounded up. The expected rate is limited to 1-99% and the margin to 0.5-50%.
- Results use a 95% Wilson score interval around the observed rate.
- Pass if the low end of the interval is at or above your target, fail if the high end is below it, otherwise unclear.
- Each experiment type comes with a default metric, target and sample, which you can change.
Worked example: a landing page smoke test
You expect about 10% of visitors to join the waitlist and want to know the rate within plus or minus 3 points: n = 1.96^2 x 0.10 x 0.90 / 0.03^2, rounded up to 385 visitors. Accepting plus or minus 5 points needs 139.
With a 10% target, 42 sign-ups from 300 visitors is 14.0%, with an interval of 10.5% to 18.4%. The low end clears 10%, so the verdict is pass.
34 sign-ups from 300 is 11.3%, with an interval of 8.2% to 15.4%. The interval straddles 10%, so the verdict is unclear.
4 deposits from 20 prospects against a 15% target is 20%, but the interval is 8.1% to 41.6%, so that is also unclear.
What the mvp experiment planner does not do
- The maths assumes each person is an independent, random sample of your audience; traffic from friends or one forum can skew it.
- A pass shows the target was met in this test, not that the business will work.
- It does not run the test, send traffic or collect sign-ups for you.
- For comparing two page variants, use a dedicated A/B test significance calculator.
MVP experiment planner FAQ
How many people do I need for a landing page test?
It depends on the rate you expect and the precision you want. At an expected 10% with a 5-point margin the formula gives 139; tightening to 3 points gives 385.
What does an unclear verdict mean?
The 95% interval around your result includes the target, so the data cannot yet say whether you passed. Collect a larger sample or accept the uncertainty.
Why set the target before the test?
Choosing the bar after seeing the result makes almost any outcome look like a success. The planner judges results against the target you wrote down first.
More IdeaLab tools
- Startup idea validatorScore the evidence behind an idea out of 100 across five dimensions.Open
- Lean canvas generatorFill the nine lean canvas blocks and see which ones are still empty.Open
- Assumption mapping toolRate each assumption by importance and evidence to find what to test first.Open
- Customer interview questionsGet a problem interview script and check your own questions for bias.Open
- LTV:CAC calculatorWork out lifetime value, LTV:CAC ratio and CAC payback in months.Open
- TAM SAM SOM calculatorSize the market bottom-up from buyer count, annual price and your share.Open
- Business idea comparisonCompare saved ideas side by side on the same evidence scorecard.Open
Guides for this step
Where to launch and get your first feedback and backlinks
Once a landing page or concierge test is live, launch directories and maker communities bring the first outside visitors, honest feedback and a profile link back to your site. Our backlinks directory lists them with estimated authority, link type, cost and the submit page.