• Matthew Lerner

Failed Experiment

Most growth experiments should fail. (If not, test bolder stuff). But, when an experiment fails, how do you know if you’re moving forward? Or wasting precious time? You can feel it.


Do you feel smarter? Does it inform your next experiment?

- Can you rule out an option?

- Did you find some qualitative insight?

- Did some other metric move? In an unexpected way?

- Did you learn something about your tech or process?


Ultimately you need to refine your understanding of the prospects' conception of the problem space, their expectations, assumptions, doubts and behaviours.


You absolutely should get smarter after each experiment. If not, you’re going sideways.


How to Fix This?

If your experiments are not making you smarter, fix that with two simple process changes:

  1. Better up-front experiment thinking: Get very clear on your (specific written) hypothesis and success metric. Ask people to write predictions. Isolate one variable (e.g. message or channel) and/or test bold things that challenge your most important beliefs & assumptions about your customers.

  2. More post-experiment thinking: People tend to overthink what might go wrong and underthink what actually did go wrong. For each failed experiment, force yourselves to spend a full 20 minutes looking through the results and talking through why the results differed from your predictions. List out as many ideas as possible.

It’s hard to know if you’re making progress when the numbers aren't going up. But it’s easy to know if you’re learning. And if you're learning, you're making progress.

35 views0 comments

Recent Posts

See All

Top Startup Pricing Mistakes (1 of 2)

If you’re thinking about lowering your prices, read this 👇🏼 Setting the right price is hard, especially for your own startup. Most startups hugely under-price their products. But why? I guess people

How do you measure "brand building?"

Are you planning to “build a stronger brand” in 2021? (Who isn’t?) But that goal is kind of fuzzy… how do you actually measure it? You could use complicated (expensive) solutions… custom “share of voi

I Always Start Clients Here

If you found analytics software like HotJar, but it could actually read your visitors' minds (not just their mouse trails), how much would you pay for it? Good news… 1. It exists 2. You prob. already