• Matthew Lerner

Failed Experiment

Most growth experiments should fail. (If not, test bolder stuff). But, when an experiment fails, how do you know if you’re moving forward? Or wasting precious time? You can feel it.


Do you feel smarter? Does it inform your next experiment?

- Can you rule out an option?

- Did you find some qualitative insight?

- Did some other metric move? In an unexpected way?

- Did you learn something about your tech or process?


Ultimately you need to refine your understanding of the prospects' conception of the problem space, their expectations, assumptions, doubts and behaviours.


You absolutely should get smarter after each experiment. If not, you’re going sideways.


How to Fix This?

If your experiments are not making you smarter, fix that with two simple process changes:

  1. Better up-front experiment thinking: Get very clear on your (specific written) hypothesis and success metric. Ask people to write predictions. Isolate one variable (e.g. message or channel) and/or test bold things that challenge your most important beliefs & assumptions about your customers.

  2. More post-experiment thinking: People tend to overthink what might go wrong and underthink what actually did go wrong. For each failed experiment, force yourselves to spend a full 20 minutes looking through the results and talking through why the results differed from your predictions. List out as many ideas as possible.

It’s hard to know if you’re making progress when the numbers aren't going up. But it’s easy to know if you’re learning. And if you're learning, you're making progress.

5 views0 comments

Recent Posts

See All

How do you measure "brand building?"

Are you planning to “build a stronger brand” in 2021? (Who isn’t?) But that goal is kind of fuzzy… how do you actually measure it? You could use complicated (expensive) solutions… custom “share of voi

I Always Start Clients Here

If you found analytics software like HotJar, but it could actually read your visitors' minds (not just their mouse trails), how much would you pay for it? Good news… 1. It exists 2. You prob. already

Start with "No."

The most expensive thing you can do as a startup is to add a product feature. I think most leaders have no idea how much a new product feature costs! Calculating The Fully Loaded Cost Engineering time

Never miss a lesson.

COPYRIGHT (C) 2018 STARTUP CORE STRENGTHS LIMITED. ALL RIGHTS RESERVED.

Terms & Conditions | Privacy Policy | Contact Us
 
Please note that participants’ comments on the program and results information refer to similar programs that members of our team have developed and run as part of 500 Startups. The information presented has been prepared in good faith by Startup Core Strengths, but has not been independently verified. Please read our Terms and Conditions for additional information.