The End of Experimentation

On his blog, Seth Godin writes (and this is the whole post):

“The only way to learn from experience is to have different experiences. The very nature of an experiment is that there’s a chance you’re doing it wrong, or at least less ‘right’ than the way you usually do it. Which leads to the trap of no new experiences. The only alternative is to eagerly engage with the possible. If you follow the recipe the same way every time, you’ll get the same results every time.”

My problem is that Seth conflates two things that sound similar but are actually quite different—experience and experiment.

The point of an experience is to…well, have an experience. To try something, or meet someone, or learn something new. The point of an experiment, on the other hand, is to test a specific hypothesis. To try to answer a question with evidence. These are two very different things.

Going to a vegetarian restaurant is an experience. There’s nothing to prove. You’ll probably eat something interesting and tasty and have a good time. You should do that. Why not

Switching to a vegetarian diet is an experiment. You are attempting to prove whether or not you feel better physically, or mentally, or morally. After some amount of time (a month?), you can look at the evidence. You either do, or you don’t feel better. You can either continue with the diet or you can stop.

For the sake of argument, let’s say you do feel better. That doesn’t mean that you won’t feel even better if you switch to a vegan diet. Or if you hire a chef to cook your vegan meals. Or if you switch to a fully liquid diet. Or if you only eat gummy bears. You could test infinitely many diet-based hypotheses and never learn the one “right” or “perfectly optimized” answer, because that is not what an experiment is designed to do.*

My life is systematic and regimented. I wear a similar outfit every day. I cook all my meals on Sunday—many are just variations of the same four or five dishes. I wake up at 7 AM every day. I write this blog on Saturday mornings. There may be better ways of doing all these things. But I have tested my hypothesis (that these things would save time or boost my productivity) and I rejected the null hypothesis of these things do not work. And I keep doing these things for the very reason Seth is concerned—I want the same results every time, because those results are working.

It is not controversial to say you should have new experiences—go on trips, read widely, and the like. But no one is saying we should experiment with a new polio vaccine. The one we have produces the same (positive) results every time. Thankfully. When you forget that an experiment serves to test a hypothesis, you end up in a world of constant experimentation. Experimentation as experience.

Once you feel you’ve found something that works well enough, stop. Wasn’t that whole point of conducting the experiment in the first place?

*In fact, an experiment is really only designed to tell you if a null hypothesis is unlikely to be true. It cannot prove something true, and it can never definitively prove anything false.

Each Monday, I share strategies to help you pursue your passions. Try it. You’ll like it.

Photo by chuttersnap on Unsplash