Experiments on Self

Empirical studies have become fashionable in software engineering, and this makes sense. In software engineering discipline, most of the time, we do not develop an algorithm or a new solution for a problem. We are looking for ways to facilitate solution development, and those “ways” need to be somehow evaluated, that’s were empirical studies play a significant role.

In these kind of studies, to my knowledge, a researcher or group of researchers perform some experiments in a controlled environment to study phenomena. Sometimes, the researches go to the real world to observe the real software developers, interview them, and survey them to find out how they think, they develop a system, how they manage development progress, and blah blah.

But, as far as I have seen, no one has ever dared to study things on him or herself. Seems reasonable, if I am a researcher, I have developed a notation, and then I use it, and announce it was useful, who is going to believe it? And that’s reasonable, if I am a desperate PhD student, so greedy to publish positive results, I am so biased that I may feel the notation is working very well for me!

So based on many ethical issues and biases, self experiments are not used in software engineering empirical studies at all. But experiments on one self is not such a huge mistake in other discipline, and in history we see a lot of example, where biologists experienced a vaccine on themselves. A very interesting example is the story of discovering that a bacteria causes stomach ulcers, not stress or spicy food. Everyone disagreed with Dr. Warren and Dr. Marshall’s (Nobel Prize winners in 2005 for this discovery) theory because they didn’t believe bacteria could survive in acid. So, Dr. Marshall drank a Petri-dish filled with Helicobacter pylori and developed gastritis, which was soon resolved with antibiotics.

So I started wondering how we can have a self experiment in empirical studies in software engineering? How can I report studying a development method which I followed myself and I discovered its advantages and disadvantages, and how I can overcome the biases? Such study is valuable since by observing and interviewing a practitioner who employs a method, the researcher may not capture the exact insights into that “method” under study.

On the other hand, a self experiment, even if it is unbiased and more informative, is limited to one report. It is a (sad) true fact that the best person for describing a routine method is not the person who performs it. Finally, individual differences make a self study less applicable.

That’s why I continue observing instead of drinking a Petri-dish filled with Helicobacter pylori