Here’s the thing. Suppose you introduce a change X to your workplace, and then business improves noticably. That doesn’t mean X caused the business to improve. Well, MAYBE it did. Or perhaps business improved for other reasons, and X was actually detrimental, and business would have improved even more without it. So did things work out well because of the great X, or despite the lousy X? You’ll never know, unless you could rewind the clock and play out the same scenario without X. Correlation doesn’t imply causation.
So people like me who work with organizational change, we are in the business of pseudo-science. We get customer feedback and anecdotal evidence, but we can’t actually prove that we are doing any good, it’s just opinions and observations and guesswork. I think that’s fine, but let’s be honest about it 🙂
Hi Henrik,
Statistical methods, probability, correlation studies etc. are fundamental tools in social sciences and psychology. They have the same scientific value and are studies using these methods are generally performed with the same scientific rigor as studies in natural sciences.
They also have a very practical value. Imagine there is a positive correlation between air humidity and flu infection rate during winter. If we monitor air humidity, we will be able to predict fluctuations in fly rates and act accordingly: open more assistance centers, stock pharmacies with enough flu medicine etc. as air humidity increases. There is no need to manipulate the air humidity in order to prove the correlation.
Going back to organizational change, one of truly innovative things about Kanban is that proposes management based on scientific, statistical and probabilistic methods. Traditional estimates based on Work Breakdown Structure and such are proven to be sorely lacking and other too complicated to be practical in knowledge work. As a comparison, some measurements in (a rather simple) Kanban model can prove to be of much more value and use to the organization.
For example, a correlation between long lead times and poor quality has been observed. So, if we reduce WIP, we can expect a drop in lead time and therefore improvement in quality.
What some are only now beginning to realize is that organizational change is a social phenomenon, so appropriate tools should be used to study it.
I’m sure you are aware of all I said. So, while I agree organizational change is a lot of observation, it certainly shouldn’t be guesswork or pseudoscience.
Unfortunately you are totally right!
But in an agile organization it’s all about inspect&adapt even at the playground of social and cultural changes of behavior.
So we as change agents need to be quite carefully in selecting our “experiments” and try to have good prognosis of the result/outcome.
Regards, Peter
Software engineering research doesn’t have to be pseudo-science. There are several empirical techniques for conducting these investigations in a manner that will allow one to generalize the results.
If what you mean by “science” is double-blind randomized controlled trials, then yes, it is very hard to conduct (although see Anda et al. for an attempt to do exactly this).
Properly conducted case studies (clear study propositions, theoretical grounding, well-chosen representative cases, clearly delineated contextual factors) do indeed give one evidence that something works. Technical action research, pioneered by Roel Wieringa, is another way of trying to determine cause and effect in information systems.
Proof is too high a standard to meet in these cases. What we should seek is evidence, and some theoretical basis for why it should work (e.g., short iterations lead to quicker feedback and happier customers). And of course, replicate those findings as much as possible.
You are quite lucky. In Sweden the universities are very strong in empirical SE. You might look at Lund (Bjorn Regnell) or BTH (Tony Gorshek) for example.
Great observation! But correlation results are not completely worthless as causality implies correlation. But, yeah, we need to be very humble in interpreting those results.