It’s always nice to look at real data and these two studies are worth their read.
- “Impact of Agile” from Rally compares the effect of WIP and estimation techniques on productivity and quality.
- “The State of Developer Productivity” by Rebel labs examines the effect of XP style techniques on quality and predictability.
Impact of Agile – Quantified
By Rally. Interesting takeaways include:
- “We can see that there is almost a 2:1 difference in throughput between teams that are 95% or more dedicated compared with teams that are 50% or less dedicated”. A strike for dedicated teams!
- Teams who do story point estimation and task breakdown (labelled “Full Scrum” in the study) have 250% better Quality than teams doing no estimating. My take on this is task breakdown forces the team to narrow down details early, rather than having them haunt them later as bugs.
- Teams who only do light weight estimation, have better overall performance considering the combined effect on productivity, predictability, quality and responsiveness. Productivity stands out as the better variable for light weight estimation teams compared with Full Scrum teams.
- Teams who aggressively control WIP – cut their lead time in half, have 1/4 as many defects, but, have 34% lower Productivity
Effect of WIP per Team member on Quality and Productivity. Source: Rally.
Interesting stuff. Read the full paper here.
The State of Developer Productivity
The “The State of Developer Productivity” study by Rebel Labs examines how XP style practices with affects predictability and quality. (I recommend reading the full paper download here).
- Code reviews have a significant impact on predictability but (surprise) minimal effect on quality. This practice likely helps developers detect major issues in design and direction but not smaller problems like bugs.
- Fixing Technical debt has a positive impact on quality and predictability.
- Automated tests showed the largest overall improvements both in the predictability and quality of software deliveries. Quality goes up most when Developers are testing the code. (see correlation below). Rock starts have 80% automated test coverage (median is 60%).
- Going from 0% to 10% automated coverage has negative effect on quality. Most likely because of overconfidence in having tests.
The effects of automated testing on predictability and quality. Source Rebel Labs.
