Yesterday we ended our second two-week sprint and on the demo, besides showing the system, we could do some bragging about test coverage using our Sonar dashboard. We also could show Fitnesse tests at system level that implements the specification by example technique, or like some say, executable requirements.
Firstly, this is a bit of tongue-in-cheek. I am proud of what we done but the bragging part is to be taken as inspiration and not me being closer to programming heaven than anyone else.
Here is what we bragged about:
- From the Sonar dashboard:
- Branch test coverage by unit tests: 100%
- Rule compliance: 100%
- LCOM4 cohesion: 1.1/class
- Pair programming: 100%
- Test driven development: 100%
- Specification driven development (ATDD): 100%
We were so hard about unit test coverage that the build was set to fail if coverage dropped below 100%.
We measured also the test coverage when running the Fitnesse tests. We verified that no service interface had less than full coverage so that every feature in the code was motivated by the specification. This was a manual step in that it involved some judgment. Some service methods are solely used by the fixtures, which is fine.
The reason for this, is that it is very easy to add a feature “while you’re at it” so this was a check that it didn’t happen without correction. The Fitnesse tests should tell the whole truth. The first column on our Scrum board after “To do” is “Write Fitnesse tests”, but that does not stop you completely.
Did it not slow you down to have 100% branch coverage?
This is hard to answer as I know of no good measure of developer speed. In terms of tested LOC we are fine, but LOC is not a good measurement. In terms of gut feeling, no, it did not.
What I do know is that the lengths of feedback loops increase the cost of fixing a problem exponentially. We shortened them by pair programming, unit tests and automatic system tests. If you have none of those practices, you will be significantly slower.
During the two sprints, we had to refactor our domain model as we learned more about it. If we hadn’t had tests backing us, we probably would had skipped it as it would have been seen as too risky, biting the bullet of a less than optimal domain model.
Is test coverage the absolute truth about code quality?
No. Testing is about mitigating risks so your code may be correct in all senses without any tests. Full coverage doesn’t guarantee anything as the tests may be wrong as well. But it is like bookkeeping, you say the same thing twice and where there is a discrepancy between the two, there is a mistake. So your chances are a lot better.
What you don’t want is your coverage going down over time, that is a sign of increasing technical debt. Same goes for other parameters such as violations to coding rules. Sonar is very versatile in comparing measurements over time and showing trends. It is free, as all tools we use.
My boss will not give me time to write tests, what should I do?
Your boss has no say over how to conduct professional programming, even if being an ex-developer. Things are moving on and we must act on the experiences we have had, e.g. increasingly expensive to add features to a system due to increasing technical debt.
Your boss will for sure come running when customers complain about bugs. So you’re damned if you do and damned if you don’t.
Did you deliver anything useful?
Yes, two applications, one for students taking examinations and one for assessing them. Not that anyone will use them now in production; we can still get important feedback.
Lucky you with a green field project, come back in six months and compare!
I am looking forward to that, being very curious on how we will keep up with changing circumstances.
Most of us developers sit with a legacy code base, considering ourselves less than lucky. But you just have to dig where you stand. Just make sure that you are improving, one step at the time.
I bet you had only senior developers with lots of guts to pull this one of!
No. I am senior but the other three are fresh from school. It is the other way around, fresh minds think that anything is possible and are open to suggestions. Where we are, it didn’t take guts.
How can I learn to program like your team does?
Join me and Henrik Kniberg for a hands-on course, Certified Scrum Developer. Watch this space for an update on the next installment.
There you will learn practices such as these and more.
The link is broken http://www.crisp.se/kurser/certified-scrum-developer-september-stockholm
Sorry about the broken link. We had to postpone the course due to the low interest.
In practical I find it’s impossible to have a 100% coverage without adding silly unit test for trivial class (get/set, glueing class). Mind sharing your approach?
If the setters and getters aren’t used by any other class, why are they there? Tests are specifications of desired behavior so somehow the simple things get tested as well, due to higher order tests. Typically the tests for a persistent entity does not cover it fully but the tests for a service using it, may give coverage to it.