Test Strategy

In september I had the great pleasure of speaking at the http://agileprague.com/ conference. It was the second time I attended and I was equally pleased with the event this year. Last time I just attended while my wife gave a talk, but this year I decided to share my thoughts on Test Driven Development. The talk was mainly aimed as a broad technical introduction to the topic, but I also brushed on the concept of Test Strategy. Many developers nowadays are familiar with the concepts of Unit testing, Integration testing and Acceptance testing, but it is not always clear how these techniques should be applied in combination for the best effect. The answer to this question is of course different for each environment and therefore I find it time well spent to develop a Test Strategy. Here is an example of how such a strategy might look like in a typical layered web-application:

  1. Strive for 100% test coverage of backend-code helped by unit tests and a mocking framework
  2. Test each layer on it’s own through Unit and Integration testing
  3. Database layer tested with Integration tests
  4. Critical User Stories tested using Acceptance tests expressed in text and tables using Concordion
  5. User interface mainly tested by manual exploratory testing

Here you can find the slides: Test-Driven Development – Why, How and Strategies for Success

4 responses on “Test Strategy

  1. Is point 5 a good idea?
    Surely the manually effort of testing (regression and new functionality) the front end behaves correctly (customer point of view) will grow as the product grows.
    Automated testing under the hood is very important but surely automated testing of the UI is important too.
    I know that that UI tests can be fragile, costly and very dependent on everything else working but is there an alternative?

  2. Thanks for commenting. This is an example of a Test Strategy and not necessarily valid for all settings. However, because of precisely the reasons you say UI-testing can be expensive and a maintenance problem. My experience, which might be dated now, was that it was really painful to test JavaScript-heavy UI’s (which is often the norm nowadays for many good reasons.). What I want to say is that there is frequently a law of diminishing returns on test automation. You always have limited resources and a test strategy can help you in spending your resources wisely.

    When speaking of an alternative I believe that testability is such a fundamental quality of system that you should design for it. You should write your UI in such a way that most of the UI-logic is testable through Unit tests. When UI’s become more complex, there is no excuse for not testing Javascript as well. I found this article giving some hints:

    All this being said I think there is almost always good value in having a few system tests that go through the UI all the way, but your main effort should probably not be there. When it comes to mitigating the maintenance concerns I find the approach of the Geb framework interesting: . I haven’t used this framework myself, but I have used test code which had a UI-abstraction layer similar to the PageObject-pattern and that was very useful.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.