Testing is a topic covered with mysteries and misunderstandings. Some people believe that testing is simply a verification of a specification. Others rely on a false assumption that everything can be automated and there is no need involving test engineer into anything else than writing automated scripts. Many think that quality is a responsibility of only a few folks in the company and should not bother the rest.
My views are different to the ones stated above, as my expectations on testing are alike to other engineering disciplines, for example, programming.
On high level I am following 3 main principles:
- Testing is a creative activity.
- What can be automated should be automated.
- Quality is everyone’s responsibility.
Testing is a creative activity
Following up on the recent paper from James Bach and Michael Bolton testing is truly creative activity and should be considered as such. Testing is not a discovery of negative test cases only. Testing is looking into how else the product can or cannot be used. Generating these unthought scenarios is what makes testing creative.
There are a number of techniques that help us, humans, to judge and make decisions in uncertain times. A single use of heuristics, for example, can enlighten you to look on your product in entirely different perspective and discover issues unseen before.
What can be automated should be automated
There is no reason to repeat actions. You get bored quite quickly when doing the same steps over and over again. Therefore, investing in automated scripts is essential to free up time for testing. Only verifications are possible to get automated unless we reach a state where AI logic is as good as human judgment.
An interesting side effect of automation is that a tester gets more time for deep and skilled test session. On projects with zero or low test automation coverage tester’s time usually filled in with manual regression testing only.
The question is where to get started, on which level to implement checks. Aiming for UI/API system level automation only is a very expensive investment that will not necessarily pay off in the long run. The more components your test integrates, the bigger risk for false positive failure is created. Therefore it’s very important to look for alternative ways to verify logic in an isolated environment.
Another question is the speed of execution. System level tests are always slow as they involve multiple components. Optimizing system tests to run faster usually results covering the most of the logic on the layers below, which sometimes leads to redesigning the system for better testability.
Speed might not be the crucial factor for teams that doesn’t ship software regularly, but the ones who do need to address these issues frequently.
Quality is everyone’s responsibility
Testers are not the only ones responsible for quality. Neither are programmers. Every team member, every employee up to the CEO and/or a board member is responsible for quality. Quality is a sum of efforts put into your product or service that shall meet the customer needs.
Testers should not act as quality gatekeepers; rather encourage everyone around them to do their best in the current situation.
Summary
Testing is often confused with checking. Checking is a set of verifications that can and should be automated. On which level of your technical stack to automate them is the question that your team need to answer. Automating checks on the lowest possible level in a technical stack gives more benefits and less pain.
When the majority of checks are automated it’s easier to devote time to more focused and dedicated testing sessions. Those sessions are the ones that find the scariest bugs, the ones that automation missed.
Once bugs are identified and there is a plan to mitigate them, start thinking about preventing measures. Think about what can your team do to hinder issues from reaching your customers, stakeholders or even themselves during a development process. Everyone in the team owns quality, not a selected individual.
Thanks for reading, please check out my course The Agile Bug Hunter for more inspiration!
Thanks Andrii for your post
I agree with the 3 main principles mentioned
especially number 3.
To me it seems like Testing is always thought of after the fact.
Whats your thoughts on TDD?
http://www.agiledata.org/essays/tdd.html
Hi Ron,
I love TDD as a programming technique and evolutionary approach for designing a system. As any technique there are places where TDD can be applied easier than in others. For example it’s not that straightforward to start writing implementation test-first for the codebase which did not have any tests for years.
In my point of view when talking about Testing one should distinguish the term into two, while defining it for a particular context. There are, usually, a combination of activities people put into word ‘Testing’.
Word ‘test’, on a higher level, can mean two things: test(noun) – like a test case or automated test, and ‘to test'(verb) – as an activity, e.g. to perform testing, look for issues, defects, new verification ideas.
Depending on what’s a situation in a team, either time, budget constraints or other constrains, usually force people to neglect either one or both of these Testing dimensions. Which later result into growing number of production issues, bugs and technical debt.
Testing, both verification(running manual or automated checks) and exploration(looking for opportunities and issues) should be taken into account early on the planning stages and be included to the team’s ways or working agreements to preserve a desirable level of end product quality.
Where TDD is a good engineering practice to get a number of automated verifications(checks) at the end of implementation, it has nothing to do with the Exploratory side of Testing.