Continue reading: Readable Java system tests with good old JUnit

Readable Java system tests with good old JUnit

This article is the third in a series about system testing:

  1. Dockerized testing vs end-to-end testing
  2. How to setup Dockerized testing
  3. Readable Java system tests with good old JUnit

JUnit is poorly named. Given the name, people tend to think that it should only be used to write Java unit tests. And then people feel a bit hesitant about writing their integration tests with JUnit too. When they start with system tests, they often think they need another driver for their tests. Sure, maybe using another abstraction layer and a custom domain specific language (BDD), you can make the tests more readable for a non-programmer. That often comes at the cost of making the tests less readable for the programmers. And if we are honest, who’s going to read the tests the most? Perhaps just naming test classes and methods well and writing readable code can suffice?

Continue reading

Continue reading: How to setup Dockerized testing

How to setup Dockerized testing

This article is the second in a series about system testing:

  1. Dockerized testing vs end-to-end testing
  2. How to setup Dockerized testing
  3. Readable Java system tests with good old JUnit

You may want to read Dockerized testing vs end-to-end testing first.

How to setup Dockerized testing?

A prerequisite for running your tests against a dockerized environment is of course that the service you want to test is packaged as a Docker image. Also, all of the services which the System Under Test (SUT) depends on needs to be dockerized. If you don’t have that, your environment is not fully dockerized and you will suffer the consequences of having to test against external services.

If your SUT is packaged as a Docker image and has no dependencies on other services, you can just start it using “docker run” and run your tests against it. However, usually services have one or more dependencies to other services, such as its own database, a queue and perhaps a external RESTish service that returns nasty XML. You want each of these services in their own container. To spin up a bunch of docker containers, you can use docker-compose.

Continue reading

Continue reading: Dockerized testing vs end-to-end testing

Dockerized testing vs end-to-end testing

I recently rewrote the end-to-end tests for a service. This brought up the question of what we should test in the end-to-end (e2e) tests vs our Dockerized tests.

This article is the first in a series about system testing:

  1. Dockerized testing vs end-to-end testing
  2. How to setup Dockerized testing
  3. Readable Java system tests with good old JUnit

e2e tests vs dockerized tests

Since the original e2e tests were written, we (my team) have also introduced another type of tests. We call them “dockerized system tests”. These are similar to the e2e tests. They also test the connections between services in the system, and how the system works as a whole. One key difference between e2e tests and dockerized tests is that the dockerized tests do not test the system when it is deployed the way we deploy it in staging and production environments. Our dockerized tests bring up all services in a single machine, and we usually don’t set up redundancy, clustering, load balancers and such things.

Continue reading

Continue reading: Automated testing is never enough

Automated testing is never enough

In the pursuit to automate testing to create faster feedback loops and build quality in, some teams forget the value of manual testing. In my experience, without manual testing (as well) we are toast.

Continue reading

Continue reading: 7 Misconceptions about TDD

7 Misconceptions about TDD

Here are some common misconceptions about TDD. I call them “myths” here, for short.

If this feels like talking to the dentists about your teeth, you are not alone. When I talk about tests sometimes people gets embarrassed about their habits, “I know you’re right but …”.

Continue reading

Continue reading: Security Test-Driven Development – Spreading the STDD-virus

Security Test-Driven Development – Spreading the STDD-virus

Agile development with short release cycles have been here for a while now. Most of us want fast feedback loops and many even Continuous Delivery with changes in production software everyday. However, most of us also want secure software and the question is: Can security engineering keep up the pace? A fast feedback that your production website has been hacked is not so nice.

Security is a quality attribute of your software, just like performance. If you don’t want to be surprised by bad performance in production, what do you do? You test and design for it of course and you preferably do so continuously from the start.

In my experience, the same however cannot be said of security. It is very often relegated to a once a year penetration-test activity. Not really an agile way of working is it? Not a secure one either since untested software is released as often as everyday. There must be a better way of working which allows us to both work in an agile way and to verify security on the way.

In the security field people like Gary McGraw have long been advocating ways of “Building Security In”. The Microsoft MVP Troy Hunt also proposes that you should “Hack yourself first”, instead of just waiting for the pentesters. Shouldn’t it be possible to weave these security activities into the process the same way as it is possible with normal testing activities using TDD? Indeed I, as well others believe it is so. Let’s look at how small extensions to an agile process can work in this direction.

Extending Sprint planning to deal with security

To start off you must first know what the requirements are. In a normal agile project this is done by eliciting User Stories from the customer or the Product Owner.

Let’s take an example of an online e-Commerce site. A User Story might be “As a customer I want to be able to add a review of a product so that information about products can be shared between customers”.

This works very well for traditional functional requirements, but for non-functional requirements a little extra thought is needed. In the case of security requirements it is often useful to state a requirement in a scenario that should NOT happen. In our case we shall call these scenarios “Abuser Stories”. These stories are non-technical descriptions of bad things you want to make sure you avoid. An Abuser story for this site might be:

“An attacker uses the Review Product-function to spread malicious Javascript”. Another might be: “An attacker abuses the Review Product-function to gain unlimited access to the database”.
A Product Owner might not be able to come up with these stories himself, but might need the help of a security engineer to help him with finding these threat scenarios.

SecurityTesting
Continue reading

Continue reading: Scientific method applied to performance improvements

Scientific method applied to performance improvements

build-measure-learn-loop In my team, we are working on improving real-time performance for our main service. The goal is to have response times below 100 ms in the 95th percentile and below 200 ms in the 99th percentile for certain database volumes and request frequencies.

We don’t know what will be needed to reach this goal. We have some ideas, but we don’t know which one, or which ones will do the trick. We call these ideas “experiments”.

We can estimate each experiment, but we don’t know how many we will need to do to reach the goal.

This is the story of how we apply the scientific method to working with performance improvements.

Continue reading

Continue reading: Slides from Agile Testing Day Scandinavia

Slides from Agile Testing Day Scandinavia

In this talk I presented a simple 2D platformer written in Java/Groovy and how to use Spock to test it. I’ll make the source code available in a while. By the way, of you’re not using Spock yet, then start!

Continue reading
Continue reading: Slides from JDays

Slides from JDays

I talked about some overlooked (and quite new) features of JUnit at JDays in Gotherburg. My presentation was a tour that started with boring example tests, and then proceeded through parameterized tests, theory tests, and generative testing. At the end of this tour, I spoke about fundamental, yet again overlooked, testing techniques such as finding

Continue reading
Continue reading: Slides från SAST Stockholm Q4: Tema agilt

Slides från SAST Stockholm Q4: Tema agilt

Igår hade jag äran att få gästa SAST Stockholm Q4, där jag fick hålla en presentation om utvecklartestning. Med handen på hjärtat, så blev det lite mycket information på få tidsenheter ibland. Dock brinner jag verkligen för ämnet och vill säga så mycket jag kan. Efter att ha checkat runt lite, gläds jag åt att

Continue reading
Continue reading: Seminar “Will automation kill the tester?”

Seminar “Will automation kill the tester?”

Last Thursday I gave a two hour talk on Agile Testing, “Will automation kill the tester?”, for Dataföreningens network for test and requirements in Stockholm. I talked about testing in agile contexts, the agile tester, busted (and approved) some myths on agile testing, and gave examples – lots of them. How different companies, project and

Continue reading
Continue reading: Continuous delivery – The simplest possible build pipeline for an integration scenario

Continuous delivery – The simplest possible build pipeline for an integration scenario

Sometimes a continuous integration/delivery scenario is more complex than just building a system in a multi-stage pipeline. The system may consist of several subsystems, or just complex components, each of which requires a build pipeline of their own. Once all systems pass through their respective build pipelines they are integrated together and subjected to a joint deployment and further testing. When facing such a scenario, I decided to build the simplest possible thing that would work and get the job done.

Two converging build pipelines

Continue reading

Continue reading: Where is that Red ‘Stop’ Button in Your Development Process?

Where is that Red ‘Stop’ Button in Your Development Process?

If you don’t dare to stop the line, continuous integration might be waste. Here is the second part of my three-part series on building the quality in on the SmartBear blog. In the first post of this series, I wrote about Toyoda Sakichi, the founder of the Toyota industries, who invented a loom that would

Continue reading
Continue reading: Stop the Line – Build Quality In with Incremental Compilation

Stop the Line – Build Quality In with Incremental Compilation

We in the software industry are still far behind when it comes to automated quality checks. Toyoda Sakichi for example invented the automated loom with stop the line capability almost 100 years ago. I write more about that in my first blog in a three-part series on building the quality in on the SmartBear blog.

Continue reading
Continue reading: The Future of Software Development

The Future of Software Development

Whar are YPU doning in the future?

What will software development be like in the future? “Agile” as we know it, will not be around, nor will test-driven development, continuous delivery, or BDD-like methodologies. I’ve been pondering this for a while, and based on some observations and a dose of wishful thinking, I’ve arrived at the conclusion above. Do you agree?

Continue reading

Continue reading: Country Ambassador for Agile Testing Days 2012

Country Ambassador for Agile Testing Days 2012

A while ago I was asked to become one of the Swedish country ambassadors for the Agile Testing Days 2012 conference. I said yes, because I think it’s a great conference. As country ambassador, I help in promoting the conference. I chose to do it, because I think it’s a good conference and I already recommend it to my friends.
Continue reading

Continue reading: Min första kurs

Min första kurs

Jag har precis lagt upp min första kurs här på Crisp – Testning av webbapplikationer med Selenium WebDriver. I detta blogginlägg tänkte jag förklara lite mer ingående vad kursen är tänkt att lära ut. Mina kolleger har också kommit med värdefull feedback och frågor som säkert kan dyka upp igen.
Continue reading

Continue reading: The link between testability and object-orientation

The link between testability and object-orientation

What’s testability? And why does object-oriented code feel more testable? In this post I argue, somewhat informally, that making code object-oriented, or just by introducing more accurate abstractions, we decrease the domain-to-range ratio, thus making our code more testable.

Continue reading