Continue reading: Breaking up the monolith at Discovery and enhancing subscription flexibility in one fell swoop

Breaking up the monolith at Discovery and enhancing subscription flexibility in one fell swoop

No developers were hurt in the making of this interview

Max – Tell me briefly what your team is up to @ Discovery?

I’m one member of a devteam which takes care of external API’s. We build services for streaming TV (OTT), for example user management, authentication, user entitlements and playback services.

We started this effort to enable a more flexible way to assign entitlements to users, for example through affiliates (like cable TV companies and mobile phone operators). Before, you had to be a subscriber at Discovery to be able to view any content. Being a direct subscriber is called the DTC model, ”Direct To Consumer”. But a large part of the business in some markets still comes from the so called “TV Everywhere” model where the customers come in via affiliates.

Continue reading
Continue reading: How to deploy a microservice application to AWS

How to deploy a microservice application to AWS

In this post, I will use an example to show how you can:

  • Write a small microservice with a REST API in Java, using the Dropwizard framework
  • Containerize the application using Google JIB
  • Deploy the application to AWS using Ansible
    • With a robust, clustered and self-healing setup

Continue reading

Continue reading: Readable Java system tests with good old JUnit

Readable Java system tests with good old JUnit

This article is the third in a series about system testing:

  1. Dockerized testing vs end-to-end testing
  2. How to setup Dockerized testing
  3. Readable Java system tests with good old JUnit

JUnit is poorly named. Given the name, people tend to think that it should only be used to write Java unit tests. And then people feel a bit hesitant about writing their integration tests with JUnit too. When they start with system tests, they often think they need another driver for their tests. Sure, maybe using another abstraction layer and a custom domain specific language (BDD), you can make the tests more readable for a non-programmer. That often comes at the cost of making the tests less readable for the programmers. And if we are honest, who’s going to read the tests the most? Perhaps just naming test classes and methods well and writing readable code can suffice?

Continue reading

Continue reading: How to setup Dockerized testing

How to setup Dockerized testing

This article is the second in a series about system testing:

  1. Dockerized testing vs end-to-end testing
  2. How to setup Dockerized testing
  3. Readable Java system tests with good old JUnit

You may want to read Dockerized testing vs end-to-end testing first.

How to setup Dockerized testing?

A prerequisite for running your tests against a dockerized environment is of course that the service you want to test is packaged as a Docker image. Also, all of the services which the System Under Test (SUT) depends on needs to be dockerized. If you don’t have that, your environment is not fully dockerized and you will suffer the consequences of having to test against external services.

If your SUT is packaged as a Docker image and has no dependencies on other services, you can just start it using “docker run” and run your tests against it. However, usually services have one or more dependencies to other services, such as its own database, a queue and perhaps a external RESTish service that returns nasty XML. You want each of these services in their own container. To spin up a bunch of docker containers, you can use docker-compose.

Continue reading

Continue reading: Dockerized testing vs end-to-end testing

Dockerized testing vs end-to-end testing

I recently rewrote the end-to-end tests for a service. This brought up the question of what we should test in the end-to-end (e2e) tests vs our Dockerized tests.

This article is the first in a series about system testing:

  1. Dockerized testing vs end-to-end testing
  2. How to setup Dockerized testing
  3. Readable Java system tests with good old JUnit

e2e tests vs dockerized tests

Since the original e2e tests were written, we (my team) have also introduced another type of tests. We call them “dockerized system tests”. These are similar to the e2e tests. They also test the connections between services in the system, and how the system works as a whole. One key difference between e2e tests and dockerized tests is that the dockerized tests do not test the system when it is deployed the way we deploy it in staging and production environments. Our dockerized tests bring up all services in a single machine, and we usually don’t set up redundancy, clustering, load balancers and such things.

Continue reading

Continue reading: Elm for backend developers

Elm for backend developers

Before there was the concept of frontend and backend developers, there were just developers and I was one of them. We did everything from HTML to SQL and it was not hard. Then something happened, JavaScript became popular. After years of server side rendering and thin clients, the pendulum was swinging back to fat clients again. And I was a backend developer.

Believe me, I have tried to get on the train with JavaScript and its hose of new frameworks. I took courses in JavaScript. I wrote applications using Meteor, tried Angular and React. But what I never managed to do, is liking it. There is simply to many ways of shooting yourself in the foot. And all these frameworks that come and go points to something, perhaps we are trying to fix the wrong problem. I suspect it is the foundation itself, JavaScript.

Continue reading

Continue reading: How to structure Ansible variables when provisioning infrastructure

How to structure Ansible variables when provisioning infrastructure

How should you structure your Ansible variables in playbooks calling cloud modules? Ansible has extensive support for variables at different levels (there are 21 different levels!).

The inventory group_vars and host_vars offer enough flexibility for many use cases. However, when you are setting up cloud infrastructure, you don’t really have any hosts yet. Maybe you are setting up load balancers, instances, security groups and such things. Perhaps you want to setup the same things in several different environments (like “staging” and “production”). You want to re-use the same playbook, but adapt what it does by using different variable values in different environments.

In this post, I will demonstrate a way to structure Ansible variables in a playbook which pretends to setup a AWS AutoScalingGroup. I will show how you can separate the configuration for different environments by using a inventory directory. As a bonus, I will give you a neat trick to automatically load a extra variable file for each environment. We will use this to load a separate secrets file which is encrypted with git-crypt.

Continue reading

Continue reading: How to run Ansible tasks in parallel

How to run Ansible tasks in parallel

When you use Ansible to provision hosts, running a task on a set of hosts in parallel is not a problem. You simply use the `serial` keyword in a playbook. See: http://docs.ansible.com/ansible/latest/playbooks_delegation.html#rolling-update-batch-size What about when you want to run a task several times in parallel, either on each host, or on a single host? A

Continue reading
Continue reading: Three “no brainer” engineering practices for developers

Three “no brainer” engineering practices for developers

In modern software development there are three development practices that everyone should strive to apply:

  • Automated testing
  • Pair or mob programming
  • TDD, test driven development

After many years of using and researching these practices in the development community there is no longer any question whether these engineering practices bring value or not – they do. It’s not a matter of opinion, it’s a matter of fact. We know that now.

Continue reading

Continue reading: 7 Misconceptions about TDD

7 Misconceptions about TDD

Here are some common misconceptions about TDD. I call them “myths” here, for short.

If this feels like talking to the dentists about your teeth, you are not alone. When I talk about tests sometimes people gets embarrassed about their habits, “I know you’re right but …”.

Continue reading

Continue reading: Security Test-Driven Development – Spreading the STDD-virus

Security Test-Driven Development – Spreading the STDD-virus

Agile development with short release cycles have been here for a while now. Most of us want fast feedback loops and many even Continuous Delivery with changes in production software everyday. However, most of us also want secure software and the question is: Can security engineering keep up the pace? A fast feedback that your production website has been hacked is not so nice.

Security is a quality attribute of your software, just like performance. If you don’t want to be surprised by bad performance in production, what do you do? You test and design for it of course and you preferably do so continuously from the start.

In my experience, the same however cannot be said of security. It is very often relegated to a once a year penetration-test activity. Not really an agile way of working is it? Not a secure one either since untested software is released as often as everyday. There must be a better way of working which allows us to both work in an agile way and to verify security on the way.

In the security field people like Gary McGraw have long been advocating ways of “Building Security In”. The Microsoft MVP Troy Hunt also proposes that you should “Hack yourself first”, instead of just waiting for the pentesters. Shouldn’t it be possible to weave these security activities into the process the same way as it is possible with normal testing activities using TDD? Indeed I, as well others believe it is so. Let’s look at how small extensions to an agile process can work in this direction.

Extending Sprint planning to deal with security

To start off you must first know what the requirements are. In a normal agile project this is done by eliciting User Stories from the customer or the Product Owner.

Let’s take an example of an online e-Commerce site. A User Story might be “As a customer I want to be able to add a review of a product so that information about products can be shared between customers”.

This works very well for traditional functional requirements, but for non-functional requirements a little extra thought is needed. In the case of security requirements it is often useful to state a requirement in a scenario that should NOT happen. In our case we shall call these scenarios “Abuser Stories”. These stories are non-technical descriptions of bad things you want to make sure you avoid. An Abuser story for this site might be:

“An attacker uses the Review Product-function to spread malicious Javascript”. Another might be: “An attacker abuses the Review Product-function to gain unlimited access to the database”.
A Product Owner might not be able to come up with these stories himself, but might need the help of a security engineer to help him with finding these threat scenarios.

SecurityTesting
Continue reading

Continue reading: Programmer productivity: SP < PR < PP < MP

Programmer productivity: SP < PR < PP < MP

In my experience, when it comes to programming productivity, mob programming beats the rest. Of course the definition of productivity in this context is debatable and these are just my observations. Thus, it is not a proper scientific study but bear with me anyway.

I wish to compare one aspect of productivity, how we work together. I look at single programming, pull requests, pair programming and mob programming.

Continue reading

Continue reading: How to append to lists in Ansible

How to append to lists in Ansible

Since I have found the Ansible documentation to be lacking, and StackOverflow insufficient in this matter, I feel the need to share how you can append to a list using Ansible.

I’ve created a demonstration playbook and published it on GitHub. See: https://github.com/betrcode/ansible-append-list

Continue reading

Continue reading: Mocka med inhoppare – del 6 i TDD på svenska

Mocka med inhoppare – del 6 i TDD på svenska

TDD är en vana man behöver etablera. Första hindret är ofta när man återvänder efter kurs att omsätta teori i praktik. I detta avsnitt berätta jag om hur man isolerar en enhet så att man kan testa den. Del 6 är i serien “TDD på svenska” handlar alltså om detta och har precis nu publicerats

Continue reading
Continue reading: Think twice before logging

Think twice before logging

balloon_irrelevant

One property of legacy code is inflation by irrelevant logging statements. Not only does this increase the size of a bulging code base, I’d also argue that it’s dead wrong.

Quite recently I’ve had the honor of making acquaintance with a piece of code that looked roughly like this:

if (LOGGER.isLoggable(Level.FINEST)) {
    LOGGER.log(Level.FINEST, "foo is now", foo.getValue());
}
boolean result = doSomeActualWork(foo);
if (LOGGER.isLoggable(Level.FINER)) {
    LOGGER.exiting(this.getClass().getName(), "bar", result);
}

Continue reading

Continue reading: Programming with kids & co-speaking with my son :)

Programming with kids & co-speaking with my son :)

Yesterday me and Dave (11 yrs) spoke together for the first time! We did a public talk at Spotify about how to help kids learn to program. We’ve been experimenting a lot with that in my family (4 kids to experiment with… muahahaha), and wanted to share some learnings. Worked out better than we could have hoped, considering all the risky tech demos and live coding involved 🙂

Shared the stage with teacher Frida Monsén who talked about how to get this kind of stuff into schools. Thanks Helena Hjertén for organizing this, and Spotify for hosting & sponsoring. Here’s an article in Computer Sweden about this event and our little “mod club”.

Here are the slides! They are in Swedish though. Might do an English version of this talk some day 🙂

Dave on Stage

Continue reading

Continue reading: 7 Rules on Code Readability

7 Rules on Code Readability

What makes good code? Many things but whatever the qualities are, readability is a cornerstone.  If you can’t read the code, you can’t fix it. So how do you write readable code? I’ll give you my view but it’s like books, what I find enjoyable may be different from you.

Continue reading

Continue reading: About the Developer Profession #1: How Many Decisions Do You Make in Real Time?

About the Developer Profession #1: How Many Decisions Do You Make in Real Time?

In this series of articles, I’ll be discussing the developer profession from different angles. The common denominator is that all articles will, in one way or another, be about professionalism. This first article is about the different factors and decisions behind every single line of code.

Notes snd Equations

A while ago I spoke to a friend who is pretty much a professional singer. He explained singing to me in a very passionate way, saying that singing is like solving multiple parallel equations in real time. He told me that it’s quite obvious that every singer has to follow notes. What’s less obvious is that the singer also takes a multitude of micro decisions during every second of his/hers singing. He told me about tempo, intensity, interpretation of the composer, matching the expectations of the audience, synchronization with other singers, following the Kapellmeister, and a bunch of other factors. I was quite impressed.

A second later it struck me that this goes for programming as well, and I started to list all the equations a developer must solve while writing a single line of code. Here it is!

Continue reading

Continue reading: Alla mjukvaruprojekt borde ha en Kodkvast!

Alla mjukvaruprojekt borde ha en Kodkvast!

– “All kod är en skuld!”
– “Den bästa koden är den kod man inte skriver!”
– “Snabbaste sättet att få upp testtäckningen är att ta bort död kod!”

Jag tror att alla är överens om att det är bra att hålla sin kodbas så liten och kompakt som möjligt. Byggtider hålls korta, testsviter går fort att genomföra, driftsättningar går fort, statisk kodanalys går fort, nya teammedlemmar kommer snabbt in i koden, risken för buggar och sårbarheter hålls nere och så vidare. Kort sagt, man blir mer lättrörlig!

Så alla utvecklingsteam borde ägna tid åt att inte bara skriva ny kod, utan också att faktiskt städa efter sig. Men det är lättare sagt än gjort…
Continue reading

Continue reading: 12 år med TDD

12 år med TDD

12år

Om det känns lockande att fördjupa sig inom olika varianter och vinklingar av TDD, säg till, så ordnar vi ett TDD-kvällsevent här på Crisp.

Continue reading

Continue reading: Test Strategy

Test Strategy

In september I had the great pleasure of speaking at the http://agileprague.com/ conference. It was the second time I attended and I was equally pleased with the event this year. Last time I just attended while my wife gave a talk, but this year I decided to share my thoughts on Test Driven Development. The talk was

Continue reading
Continue reading: Another builder pattern for Java

Another builder pattern for Java

Whenever you have a domain object, data transfer object, parameter object or any other object that can’t be instantiated with constructor parameters only, you need to create a builder for its class.

The great thing about Java is that it is strongly (statically) typed so a builder can give you compiler errors if you forget to set the compulsory parameters. If you use the builder in this pattern, you also get a very fluent interface that imposes order on the parameters. When using the builder, your IDE will suggest the next parameter to set.

Continue reading

Continue reading: The proper use of @Ignore in unit testing

The proper use of @Ignore in unit testing

What is the purpose of the @Ignore annotation in unit testing?

Either a test will be green and everything is fine or it is red so you fix it before commit. So if there is an ignored test, it is the same thing as dead code and we know what to do with that: delete it! Keep the code clean!Continue reading

Continue reading: Hur man kan hantera Continuous Delivery med MongoDB

Hur man kan hantera Continuous Delivery med MongoDB

MongoDB är en schemalös, dokumentorienterad databas som har fått stor popularitet i den agila världen bland annat därför att man inte behöver underhålla något databasschema.

MongoDBs schemalöshet gör att många leds att tro att Continuous Delivery blir en promenad i parken, eftersom det ju inte behövs några datamigreringar när man driftsätter en ny version av koden!

Rent teoretiskt är detta sant, men är ett sluttande plan in i Land of Crappy Code™ !

För att slippa onödig komplexitet i form av varierande utseende på lagrade domänobjekt beroende på deras ålder, rekommenderar jag att man utför regelrätta datamigreringar även när man använder MongoDB!

Jag rekommenderar även att datamigreringen är en del av applikationen — till skillnad från skript som skall köras vid sidan av innan applikationsstart — helt enkelt för att eliminera risken för misstag.

Jag har i mitt sidoprojekt Varmfront.nu utvecklat en kompakt liten lösning som i MongoDB implementerar det som Flyway gör för SQL.

Mönstret bygger på Spring Data for MongoDB och Spring JavaConfig, och migreringarna är skrivna i Java. That’s right folks, no XML here 😀

Läs vidare, så får du se hur man kan göra!

Continue reading

Continue reading: Continuous delivery – The simplest possible build pipeline for an integration scenario

Continuous delivery – The simplest possible build pipeline for an integration scenario

Sometimes a continuous integration/delivery scenario is more complex than just building a system in a multi-stage pipeline. The system may consist of several subsystems, or just complex components, each of which requires a build pipeline of their own. Once all systems pass through their respective build pipelines they are integrated together and subjected to a joint deployment and further testing. When facing such a scenario, I decided to build the simplest possible thing that would work and get the job done.

Two converging build pipelines

Continue reading

Continue reading: Slacker? No, hacker!

Slacker? No, hacker!

Last friday, I had the day off. Again.

Code
Yes, there were some code!

This was just one of several days that I have had off, not counting weekends and vacations. So am I just a slacker that don’t work 5 five days a week? No, on the contrary! This makes me a better programmer. How? Read on.

Continue reading

Continue reading: By code thou shall present! ITAKE

By code thou shall present! ITAKE

Can you run a conference where every presenter has to present using code? Why not! Meet ITAKE (Bucharest, ay 30-31). A coders delight. Our friends in Eastern Europe, Mozaic are trying out a new conference format where each presenter has to present using code.  Gotta love initatives like that 🙂      

Continue reading
Continue reading: Still not automating tests? Here’s why you should (again)!

Still not automating tests? Here’s why you should (again)!

The other day I read a blog by Uncle Bob. It more or less stated that no matter what situation you are in, writing automated tests will make you go faster. Ok, this is old news I thought, until I checked Uncle Bob’s tweets. A fair amount of people argued against this statement, and that surprised me!

Campfire
Join me at the campfire!

So I started thinking about why there still are fellow software developers that doesn’t believe in automated testing? Have they not seen them in action and understood what they are for? Please, gather around the campfire, and I will tell you one, just one, of my war stories, and then I will tell you why also you should write automated tests!

Continue reading

Continue reading: Where is that Red ‘Stop’ Button in Your Development Process?

Where is that Red ‘Stop’ Button in Your Development Process?

If you don’t dare to stop the line, continuous integration might be waste. Here is the second part of my three-part series on building the quality in on the SmartBear blog. In the first post of this series, I wrote about Toyoda Sakichi, the founder of the Toyota industries, who invented a loom that would

Continue reading
Continue reading: Stop the Line – Build Quality In with Incremental Compilation

Stop the Line – Build Quality In with Incremental Compilation

We in the software industry are still far behind when it comes to automated quality checks. Toyoda Sakichi for example invented the automated loom with stop the line capability almost 100 years ago. I write more about that in my first blog in a three-part series on building the quality in on the SmartBear blog.

Continue reading