Continue reading: How to structure Ansible variables when provisioning infrastructure

How to structure Ansible variables when provisioning infrastructure

How should you structure your Ansible variables in playbooks calling cloud modules? Ansible has extensive support for variables at different levels (there are 21 different levels!).

The inventory group_vars and host_vars offer enough flexibility for many use cases. However, when you are setting up cloud infrastructure, you don’t really have any hosts yet. Maybe you are setting up load balancers, instances, security groups and such things. Perhaps you want to setup the same things in several different environments (like “staging” and “production”). You want to re-use the same playbook, but adapt what it does by using different variable values in different environments.

In this post, I will demonstrate a way to structure Ansible variables in a playbook which pretends to setup a AWS AutoScalingGroup. I will show how you can separate the configuration for different environments by using a inventory directory. As a bonus, I will give you a neat trick to automatically load a extra variable file for each environment. We will use this to load a separate secrets file which is encrypted with git-crypt.

Continue reading

Continue reading: Security Test-Driven Development – Spreading the STDD-virus

Security Test-Driven Development – Spreading the STDD-virus

Agile development with short release cycles have been here for a while now. Most of us want fast feedback loops and many even Continuous Delivery with changes in production software everyday. However, most of us also want secure software and the question is: Can security engineering keep up the pace? A fast feedback that your production website has been hacked is not so nice.

Security is a quality attribute of your software, just like performance. If you don’t want to be surprised by bad performance in production, what do you do? You test and design for it of course and you preferably do so continuously from the start.

In my experience, the same however cannot be said of security. It is very often relegated to a once a year penetration-test activity. Not really an agile way of working is it? Not a secure one either since untested software is released as often as everyday. There must be a better way of working which allows us to both work in an agile way and to verify security on the way.

In the security field people like Gary McGraw have long been advocating ways of “Building Security In”. The Microsoft MVP Troy Hunt also proposes that you should “Hack yourself first”, instead of just waiting for the pentesters. Shouldn’t it be possible to weave these security activities into the process the same way as it is possible with normal testing activities using TDD? Indeed I, as well others believe it is so. Let’s look at how small extensions to an agile process can work in this direction.

Extending Sprint planning to deal with security

To start off you must first know what the requirements are. In a normal agile project this is done by eliciting User Stories from the customer or the Product Owner.

Let’s take an example of an online e-Commerce site. A User Story might be “As a customer I want to be able to add a review of a product so that information about products can be shared between customers”.

This works very well for traditional functional requirements, but for non-functional requirements a little extra thought is needed. In the case of security requirements it is often useful to state a requirement in a scenario that should NOT happen. In our case we shall call these scenarios “Abuser Stories”. These stories are non-technical descriptions of bad things you want to make sure you avoid. An Abuser story for this site might be:

“An attacker uses the Review Product-function to spread malicious Javascript”. Another might be: “An attacker abuses the Review Product-function to gain unlimited access to the database”.
A Product Owner might not be able to come up with these stories himself, but might need the help of a security engineer to help him with finding these threat scenarios.

SecurityTesting
Continue reading

Continue reading: How to append to lists in Ansible

How to append to lists in Ansible

Since I have found the Ansible documentation to be lacking, and StackOverflow insufficient in this matter, I feel the need to share how you can append to a list using Ansible.

I’ve created a demonstration playbook and published it on GitHub. See: https://github.com/betrcode/ansible-append-list

Continue reading

Continue reading: The worlds fastest CI

The worlds fastest CI

Viaplay broadcasts TV over the web and to wireless devices. No need to wait for another build.. they have the worlds fastest CI! Let me share with you how it works! 1. State normal – everything works We can code along or have another coffee. 2. Big trouble! Mainline is clearly broken! All hands on

Continue reading
Continue reading: Schema migration, an important part of continuous delivery

Schema migration, an important part of continuous delivery

Most of you have a relational database involved in your persistence. As soon as your first version has been set in use, you can’t change the database schema as easy anymore or you might lose valuable production data.

At the same time, continuous delivery demands that there should be as few manual steps as possible. See here for motivation on continuous delivery.

You no longer have a few database instance, there are numerous for different levels of testing and every developer nowadays run a personal database. To keep track of all database instances and keep them updated becomes a steep task.

To tackle this, we started using Flyway as a tool to manage our database scripts. Our applications migrates the database automatically on startup so we get a hands-free solution that will guarantee that the code and database schema is in sync.

Here is a technical recipe for accomplishing this when using Flyway in combination with Spring.
Continue reading

Continue reading: Hur man kan hantera Continuous Delivery med MongoDB

Hur man kan hantera Continuous Delivery med MongoDB

MongoDB är en schemalös, dokumentorienterad databas som har fått stor popularitet i den agila världen bland annat därför att man inte behöver underhålla något databasschema.

MongoDBs schemalöshet gör att många leds att tro att Continuous Delivery blir en promenad i parken, eftersom det ju inte behövs några datamigreringar när man driftsätter en ny version av koden!

Rent teoretiskt är detta sant, men är ett sluttande plan in i Land of Crappy Code™ !

För att slippa onödig komplexitet i form av varierande utseende på lagrade domänobjekt beroende på deras ålder, rekommenderar jag att man utför regelrätta datamigreringar även när man använder MongoDB!

Jag rekommenderar även att datamigreringen är en del av applikationen — till skillnad från skript som skall köras vid sidan av innan applikationsstart — helt enkelt för att eliminera risken för misstag.

Jag har i mitt sidoprojekt Varmfront.nu utvecklat en kompakt liten lösning som i MongoDB implementerar det som Flyway gör för SQL.

Mönstret bygger på Spring Data for MongoDB och Spring JavaConfig, och migreringarna är skrivna i Java. That’s right folks, no XML here 😀

Läs vidare, så får du se hur man kan göra!

Continue reading

Continue reading: Continuous delivery – The simplest possible build pipeline for an integration scenario

Continuous delivery – The simplest possible build pipeline for an integration scenario

Sometimes a continuous integration/delivery scenario is more complex than just building a system in a multi-stage pipeline. The system may consist of several subsystems, or just complex components, each of which requires a build pipeline of their own. Once all systems pass through their respective build pipelines they are integrated together and subjected to a joint deployment and further testing. When facing such a scenario, I decided to build the simplest possible thing that would work and get the job done.

Two converging build pipelines

Continue reading