I love building up and working in agile software development teams! Scrum, Java, Python, DevOps, Databases, Architecture, Integrations, Leadership
Max Wenzin
Breaking up the monolith at Discovery and enhancing subscription flexibility in one fell swoop
2020-04-17 – 08:00 by Max Wenzin and Mattias Skarin
Max – Tell me briefly what your team is up to @ Discovery?
I’m one member of a devteam which takes care of external API’s. We build services for streaming TV (OTT), for example user management, authentication, user entitlements and playback services.
We started this effort to enable a more flexible way to assign entitlements to users, for example through affiliates (like cable TV companies and mobile phone operators). Before, you had to be a subscriber at Discovery to be able to view any content. Being a direct subscriber is called the DTC model, ”Direct To Consumer”. But a large part of the business in some markets still comes from the so called “TV Everywhere” model where the customers come in via affiliates.
read moreHow to deploy a microservice application to AWS
2019-01-03 – 15:05
In this post, I will use an example to show how you can:
- Write a small microservice with a REST API in Java, using the Dropwizard framework
- Containerize the application using Google JIB
- Deploy the application to AWS using Ansible
- With a robust, clustered and self-healing setup
Readable Java system tests with good old JUnit
2018-05-26 – 14:14
This article is the third in a series about system testing:
- Dockerized testing vs end-to-end testing
- How to setup Dockerized testing
- Readable Java system tests with good old JUnit
JUnit is poorly named. Given the name, people tend to think that it should only be used to write Java unit tests. And then people feel a bit hesitant about writing their integration tests with JUnit too. When they start with system tests, they often think they need another driver for their tests. Sure, maybe using another abstraction layer and a custom domain specific language (BDD), you can make the tests more readable for a non-programmer. That often comes at the cost of making the tests less readable for the programmers. And if we are honest, who’s going to read the tests the most? Perhaps just naming test classes and methods well and writing readable code can suffice?
How to setup Dockerized testing
2018-05-26 – 14:04
This article is the second in a series about system testing:
- Dockerized testing vs end-to-end testing
- How to setup Dockerized testing
- Readable Java system tests with good old JUnit
You may want to read Dockerized testing vs end-to-end testing first.
How to setup Dockerized testing?
A prerequisite for running your tests against a dockerized environment is of course that the service you want to test is packaged as a Docker image. Also, all of the services which the System Under Test (SUT) depends on needs to be dockerized. If you don’t have that, your environment is not fully dockerized and you will suffer the consequences of having to test against external services.
If your SUT is packaged as a Docker image and has no dependencies on other services, you can just start it using “docker run” and run your tests against it. However, usually services have one or more dependencies to other services, such as its own database, a queue and perhaps a external RESTish service that returns nasty XML. You want each of these services in their own container. To spin up a bunch of docker containers, you can use docker-compose.
Dockerized testing vs end-to-end testing
2018-05-26 – 13:53
I recently rewrote the end-to-end tests for a service. This brought up the question of what we should test in the end-to-end (e2e) tests vs our Dockerized tests.
This article is the first in a series about system testing:
- Dockerized testing vs end-to-end testing
- How to setup Dockerized testing
- Readable Java system tests with good old JUnit
e2e tests vs dockerized tests
Since the original e2e tests were written, we (my team) have also introduced another type of tests. We call them “dockerized system tests”. These are similar to the e2e tests. They also test the connections between services in the system, and how the system works as a whole. One key difference between e2e tests and dockerized tests is that the dockerized tests do not test the system when it is deployed the way we deploy it in staging and production environments. Our dockerized tests bring up all services in a single machine, and we usually don’t set up redundancy, clustering, load balancers and such things.
How to structure Ansible variables when provisioning infrastructure
2018-02-18 – 22:08
How should you structure your Ansible variables in playbooks calling cloud modules? Ansible has extensive support for variables at different levels (there are 21 different levels!).
The inventory group_vars and host_vars offer enough flexibility for many use cases. However, when you are setting up cloud infrastructure, you don’t really have any hosts yet. Maybe you are setting up load balancers, instances, security groups and such things. Perhaps you want to setup the same things in several different environments (like “staging” and “production”). You want to re-use the same playbook, but adapt what it does by using different variable values in different environments.
In this post, I will demonstrate a way to structure Ansible variables in a playbook which pretends to setup a AWS AutoScalingGroup. I will show how you can separate the configuration for different environments by using a inventory directory. As a bonus, I will give you a neat trick to automatically load a extra variable file for each environment. We will use this to load a separate secrets file which is encrypted with git-crypt.
How to run Ansible tasks in parallel
2018-01-27 – 15:48
When you use Ansible to provision hosts, running a task on a set of hosts in parallel is not a problem. You simply use the `serial` keyword in a playbook. See: http://docs.ansible.com/ansible/latest/playbooks_delegation.html#rolling-update-batch-size
What about when you want to run a task several times in parallel, either on each host, or on a single host? A common use case is when you want to provision infrastructure at a cloud provider, such as Amazon Web Services (AWS). In that case, you normally run the task on localhost and the task just calls out to the cloud provider API.
The reason you might want to do these calls in parallel is because creating cloud resources sometimes takes a long time. AWS EC2 instances and RDS instances are examples of some resources which take a long time to create. If you want to create several, you can use a loop and wait for each to finish, but that takes a long time. As a alternative, you can create them all in parallel.
Here is a example Ansible play of how you can run any Ansible task in parallel and wait for them all to finish.
---
- name: Run tasks in parallel
hosts: localhost
connection: local
gather_facts: no
tasks:
- name: Pretend to create instances
command: "sleep {{ item }}" # Instead of calling a long running operation at a cloud provider, we just sleep.
with_items:
- 6
- 8
- 7
register: _create_instances
async: 600 # Maximum runtime in seconds. Adjust as needed.
poll: 0 # Fire and continue (never poll)
- name: Wait for creation to finish
async_status:
jid: "{{ item.ansible_job_id }}"
register: _jobs
until: _jobs.finished
delay: 5 # Check every 5 seconds. Adjust as you like.
retries: 10 # Retry up to 10 times. Adjust as needed.
with_items: "{{ _create_instances.results }}"
The output will be:
PLAY [Run tasks in parallel] **********************************************************************************************************
TASK [Pretend to create instances] ****************************************************************************************************
changed: [localhost] => (item=6)
changed: [localhost] => (item=8)
changed: [localhost] => (item=7)
TASK [Wait for creation to finish] ****************************************************************************************************
FAILED - RETRYING: Wait for creation to finish (10 retries left).
changed: [localhost] => (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'957883465297.7487', 'item': 6, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/home/max/.ansible_async/957883465297.7487'})
FAILED - RETRYING: Wait for creation to finish (10 retries left).
changed: [localhost] => (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'450659566541.7513', 'item': 8, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/home/max/.ansible_async/450659566541.7513'})
changed: [localhost] => (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_no_log': False, u'ansible_job_id': u'763185500456.7538', 'item': 7, u'started': 1, 'changed': True, 'failed': False, u'finished': 0, u'results_file': u'/home/max/.ansible_async/763185500456.7538'})
PLAY RECAP ****************************************************************************************************************************
localhost : ok=3 changed=3 unreachable=0 failed=0
See the example code at GitHub.
One of the plays runs the tasks in parallel, and one runs then in sequence, for comparison.
Warning! These 6 Pitfalls Will Slow Down Your Organization
2017-03-02 – 06:30 by Max Wenzin, Jimmy Janlén, Michael Göthe, Yassal Sundman, Jan Grape and Mia Kolmodin
You have probably read about “at scale” implementations, activity based offices, globally distributed teams, SAFe, Agile transformations and outsourcing. Beware. Danger can be lurking beneath the surface of these popular phenomena.
Reactions to “No CEO” by the BBC
2017-02-19 – 13:17 by Max Wenzin, Yassal Sundman and Hans Brattberg
When the BBC published their “No CEO” piece where Crisp is featured with an article and a 4 minute video, there were a lot of reactions. Friends cheered on Facebook. Colleagues gave a thumbs up on LinkedIn. The article was featured on Hacker News and Slashdot. Here are our reflections on some of the comments we found.
Scientific method applied to performance improvements
2016-11-07 – 21:38
In my team, we are working on improving real-time performance for our main service. The goal is to have response times below 100 ms in the 95th percentile and below 200 ms in the 99th percentile for certain database volumes and request frequencies.
We don’t know what will be needed to reach this goal. We have some ideas, but we don’t know which one, or which ones will do the trick. We call these ideas “experiments”.
We can estimate each experiment, but we don’t know how many we will need to do to reach the goal.
This is the story of how we apply the scientific method to working with performance improvements.
How to append to lists in Ansible
2016-10-20 – 07:00
Since I have found the Ansible documentation to be lacking, and StackOverflow insufficient in this matter, I feel the need to share how you can append to a list using Ansible.
I’ve created a demonstration playbook and published it on GitHub. See: https://github.com/betrcode/ansible-append-list
Continuous Delivery of custom WordPress using Vagrant, Ansible and AWS
2016-01-17 – 14:34
Vagrant + Ansible + AWS = <3
The www.crisp.se website is based on WordPress, with a custom Crisp theme.
This is the story about how we’ve developed our custom WordPress theme, how we’ve made it easy for any Crisper and external consultants to work on the theme, how we’ve setup version control, continuous delivery, staging and production environments on Amazon Web Services (AWS). And how all of this is setup with absolutely no automated tests whatsoever.
Dedicated Scrum Master or not?
2015-06-21 – 09:54
Should the Scrum Master role be full time or part time? What if there is not enough Scrum Master work to do? Can the Scrum Master also do other work in the team? Can the Scrum Master be Scrum Master for several teams?
There was a debate about this and Scrum Alliance created the Scrum Master Manifesto in 2011.
Even though this has been debated by many minds before, I still get asked what my views are on this topic.
I’ve done all kinds of variations on this role. I’ve been a dedicated Scrum Master for a single team. I have done the SM role and a developer role at the same time. I’ve been a Scrum Master for several teams at the same time. These alternatives have their own advantages and challenges. In this post I intend to describe my view and recommendations.
Crisp DNA is now open source!
2015-01-23 – 13:01 by Max Wenzin, Mats Henricson and Henrik Kniberg
Crisp DNA screenshot
We get a lot of questions about how Crisp works and why, especially from other consultants looking to create something similar. After many years of experimenting we’ve converged on a model that works well, basically the sweet spot between being an independent consultant and being an employee. So we decided to open source it.
In January 2015, at Crisp Hack Summit 10, Mats Henricsson, Henrik Kniberg and Max Wenzin huddled up and created the first version of the open-sourced Crisp DNA. It is published at http://dna.crisp.se/
The Crisp DNA is version controlled using Git and both Crisp DNA repository and Crisp DNA web site is hosted at GitHub using GitHub Pages. The web site is a bunch of static web pages and images that is generated from Markdown and Textile files using Jekyll. This is the same setup that GitHub themselves use.
By using open, version-control friendly formats such as Markdown and Textile, we hope to benefit from the open source community which can fork our model, make changes and perhaps suggest improvements. This can actually change how Crisp works!
Why I prefer ToDo over Trello for agile teams
2014-03-13 – 14:42
The Gist
- ToDo has a flow. It knows about cycle times and about being DONE. Trello does not.
- ToDo has Planning Poker Estimates. Trello does not have any estimates.
- ToDo has automatic burn up charts. Trello does not.
- ToDo has swim lanes which groups cards by your dimensions. Trello does not.
- ToDo has Work-In-Progress limits. Trello does not.
- ToDo has upgrade possibilities to the full tool set of Projectplace. Trello has a bunch of plugins from different vendors of various quality.
Swimlanes on a ToDo board
Already convinced? Sign up for ToDo by Projectplace! Want to know more? Read on.
Five team principles
2012-10-22 – 22:28
Building a well-functioning software delivery team is complicated. There are many factors to consider. Current team (if any), needed skills, available people, company politics etc.
There are some fundamentals that often (but not always) seem to work.
My fundamental principles for teams
- Static
- Cross-functional
- 5-9 people
- Co-located
- Dedicated team members (belong to only one team)
I find these principles to be a useful basis for discussion, when helping managers configure their teams.
The principles are goals, and one must realize that all cannot be achieved all of the time, nor instantly.
A Crisp New Web
2012-08-15 – 20:08
As a visitor to the Crisp Blog, you may have noticed that we’ve launched a new web site on www.crisp.se?
Apart from a new design, we’ve also changed CMS from imCMS by imCode to open source WordPress.
Old site, based on imCMS
New site, based on WordPress
Case Study of Mobile Team at Projectplace
2012-03-26 – 21:11
Projectplace
I am currently working as a Scrum Master for multiple teams at Projectplace in Stockholm, Sweden. One of those teams is the Mobile Team. They are developing Action Boards for both iOS (iPad) and Android platforms. These Action Boards are also available in the Customer Preview of the Projectplace web service. Both Web Team and Mobile Team share the same API’s. The iPad app is planned to be released in 2-3 Sprints from now.
This case study can be written from many perspectives, but in this article I am going to focus on how we are working with the challenges of having a distributed Scrum team.
My first Visual Agenda
2012-03-22 – 20:23
I have been reading Gamestorming lately and found that I’ve practiced many of the games and followed most of the principles of game design already.
Visual Agenda
However, there are a lot of new things in that book that I’d like to try. Today I tried drawing a Visual Agenda for the first time.
I think it was well recieved, because the team didn’t want to erase it even though we were running out of whiteboard space during the meeting. 😉
The purpose of the exercise was to make it a bit more fun and rememberable, to show the team that they and the project are important enough to spend more time than usual on an agenda.
If you are a recurring victim of my presence, you can be sure to be exposed to many more of the games from the book; Gamestorming.
