Simplifying cloud-config creation for clusters

Coreos, Cloud Comments

I’m still experimenting with container orchestration. Currently I’m in the process of building a three node CoreOS cluster with Kubernetes on top of it, connected over the Internet. One problem I was constantly struggling with was keeping my cloud-configs in sync. Most of the configuration settings were identical or nearly identical on all three nodes. Still, when adding a small change, I needed to apply this change to all three files. Forgetting one or mistyping led to errors and unnecessary debugging sessions.

This weekend I decided I’ve had enough of it. I created a small Python script to simplify working on several nearly identical configuration files, cloud-config-creator. By iterating over a set of node values and one master template the script creates the cloud-configs for all nodes. It’s little more than a wrapper for the Jinja2 templating engine, but I still find this incredibly useful. That’s why I want to add a bit more explanation around it.


You will need Python. I used Python 3, never tested it with 2.x. Furthermore you need to install the pyyaml and jinja2 modules. Before starting to use cloud-config-creator you should have basic knowledge of how to use templates. If you ever worked with a templating engine (e.g. for consul-template or Jekyll), you’ll quickly feel at home. Otherwise I recommend the Jinja2 documentation.
Furthermore you should know how to format YAML. My script uses PyYAML, which isn’t YAML 1.2 compatible (yet), so you’ll need to use YAML 1.1.

Script usage

./cloud-config --templatefile master.tmpl --valuesfile values.yml --outpath out/ --includepath includes/

Running Caddy and Go on ARMv6 Alpine Linux

Docker, Go, Raspberry Pi, Linux Comments

My goal was compiling Caddy for my old Raspberry Pi 1 Model B. Caddy only provides an ARMv7 binary which isn’t compatible to the original Pis ARMv6. My Raspi is running on Hypriot, the Docker distribution for the Pi, therefore I wanted Caddy to run in a container as well. I chose my own Alpine Linux base image as its foundation.

As Caddy is written in Go, compiling it from source should be very easy:

  1. go get github.com/mholt/caddy/caddy
  2. cd into your website’s directory
  3. Run caddy (assuming $GOPATH/bin is in your $PATH)

While trying to get Go running in my Alpine container I encountered a small problem.
Go officialy provides an ARMv6 binary package, which is able to natively run on the Pi. But when trying to run this package in my Alpine container, a rather nondescriptive error blocked me:

/bin/sh: go: not found

I admit I needed longer than expected to solve this problem. After straceing, debugging and a lot of Internet research without result, using a simple file gave me the deciding hint:

/go # file $(which go)
/usr/local/go/bin/go: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, not stripped

Go is looking for /lib/ld-linux-armhf.so.3 which isn’t available on a minimal Alpine Linux installation. Running apk add libc6-compat finally solved this problem.

SSH: Disable Host Key Checking temporarily

Linux Comments

A couple of days ago I found an easy solution to a problem I ignored way too long. When working with Virtual Servers it’s a common occurence that you test something, it doen’t go as planned and the server doesn’t boot properly anymore. Most VPS providers offer some kind of Recovery OS or a Rescue System for those situations. Just boot the server into this OS, revert your faulty changes, reboot the system and you’re set to nuke your server again.
Sadly, I always had a small problem. As the Recovery OS uses a different SSH Host Key, you get a warning when connecting to the server:

○ → ssh testserver
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
Please contact your system administrator.
Add correct host key in /home/m3adow/.ssh/known_hosts to get rid of this message.
Offending RSA key in /home/m3adow/.ssh/known_hosts:27
  remove with: ssh-keygen -f "/home/m3adow/.ssh/known_hosts" -R testserver.adminswerk.de
RSA host key for testserver.adminswerk.de has changed and you have requested strict checking.
Host key verification failed.

Video: Dockers '--userns-remap' feature explained

Docker Comments

I have been up to the ears in work and in projects, that’s why I haven’t been posting a lot. I initially wanted to create a series about automatic service discovery and configuration with Consul, Registrator and consul-template, but decided to switch to Rancher in the process as I encountered too much hassle and had to make too much workarounds.

But that’s not the topic of this post. I recently created a short video on asciinema to further explain dockers --userns-remap feature which significantly improves security.

What is it you ask? The original description from the docker-daemon manpage:

--userns-remap=default|uid:gid|user:group|user|uid Enable user namespaces for containers on the daemon. Specifying “default” will cause a new user and group to be created to handle UID and GID range remapping for the user namespace mappings used for contained processes. Specifying a user (or uid) and optionally a group (or gid) will cause the daemon to lookup the user and group’s subordinate ID ranges for use as the user namespace mappings for contained processes.

Here’s my video explaining it:

Simplifying Let's Encrypt on Uberspace

Uberspace Comments

I’m regularly using Uberspace, a provider for Shared Hosting with a lot of special quirks (sadly only available in German). I’m also using Let’s Encrypt to create certificates for most of my websites. The hard working folks at Uberspace have integrated a very easy way to apply Let’s Encrypt certificates for your domains, but it’s also very limited:

Übrigens ist dieses Zertifikat für alle oben angegebenen Domains gültig, du bekommst also nicht ein Zertifikat pro Domain, sondern ein Zertifikat, welches für alle Domains gültig ist:

“Your certificate is valid for all entered domains. You don’t get one certificate per domain, but one certificate which can be used for all domains.”

I don’t want this at all. I’ve got several domains set up which are not logically linked, some are not even linked with my name and shall stay this way.

So I created a small script to allow the creation of Let’s Encrypt certificates for several Uberspace domains independently. Introducing letsencrypt-uberspace-mgr.