Running Docker swarm mode on Scaleway

• Docker and Scaleway • Comments

I’m currently running a couple of tests with Docker swarm mode on Scaleway. For those who don’t know Scaleway, it’s a PaaS provider which is part of online.net. Due to capped maximum prices per month and no Ingress/egress prices it’s good for smaller projects.

But small prices often also mean a couple of limitations as well. In my case I wasn’t able to run Docker swarm mode properly. I tried running a stack with a couple of web servers and Traefik as Reverse Proxy, but for some reason this didn’t work. When running a Traefik container manually in Docker, everything worked fine. But as soon as I tried using it in swarm mode context, there was no connectivity. Dockers logs showed some strange errors similar to these:

Apr 14 20:39:40 <hostname> docker[15787]: time="2017-04-14T20:39:40Z" level=error msg="Failed to write to /proc/sys/net/ipv4/vs/conntrack: open /proc/sys/net/ipv4/vs/conntrack: nno such file or directory"
Apr 14 20:39:40 <hostname> docker[15787]: time="2017-04-14T20:39:40.154260407Z" level=error msg="Failed to add firewall mark rule in sbox ingress (ingress): reexec failed: exit status 8"
Apr 14 20:41:17 <hostname> docker[15787]: time="2017-04-14T20:41:17.432619182Z" level=error msg="Failed to delete real server for vip fwmark 259 in sbox ingress (ingress): no such process"
Apr 14 20:41:17 <hostname> docker[15787]: time="2017-04-14T20:41:17.432762944Z" level=error msg="Failed to delete service for vip fwmark 259 in sbox ingress (ingress): no such process"

Redirecting ports with iptables

• linux • Comments

From time to time I need to test the connectivity of applications listening on a port which has not yet been opened in the firewall. Therefore I use iptables to redirect an open port to the port the application is listening on. Lets assume I want to redirect port 80 to 8080 and port 443 to port 8443. Here’s how to do it:

First enable IP Forwarding

sysctl -w net.ipv4.ip_forward=1

Afterwards enable the redirection for said ports

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 8443

Optional: If you also want to redirect the port on the local machine, you have to set an additional iptables rule

iptables -t nat -I OUTPUT -p tcp -o lo --dport 80 -j REDIRECT --to-port 8080
iptables -t nat -I OUTPUT -p tcp -o lo --dport 443 -j REDIRECT --to-port 8443

This is of course not reboot safe. For reboot safety you need to set ip_forward in the /etc/sysctl.conf or a respective /etc/sysctl.d/ file and use the iptables-restore functionality of your distribution.

PS: This could of course also be used to circumvent firewall restrictions, e.g. to connect to SSH via another port without reconfiguring sshd. Do this at your own risk!

OpenSSL scribblings

• ssl and linux • Comments

Just a quick writeup from my notes so I know where to look it up if I ever search for it again. In my examples I use Comodo as the certificate authority and ssl.example.org as domain.

Certificate chain verification

There are two scenarios I normally encounter, either verify if the certificate chain is complete or find out where the certificate chain breaks.

Verifying the whole chain

  1. Concatenate the certificate chain including the root certificate in one file. From the top of my head I’m not quite certain if order is important, leaf to root worked for me.
  2. Do an openssl verify with -CApath /dev/null to prevent taking the systems trust stores into account:
openssl verify -verbose -CApath /dev/null -CAfile concatenated-chain-file.pem ssl.example.org.crt

If the output of the command contains ‘OK’, the chain is complete.

Xiaomi Mi Band: Re-enable Early Bird Alarm

• Android • Comments

I bought a Xiaomi Mi Band 1S one year ago solely for the reason of taking advantage of the “Early Bird Alarm”. This feature essentially tells the band to watch your sleep cycles 30 minutes before your set alarm and wake you up if it detects you are in a light sleep some time during this timespan (you probably knew this already, why are you here otherwise? ;-}). Sadly, Xiaomi removed this feature some time in 2016 without ever mentioning why or if it ever returns. As I don’t use my band for everything else, I recently decided to recover my “Early Bird”. I was successful and want to share all the actions I tried in case others have a similar need for returning their smart alarm. I assume these steps can be applied to other Mi Band versions (Mi Band 1, Mi Band 1S, Mi Band 2) as well.

Prelude: I’m not certain if every action is needed. I tried dry testing my changes but all changes failed the tests, including the (fully working) one I’m using now. So I advise on “live” testing every change for two or three nights if you don’t want to pull completely through with everything. If you find out only parts of it are needed, I’d appreciate if you left a comment afterwards.

Kubernetes Single Node installation on CoreOS Container Linux

• docker, Linux, and coreos • Comments

I’ve been playing with Kubernetes on CoreOS Container Linux for a couple of months now. As I prefer to implement real world workloads instead of examples, I planned on containerising a couple of applications my family and I rely on. Beforehand I wanted to create an easy way for installing a Kubernetes cluster spread across multiple VPS providers, securely connected over the internet via VPN or something similar. I did a speed test for initial evaluation of different methods and began working on a proof of concept CoreOS Kubernetes cluster secured via tinc. This prove more difficult than initially planned, taking me months for a simple setup and setting me back multiple times.

Therefore I settled with a Kubernetes installation residing on a single node. The clustering was a bit overkill for family applications anyways and this step just took to much time. For the Single Node installation I found a blog entry from Victor Palau who also created a GitHub repository for an automated Kubernetes installation to an existing Container Linux server. I tested his script on my server from Chicago VPS and although it largely worked, I was dissatisfied. As the script was initially intended to be run on Microsofts Azure, the requirements a bare metal installation has were not taken into account:
A couple of ports meant for internal use only were publicy accessible, insecure etcd2 being the worst one. Furthermore I didn’t like the apiserver listening on port 443. While on Azure you’d normaly prepend a Load balancer in front of the Single node, this doesn’t apply to a Bare Metal installation. Thusly HTTPS was effectively blocked on the node and there was no easy way for integrating a containerised load balancer like Træfɪk. Addtionally there were some smaller problems or additions I had in mind and wanted to integrate into the automation to ease installations for others and a possible reinstallation for myself.