September 09, 2020

#Install Kubernetes on Raspberry Pi OS

We’re going to use k3s, a lightweight Kubernetes distribution, to get the most of our hardware. This tutorial uses a Raspberry Pi 4 and the latest version of Raspberry Pi OS 32-bit (formerly known as Raspbian).

Flash the OS image on your SD Card and, if necessary, add Wi-Fi credentials so you can access it.

Enable cgroups support and disable IPv6 by appending the following on /boot/cmdline.txt (remember that /boot refers to the boot partition on your SD Card).

cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory ipv6.disable=1

Personally, I also recommend disabling swap.

dphys-swapfile swapoff && systemctl disable dphys-swapfile.service

If your workloads won’t require GPU, you may want to change the Memory Split to 16 using raspi-config. You’ll have a little extra RAM this way.

Ensure that your OS is using legacy iptables.

iptables -F
update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

Install and test Docker.

curl -fsSL https://get.docker.com | sh -
# Test if everything is running
docker run hello-world
# Optional: allow the "pi" user to run Docker as well
usermod -aG docker Pi

Install and test k3s.

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable=traefik --docker" sh -
# After a minute, you should be able to test it
kubectl get nodes

The kubeconfig yaml will be available at /etc/rancher/k3s/k3s.yaml.

Further reading

Configuration options for k3s

Section break
September 08, 2020

#Use the RGB Cooling HAT, by Yahboom, with Docker on your Raspberry Pi

This tutorial was tested on a Raspberry Pi 4 with RGB Cooling HAT, model “YB-EBV02 VER1.1”, by Yahboom.

The hardware creators do provide installation documentation and official code, but putting all the parts together can be challenging.

If you’re willing to use Docker, you can activate everything with one command (after enabling I2C with raspi-config).

docker run -d --restart unless-stopped --network host --privileged laury/raspberry-pi-rgb-cooling-hat:latest

You can also build the Docker image yourself using the following Dockerfile. Just remember to target ARMv7 architecture. The easiest way to achieve that is to build in the Raspberry Pi itself.

FROM python:2.7

RUN apt update &&\
    apt install -y i2c-tools git

RUN pip install Adafruit-GPIO==1.0.3 \
                Adafruit-BBIO==1.2.0 \
                Adafruit-SSD1306==1.6.2 \
                smbus==1.1.post2 \
                image==1.5.32 \

RUN git clone --depth 1 https://github.com/raspberrypi/firmware.git &&\
    cp -a firmware/hardfp/opt/vc/* /usr

RUN git clone https://github.com/YahboomTechnology/Raspberry-Pi-RGB-Cooling-HAT.git &&\
    unzip Raspberry-Pi-RGB-Cooling-HAT/2.Python\ programming/RGB_Cooling_HAT.zip &&\
    mkdir /app &&\
    cp -a RGB_Cooling_HAT/* /app &&\
    rm -rf /tmp/*

CMD python /app/RGB_Cooling_HAT.py

Section break
August 29, 2020

#Connect to a headless Raspberry Pi through SSH for the first time

This will work for Raspberry Pi OS (formely known as Raspbian) and no monitor or keybord is needed.

After flashing the OS, create a wpa_supplicant.conf file on the boot partition of your SD Card (not the boot folder).

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
country=<Insert 2 letter ISO 3166-1 country code here>

 ssid="<Name of your wireless LAN>"
 psk="<Password for your wireless LAN>"

On the same partition, create an empty file called ssh. It will instruct the OS to enable the SSH server.

touch ssh

To find the Raspberry Pi in your local network, you can use nmap. Assuming your local addresses start with 192.168.0, run:

nmap -sn

If your Pi connected correctly, you will see something similar to the following in the output.

Nmap scan report for raspberrypi (<IP>)
Host is up (0.11s latency).
MAC Address: <MAC> (Raspberry Pi Trading)

Further reading

Official docs on remote access and Wi-Fi settings.

Section break
August 26, 2020

#Combine kubectl and JSONPath to read data from a Kubernetes cluster

To customize how data about cluster objects is presented on the terminal, its possible to use JSONPath syntax. Keep in mind that, under the hood, Kubernetes’ API already sends and receives data in JSON format (the YAMLs we’re used to see are an abstraction to ease reading and editing).

Since data hierarchy on the YAML representation may be a bit off, start by getting a pure JSON version of the data you want to filter.

In the following examples, let’s use a specific deployment.

kubectl -n <my_namespace> get deploy/<my_deployment> -o json

Get only the deployment name.

kubectl -n <my_namespace> get deploy/<my_deployment> -o jsonpath='{.metadata.name}'

range is one of the improvements to JSONPath available here. It iterates over JSON lists and can be paired with @ to make references to each item.

Using both, you can, for instance, print image names for all containers created by a deployment (with line breaks).

kubectl -n <my_namespace> get deploy/<my_deployment> -o jsonpath='{range .spec.template.spec.containers[*]}{@.image}{"\n"}{end}'

A good use case for JSONPath is to list annotations for every deployment in a cluster. Tools like Velero depend on the proper configuration of annotations to do volume backups, so it’s convenient to have a consolidated output.

The following command prints namespace, name and template annotations for every deployment. Additionally, it pipes the list to grep so we can have color highlights.

kubectl get -A deployments -o jsonpath='{range .items[*]}{@.metadata.namespace}{" / "}{@.metadata.name}{"\n"}{@.spec.template.metadata.annotations}{"\n"}{"\n"}{end}' | grep --color "backup.velero.io/backup-volumes\|$"

Note: grep is also matching line endings to ensure all lines are printed.

Further reading

Kubernetes docs on JSONPath

JSONPath evaluator

JSONPath syntax details

Section break
August 23, 2020

#Create, test and run a Java project with Gradle using Docker and no IDE

In order to create and manage Java projects, its common to employ the help of an IDE like IntelliJ IDEA or Eclipse.

However, if you would rather use only the command line, Docker can be a great help.

For convenience and reproducibility, create a Dockerfile in your local filesystem with the following contents.

FROM openjdk:14

ENV GRADLE_HOME=/opt/gradle/gradle-6.4.1

RUN yum install -y wget unzip

RUN wget https://services.gradle.org/distributions/gradle-6.4.1-bin.zip -P /tmp &&\
    unzip -d /opt/gradle /tmp/gradle-*.zip &&\
    rm -rf /tmp/*

Build the Docker image and run a container (takes a while). Again, for convenience, -v and -w are used so that file creations and modifications made inside the container are reflected outside.

docker run --rm -it -v ${PWD}:/app -w /app $(docker build -q .) bash

Now, we can use gradle to create and manage a project from the command line.

# Create a new project
# The "application" template most likely will suit your needs
gradle init

# Run unit tests (if you enabled JUnit during project creation)
gradle test

# Run your code
gradle run

Write source code in src/main/java/<your_package> and test code in src/test/java/<your_package>.

Further reading

Official Gradle guide for Java applications

Docker command line

Gradle command line

List of JUnit Assertions

Section break
August 20, 2020

#Issue Let's Encrypt certificates for domains in a Kubernetes cluster

We will use cert-manager to issue HTTPS certificates for domains served publicly by a Kubernetes cluster.

The application can be installed via Helm, but it’s recommended to install CRDs (Custom Resource Definitions) separately. That may come in handy if you need to delete cert-manager without losing already issued certificates. Don’t forget to change the “version” accordingly on all commands.

kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.16.1/cert-manager.crds.yaml

Add JetStack charts to your Helm client.

helm repo add jetstack https://charts.jetstack.io
helm repo update

Install the application.

helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --version v0.16.1

To issue actual certificates, you will need a production Issuer or ClusterIssuer. The following yaml defines a ClusterIssuer and assumes that you have a nginx ingress controller. If it suit your needs, apply it to your cluster.

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
  name: letsencrypt-prod
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: <your_email>
    # Name of a secret used to store the ACME account private key
      name: letsencrypt-prod
    # Enable the HTTP-01 challenge provider
    - http01:
          class: nginx

Verify if your set up is working.

$ kubectl get clusterissuers -A
NAME               READY   AGE
letsencrypt-prod   True    30m

The cert-manager stack will act upon ingresses that comply with a couple of conditions to generate HTTPS certificates. The following example ilustrates that. Take special note of the annotations and the “secretName” property under “tls”.

apiVersion: extensions/v1beta1
kind: Ingress
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/tls-acme: "true"
  name: nginx-ingress
  namespace: ingress-testing
  - host: <public_URL>
      - backend:
          serviceName: <service_name>
          servicePort: 80
  - hosts:
    - <public_URL>
    secretName: <secret_name_of_your_choosing>

Rancher users

If you’re installing the Helm Chart via Rancher, you may see the error release cert-manager failed: resource's namespace kube-system doesn't match the current namespace cert-manager. This happens because cert-manager tries to make changes to more than one namespace and Rancher doesn’t support that. There is an issue discussing this.

Thankfully, cert-manager offers an option to only operate in one namespace. Set “global.leaderElection.namespace” to “cert-manager” to achieve this.

helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --version v0.16.1 \
  --set global.leaderElection.namespace=cert-manager

k3s users

In k3s, at least up to version 1.18.x, cert-manager has problems running its webhook. At this time, to deal with this, we have to use an older version of cert-manager that can run without it. Set “webhook.enabled” to “false” and use v0.13.1.

helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --version v0.13.1 \
  --set webhook.enabled=false

Further reading

Official documentation on the installation process

Section break
August 11, 2020

#Pretty-print JSON files using the command line

If you have either Python 2 or 3 installed, use the “json.tool” module to pretty-print.

python -m json.tool <json_file> | less

You can also stream input to Python instead of passing a file directly.

cat <json_file> | python -m json.tool | less

This may be useful for viewing large minified JSON files that could slow down GUI applications.

Section break
August 03, 2020

#Quick usage examples of "find" on the command line

Search recursively for files using a regex pattern. Use -name for a case sensitive search.

find . -iname '<partial_name>*'

Execute a command with every result found.

find . -iname '*.md' -type f -exec ls -lh {} \;

Note: every command needs to be terminated by ; or +. But these signs may need to be escaped as in ; or \;. Outputs given by find will be placed on {}. Multiple uses of -exec are allowed.

Section break
August 02, 2020

#Read Kubernetes API data using Golang

Get the API client.

import (
  metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

// use the current context in kubeconfig
config, err := clientcmd.BuildConfigFromFlags("", "<path_to_KUBECONFIG>")
if err != nil {

// create the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {

Usage examples:

  • read data about servers
nodeList, err := clientset.CoreV1().Nodes().List(context.TODO(), metav1.ListOptions{})
if err != nil {
  • read data about pods
namespace := "example"  // leave empty to get data from all namespaces
podList, err := clientset.CoreV1().Pods(namespace).List(context.TODO(), metav1.ListOptions{})
if err != nil {

In the above examples, ListOptions can take two important strings, field selectors and label selectors. This can be used for filtering results.

  LabelSelector: "labelName=labelKey",
  FieldSelector: "spec.nodeName=<node_name>",  // Example for filtering by node name

Field selectors and label selectors on this case operate in the same way that the CLI options for kubectl do. So, the same rules apply here.

Further reading

Official docs on Field Selectors and Labels.

Section break
August 02, 2020

#Read Kubernetes metrics-server data using Golang

First, get the metrics API client.

import (
  metricsv "k8s.io/metrics/pkg/client/clientset/versioned"

// Get the config
config, err := clientcmd.BuildConfigFromFlags("", "<path_to_KUBECONFIG>")
if err != nil {

// Get the metrics client
metricsClientset, err := metricsv.NewForConfig(config)
if err != nil {

Get the data and store it in podMetricsList array.

namespace := "example"  // leave empty to get data from all namespaces
podMetricsList, err := metricsClientset.MetricsV1beta1().PodMetricses(namespace).List(context.TODO(), metav1.ListOptions{})
if err != nil {

Iterate over the results found.

for _, v := range podMetricsList.Items {
  fmt.Printf("%s\n", v.GetName())
  fmt.Printf("%s\n", v.GetNamespace())
  fmt.Printf("%vm\n", v.Containers[0].Usage.Cpu().MilliValue())
  fmt.Printf("%vMi\n", v.Containers[0].Usage.Memory().Value()/(1024*1024))

Note: if you’re inspecting pods that may have more than one container, you’ll need to iterate over v.Containers as well.

Section break
August 01, 2020

#Receive data from Kubernetes API using curl

Start by running kubectl’s proxy. With its help, we avoid dealing with authentication headers.

kubectl proxy

Test the connection with a basic call.

curl http://localhost:8001/api/

To check the APIs groups available, access the root.

curl http://localhost:8001/apis/

Based on the groups list, you can create the URLs for mores requests.

For cluster-scoped resources, use:


For namespace-scoped resources:


To get deployment data, for instance, the API Group is “apps”, Version is “v1” and Resourcetype is “deployments”. The final request looks like this:

curl http://localhost:8001/apis/apps/v1/deployments

If the Kubernetes cluster has a metrics-server, the following request will get node resource usage data:

curl http://localhost:8001/apis/metrics.k8s.io/v1beta1/nodes

Rancher users

However, if your cluster was created using Rancher, you need to have “Authorized Cluster Endpoint” activated, so your requests can reach the API Server directly. If not, your URL paths will differ from the examples above.

On your command line, you will also have to select the correct context for kubectl before activating the proxy.

# List availabe contexts
kubectl config get-contexts

# Select the direct API context
# Its name has the format <cluster_name>-<pilot_name>
kubectl config use-context <direct_context>

Further reading

Overview of the Kubernetes API

More info about API concepts

Accessing the API without kubectl proxy

Full list of Groups, Versions and Resourcetypes

Section break
July 31, 2020

#Activate the replicaset mode in MongoDB

Start the MongoDB deamon with the --replSet flag.

mongod --replSet rs0

After first start, initialize the replica set with the following:

mongo <mongo_host>/<database> --eval "rs.initiate({_id: 'rs0', members: [ { _id: 0, host: '<mongo_host>:27017' } ]})"

Section break
Section break
July 23, 2020

#Test networking performance between two Linux machines

Let’s use iperf to measure networking performance. First, set a “server” machine.

iperf -s

From a “client” machine, run the test.

iperf -c <server_ip>

If you are having trouble getting the server’s IP, use ip addr show to find addresses used by the current machine in any connected networks.

Section break
July 23, 2020

#Wipe any filesystem data and partitions from devices on Linux

wipefs should work in most cases to clean filesystems.

# View filesystem data from device
wipefs /dev/<my_device>

# Wipe any filesystem found
wipefs --all /dev/<my_device>

If wipefs can’t detect a filesystem, you can quickly wipe the header data of the device with dd.

dd if=/dev/zero of=/dev/<my_device> bs=1M count=100 oflag=direct,dsync status=progress

You may need to restart your machine to make the filesystem deletion known to the OS, if your device is a “LVM2_member” for instance.

Also, as a last resort, you can completely wipe the device.

dd if=/dev/zero of=/dev/<my_device> bs=1M count=5000000 oflag=direct,dsync status=progress

To delete partitions, sgdisk should work well.

sgdisk --zap-all /dev/<my_device>

Section break
July 16, 2020

#Bootstrap a Go Cobra application with Go Modules

To bootstrap a Go application using Cobra Generator and Go Modules, start by installing the Cobra command line:

# Install Cobra
go get github.com/spf13/cobra/cobra

Initialize your Cobra project, but remember to use your package’s fully qualified name, as in: github.com/laurybueno/kubectl-hoggers. Don’t worry if your package is not publicly available yet. And there is no need to put your source code inside GOPATH anymore (since Go 1.11 at least):

cobra init --pkg-name <fully_qualified_package_name>

Initialize your new module:

go mod init <fully_qualified_package_name>

Get recursively all its dependencies and install it:

go get -d -v ./...
go install -v ./...

Your app’s command line is now ready to be used:


Further reading

Docker and Go: https://www.docker.com/blog/docker-golang/

Go Modules: https://github.com/golang/go/wiki/Modules

Section break
July 16, 2020

#Download files recursively through a FTP conection using wget

Unfortunately, SCP and Rsync aren’t always an option to get our files out of a server. When FTP is all we have, wget can still do a pretty good job and not re-download files you already have, like a primitive Rsync.

wget -nc -nH --cut-dirs=<NUMBER_REMOTE_FOLDERS_TO_CUT_FROM_PATH> --ftp-user='<FTP_USER>' --ftp-password='<FTP_PASSWORD>' -rl 0 -P <LOCAL_PATH> 'ftp://<REMOTE_HOST>/<PATH_TO_FILES>'

Explanation for the used flags:

-nc, --no-clobber                skip downloads that would download to existing files (overwriting them)
-nH, --no-host-directories       don't create host directories
--cut-dirs=NUMBER                ignore NUMBER remote directory components
-r,  --recursive                 specify recursive download
-l,  --level=NUMBER              maximum recursion depth (inf or 0 for infinite)
-P,  --directory-prefix=PREFIX   save files to PREFIX/..

Section break
June 26, 2020

#Check a site's downtime with this bash one-liner

During a site’s downtime, the following line will print the current time and try to access the given URL one time every 2 seconds. Only the received HTTP headers will be shown.

You may find it useful for monitoring downtime while you try to solve a problem.

while true; do date; curl -IX GET '<URL>'; sleep 2; done;

Section break
Section break
March 13, 2020

#How to dump and restore MySQL/MariaDB databases

Options for dumping data:

# Directly
mysqldump --databases $MYSQL_DATABASE -u$MYSQL_USER -p$MYSQL_PASSWORD > dump-`date '+%Y_%m_%d__%H_%M_%S'`.sql

# Docker
docker exec -it <container_name> sh -c 'mysqldump --databases $MYSQL_DATABASE -u$MYSQL_USER -p$MYSQL_PASSWORD' > dump-`date '+%Y_%m_%d__%H_%M_%S'`.sql

# Kubernetes (vulnerable to networking failures)
kubectl -n <namespace> exec deploy/<deploy_name> -- bash -c 'mysqldump --databases $MYSQL_DATABASE -u$MYSQL_USER -p$MYSQL_PASSWORD' > dump-`date '+%Y_%m_%d__%H_%M_%S'`.sql

Additionally, if your database operates in a remote server (like Amazon RDS), you may still use a local Docker container for dumping the data.

# Create a local container with the desired version (in this example, MariaDB 10.4 is used)
docker run -it --rm -v ${PWD}:/dump -w /dump mariadb:10.4 bash

# Get the data
mysqldump -h <hostname> --databases <database_name> -u <database_user> --password='<password>' > dump-`date '+%Y_%m_%d__%H_%M_%S'`.sql

For restoring data, you can use:

mysql -u$MYSQL_USER -p$MYSQL_PASSWORD < dump.sql

If you’re using Docker, you can also place dump files on /docker-entrypoint-initdb.d and those will be imported on the fisrt run. Accepted files types are *.sql, *.sql.gz, and *.sh.

Section break
Section break
March 11, 2020

#How to publish Ionic apps on the Google Play Store

For this task, we’re going to use Docker to avoid installing the Android SDK on our computer. This is possible thanks to this ionic Docker image.

# Enter in the container command line
docker run -it --rm --net host --privileged -v /dev/bus/usb:/dev/bus/usb -v ~/.gradle:/root/.gradle -v $PWD:/Sources:rw -v /home/<your_user>/.ssh:/root/.ssh hacklab/ionic:android-28 bash

# Install javascript dependencies
npm install

# Execute Ionic build
ionic cordova platform add android
ionic cordova build android --prod --release

# Sign the generated build
jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore ~/.ssh/<your_keystore> platforms/android/app/build/outputs/apk/release/app-release-unsigned.apk myapp
/opt/android-sdk-linux/build-tools/28.0.3/zipalign -v 4 platforms/android/app/build/outputs/apk/release/app-release-unsigned.apk myapp-2.0.0.apk

Section break
March 10, 2020

#How to dump and restore PostgreSQL databases

Options for dumping data:

# Binary mode
pg_dump -Fc -U $POSTGRES_USER $POSTGRES_DB > dump-`date '+%Y_%m_%d__%H_%M_%S'`.psqlc

# SQL mode
pg_dump -U $POSTGRES_USER $POSTGRES_DB > dump-`date '+%Y_%m_%d__%H_%M_%S'`.sql

# Docker and SQL mode
docker exec -it -u postgres <container_name> sh -c 'pg_dump -U $POSTGRES_USER $POSTGRES_DB' > dump-`date '+%Y_%m_%d__%H_%M_%S'`.sql

# Kubernetes and SQL mode (vulnerable to networking failures)
kubectl -n <namespace> exec deploy/postgres -- sh -c 'pg_dump -U $POSTGRES_USER $POSTGRES_DB' > dump-`date '+%Y_%m_%d__%H_%M_%S'`.sql

Options for restoring data:

# Binary mode
pg_restore -O -U $POSTGRES_USER -c -x -n public -d $POSTGRES_DB dump.psqlc

# SQL mode
psql -d $POSTGRES_DB -a -f /backups/dump.sql

If you’re using Docker, you can also place dump files on /docker-entrypoint-initdb.d and those will be imported on the fisrt run. Accepted files types are *.sql, *.sql.gz, and *.sh.

Section break
Section break
Section break
March 02, 2020

#Format a new SSD/HD drive for usage

# Create the partition
parted --align optimal <raw_device>
mklabel msdos
mkpart primary ext4 0% 100%

# Get the new partition device path

# Make the filesystem
mkfs.ext4 <new_partition_device_path>

# Test the partition
mkdir /tmp/partition-test
mount -t ext4 <new_partition_device_path> /tmp/partition-test

# Get the UUID

# Edit the /etc/fstab to mount partition during boot
UUID=<uuid>     /storage        ext4    defaults,discard        0       2
## OR, if using XFS
UUID=<uuid>     /storage       xfs     defaults        1       2

Section break
Section break
July 01, 2019

#Dump and restore data from MongoDB in Docker

# Create the database dump on your server
docker exec <my_mongodb_container> mongodump --archive=/backups/mongodb-`date +%Y%m%d`.gz --gzip --db <database_name>

# Copy it to your local machine, if needed
scp -r <server_user>@<server_ip>:<remote_backup_path> <local_backup_path>

# Restore it directly into a running Docker container
zcat <backup_path> | docker exec -i <my_mongodb_container> mongorestore --archive --drop

Section break
Section break
Section break
Section break
September 01, 2017

#Some useful Vagrant commands

# Bring up the system
vagrant up

# Get into the running virtual machine
vagrant ssh

# Bring the machine down and up again
# (equivalent of running a halt followed by an up)
vagrant reload

# Destroy the current created virtual machines
vagrant destroy