#Snippets

July 03, 2022

#Use iptables to allow only Cloudflare HTTP requests to target your server

If you’re using Cloudflare to protect HTTP endpoints, it might be worth it to block any web traffic that doesn’t come from their servers.

To do that, let’s use iptables. The following commands will do the trick, but take note that these IPs might change in the future.

# Allow IPv4 traffic
iptables -I INPUT -p tcp -m multiport --dports http,https -s 173.245.48.0/20 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 103.21.244.0/22 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 103.22.200.0/22 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 103.31.4.0/22 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 141.101.64.0/18 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 108.162.192.0/18 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 190.93.240.0/20 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 188.114.96.0/20 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 197.234.240.0/22 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 198.41.128.0/17 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 162.158.0.0/15 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 104.16.0.0/13 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 104.24.0.0/14 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 172.64.0.0/13 -j ACCEPT
iptables -I INPUT -p tcp -m multiport --dports http,https -s 131.0.72.0/22 -j ACCEPT

# Allow IPv6 traffic
ip6tables -I INPUT -p tcp -m multiport --dports http,https -s 2400:cb00::/32 -j ACCEPT
ip6tables -I INPUT -p tcp -m multiport --dports http,https -s 2606:4700::/32 -j ACCEPT
ip6tables -I INPUT -p tcp -m multiport --dports http,https -s 2803:f800::/32 -j ACCEPT
ip6tables -I INPUT -p tcp -m multiport --dports http,https -s 2405:b500::/32 -j ACCEPT
ip6tables -I INPUT -p tcp -m multiport --dports http,https -s 2405:8100::/32 -j ACCEPT
ip6tables -I INPUT -p tcp -m multiport --dports http,https -s 2a06:98c0::/29 -j ACCEPT
ip6tables -I INPUT -p tcp -m multiport --dports http,https -s 2c0f:f248::/32 -j ACCEPT

Now, block everything else on ports 80 and 443.

iptables -A INPUT -p tcp -m multiport --dports http,https -j DROP
ip6tables -A INPUT -p tcp -m multiport --dports http,https -j DROP

Test your new rules and save your changes to persist them across reboots.

netfilter-persistent save

Further reading

Cloudflare’s IP list

Official Cloudflare recommendations to protect your origin servers

Section break
June 26, 2022

#Protect public URLs with ModSecurity for Nginx on a Kubernetes cluster

If you’re using Ingress Nginx to manage public endpoints, ModSecurity is already installed but disabled by default. To enable it, add the following to the ConfigMap ingress-nginx-controller:

allow-snippet-annotations: "true"
enable-modsecurity: "true"
enable-owasp-modsecurity-crs: "true"

To customize the Paranoia and Anomaly Score threshold, you must change rules 900000 and 900110. But remember that higher Paranoia levels lead to stricter rules, and a lower anomaly threshold leads to a more tolerant setup.

modsecurity-snippet: |-
    SecRuleEngine On
    SecAuditEngine Off #  Use RelevantOnly for debugging
    SecAuditLog /dev/stdout
    SecAuditLogFormat JSON
    SecAuditLogType Serial
    SecAuditLogParts ABIJDEFHZ
    SecDebugLog /dev/stdout
    SecDebugLogLevel 3
    SecAction "id:900000,phase:1,nolog,pass,t:none,setvar:tx.paranoia_level=5"
    SecAction "id:900110,phase:1,nolog,pass,t:none,setvar:tx.inbound_anomaly_score_threshold=4,setvar:tx.outbound_anomaly_score_threshold=3"
    Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf

Customize ModSecurity rules on a per-host basis

It’s possible to adjust the ModSecurity configuration for each Ingress object on the cluster using annotations.

annotations:
  nginx.ingress.kubernetes.io/enable-modsecurity: "true"
  nginx.ingress.kubernetes.io/enable-owasp-core-rules: "false"
  nginx.ingress.kubernetes.io/modsecurity-snippet: |
    SecRuleRemoveById <rule_id>

How to completely customize modsecurity.conf

With the modsecurity-snippet option, adding custom configuration to ModSecurity is possible. However, you can override the modsecurity.conf entirely if you want.

To do so, first copy the file inside the pod.

kubectl -n ingress-nginx cp <ingress-controller-pod-name>:/etc/nginx/modsecurity/modsecurity.conf ./modsecurity.conf

Modify it as required and save the file in a ConfigMap.

kubectl -n ingress-nginx create configmap modsecurityconf --from-file=modsecurity.conf

If you need to change the file again locally, you can update the ConfigMap by doing the following.

kubectl -n ingress-nginx create configmap modsecurityconf \
  --from-file=modsecurity.conf -o yaml \
  --dry-run=client | kubectl apply -f -

To mount the ConfigMap to the controller deployment, create a patch file.

spec:
  template:
    spec:
      volumes:
      - name: modsecurityconf
        configMap:
          name: modsecurityconf
      containers:
      - name: controller
        volumeMounts:
        - name: modsecurityconf
          mountPath: "/etc/nginx/modsecurity/modsecurity.conf"
          subPath: modsecurity.conf
          readOnly: true

And then apply the patch file.

kubectl -n ingress-nginx patch deployment/ingress-nginx-controller --patch-file deployment-patch.yml

Further reading

ConfigMap options for Ingress Nginx

Annotations options for Ingress objects

Overview of Anomaly Scoring

Working with Paranoia Levels

Handling False Positives with the OWASP ModSecurity Core Rule Set

Section break
June 24, 2022

#Host publicly accessible web URLs from a home server using a remote proxy

Ideally, you would be able to open ports 80 and 443 of your public IP address and update DNS entries with solutions like ddclient or DNS-O-Matic.

However, if your ISP blocks direct traffic to those ports, using a remote server and SSH tunneling can bypass such limitations. With this setup, you can use Kubernetes ingresses as usual. From the ingress perspective, HTTP traffic will reach your home server directly. Therefore, tooling like Nginx and Cert Manager will work with no unique tweaks.

Any Linux box with a public IP can be your remote proxy, but Oracle Cloud has a generous free tier and can be a good starting option. Remember that the smallest available machine is probably enough for your needs since it will only forward traffic and not act on it.

After setting up the remote server with your provider, ensure it has an SSH server running. We’re going to use it to forward network packets for us.

On your remote instance, edit the /etc/ssh/sshd_config file and add GatewayPorts yes to it. Apply the new configuration with service sshd restart.

If you opted for an Oracle Cloud server, go to the Ingress Rules section of the Virtual Cloud Network panel and open ports 80 and 443. Also, adjust iptables in the instance to allow public traffic to ports 80 and 443.

iptables -I INPUT 6 -m state --state NEW -p tcp --dport 80 -j ACCEPT
iptables -I INPUT 6 -m state --state NEW -p tcp --dport 443 -j ACCEPT
netfilter-persistent save

From your home server, you can run a quick test, as long as you already have a HTTP server running locally. Don’t forget to generate a new SSH key for your local server and authorize it on the remote machine. If everything went smoothly until now, you can access a local web endpoint using the remote server IP as the DNS entry for your domain.

ssh -N -R 80:<local_ip>:80 root@<remote_ip>

If the port forwarding worked, you can now create deployment files so that Kubernetes can keep the tunnel running. Customize and apply the following to your local cluster.

apiVersion: v1
kind: Namespace
metadata:
  name: proxy-router

--
apiVersion: apps/v1
kind: Deployment
metadata:
  name: autossh-80
  namespace: proxy-router
  labels:
    app: autossh-80
spec:
  replicas: 1
  strategy:
    rollingUpdate:
      maxSurge: 0
      maxUnavailable: 1
    type: RollingUpdate
  selector:
    matchLabels:
      app: autossh-80
  template:
    metadata:
      labels:
        app: autossh-80
    spec:
      containers:
      - name: autossh-80
        image: jnovack/autossh:2.0.1
        env:
          - name: SSH_REMOTE_USER
            value: "root"
          - name: SSH_REMOTE_HOST
            value: "<remote_server_ip>"
          - name: SSH_REMOTE_PORT
            value: "22"
          - name: SSH_TUNNEL_PORT
            value: "80"
          - name: SSH_BIND_IP
            value: "0.0.0.0"
          - name: SSH_TARGET_HOST
            value: "<local_server_ip>"
          - name: SSH_TARGET_PORT
            value: "80"
          - name: SSH_MODE
            value: "-R"
        volumeMounts:
          - name: keys
            mountPath: /id_rsa
      nodeName: <node_with_ingress_enabled>
      hostNetwork: true
      volumes:
        - name: keys
          hostPath:
            path: <node_path_for_ssh_keys>
            type: File

To redirect port 443, create another deployment using the previous one as a reference.

Any public URLs should have their DNS entries pointing to the remote server’s IP.

Section break
September 09, 2020

#Install Kubernetes on Raspberry Pi OS

We will use k3s, a lightweight Kubernetes distribution, to get the most out of our hardware. This tutorial uses a Raspberry Pi 4 and the latest Raspberry Pi 32-bit (formerly known as Raspbian). The 64-bit version is pretty much the same for this step-by-step.

Flash the OS image on your SD Card and, if necessary, add Wi-Fi credentials so you can access it.

Enable cgroups support and disable IPv6 by appending the following on /boot/cmdline.txt (remember that /boot refers to the boot partition on your SD Card).

cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory ipv6.disable=1

I also recommend disabling swap.

dphys-swapfile swapoff && systemctl disable dphys-swapfile.service

If your workloads don’t require GPU, you may want to change the Memory Split to 16 using raspi-config. You’ll have a little extra RAM this way.

You can force the OS to use legacy iptables to ensure compatibility with older Kubernetes versions, but this step is optional.

iptables -F
update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

Install and test k3s.

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable=traefik" sh -

# After a minute, you should be able to test it
kubectl get nodes

The kubeconfig yaml will be available at /etc/rancher/k3s/k3s.yaml.

If you have worker nodes to add, use the following command. Find the cluster token at /var/lib/rancher/k3s/server/node-token on the server node.

curl -sfL https://get.k3s.io | K3S_URL=https://<server_ip>:6443 K3S_TOKEN="<cluster_token>" sh -

Maintenance

Things should run smoothly from here, but if they don’t, the next few commands might help:

# See current status of k3s
systemctl status k3s

# See system logs (including k3s)
journalctl -xe

# Uninstall k3s
/usr/local/bin/k3s-uninstall.sh

Kubernetes automatically performs cleanup of unused containers and images on nodes. Still, if you’re under the impression that your filesystem is getting bloated by these, you can force a safe deletion with:

k3s crictl rmi --prune

Further reading

Configuration options for k3s

Section break
September 08, 2020

#Use the RGB Cooling HAT, by Yahboom, with Docker on your Raspberry Pi

This tutorial was tested on a Raspberry Pi 4 with RGB Cooling HAT model “YB-EBV02 VER1.1”, by Yahboom, on 32-bit and 64-bit OS.

The hardware creators do provide installation documentation and official code, but putting all parts together can be challenging.

If you’re willing to use Docker and run the 32-bit version of Raspberry Pi OS, you can activate everything with one command (after enabling I2C with raspi-config).

docker run -d --restart unless-stopped --network host --privileged laury/raspberry-pi-rgb-cooling-hat:latest

To run the software on a 64-bit OS, you’ll need to enable Qemu first and then run the container with the --platform flag.

docker run --privileged --rm tonistiigi/binfmt --install all

docker run -d --restart unless-stopped --platform="linux/arm/v7" --network host --privileged laury/raspberry-pi-rgb-cooling-hat:latest

The container image tagged with latest enables the default functionality of the accessory. If you’d like for the RGBs to the turned off, use the v1.1.0-noRGB image tag.

You can also build the Docker image yourself using the following Dockerfile. Just remember to target ARMv7 architecture. The easiest way to achieve that is to build in the Raspberry Pi itself.

FROM python:2.7

RUN apt update &&\
    apt install -y i2c-tools git

RUN pip install Adafruit-GPIO==1.0.3 \
                Adafruit-BBIO==1.2.0 \
                Adafruit-SSD1306==1.6.2 \
                smbus==1.1.post2 \
                image==1.5.32 \
                vcgencmd==0.1.1 \
                RPi.GPIO==0.7.1

WORKDIR /tmp
RUN git clone --depth 1 https://github.com/raspberrypi/firmware.git &&\
    cp -a firmware/hardfp/opt/vc/* /usr

RUN git clone https://github.com/YahboomTechnology/Raspberry-Pi-RGB-Cooling-HAT.git &&\
    unzip Raspberry-Pi-RGB-Cooling-HAT/4.Python\ programming/RGB_Cooling_HAT.zip &&\
    mkdir /app &&\
    cp -a RGB_Cooling_HAT/* /app &&\
    rm -rf /tmp/*

WORKDIR /app
CMD python /app/RGB_Cooling_HAT.py

Section break
August 29, 2020

#Connect to a headless Raspberry Pi through SSH for the first time

This will work for Raspberry Pi OS (formerly known as Raspbian) and no monitor or keyboard are needed.

After flashing the OS, create a wpa_supplicant.conf file on the boot partition of your SD Card (not the boot folder).

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=<Insert 2 letter ISO 3166-1 country code here>

network={
 ssid="<Name of your wireless LAN>"
 psk="<Password for your wireless LAN>"
}

On the same partition, create an empty file called ssh. It will instruct the OS to enable the SSH server. Also, create your SSH user by creating a file named userconf.txt.

touch ssh

echo <user>:`echo '<pass>' | openssl passwd -6 -stdin` > userconf.txt

To find the Raspberry Pi in your local network, you can use nmap. Assuming your local addresses start with 192.168.0, run:

nmap -sn 192.168.0.0/24

If your Pi connected correctly, you will see something similar to the following in the output.

Nmap scan report for raspberrypi (<IP>)
Host is up (0.11s latency).
MAC Address: <MAC> (Raspberry Pi Trading)

Further reading

Official docs on remote access and Wi-Fi settings.

Security concerns on default user and password for SSH

Section break
August 20, 2020

#Issue Let's Encrypt certificates for domains in a Kubernetes cluster

We will use cert-manager to issue HTTPS certificates for domains served publicly by a Kubernetes cluster.

The application can be installed via Helm, but it’s recommended to install CRDs (Custom Resource Definitions) separately. That may come in handy if you need to delete cert-manager without losing already issued certificates. Don’t forget to change the “version” accordingly on all commands.

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.1/cert-manager.crds.yaml

Add JetStack charts to your Helm client.

helm repo add jetstack https://charts.jetstack.io --force-update
helm repo update

Install the application.

helm upgrade --install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.14.1 \
  --set ingressShim.defaultIssuerName=letsencrypt-prod \
  --set ingressShim.defaultIssuerKind=ClusterIssuer \
  --set ingressShim.defaultIssuerGroup=cert-manager.io

To issue actual certificates, you will need a production Issuer or ClusterIssuer. The following yaml defines a ClusterIssuer and assumes that you have a nginx ingress controller. If it suit your needs, apply it to your cluster.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: <your_email>
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-prod
    # Enable the HTTP-01 challenge provider
    solvers:
    - http01:
        ingress:
          ingressClassName: nginx

Verify if your set up is working.

$ kubectl get clusterissuers -A
NAME               READY   AGE
letsencrypt-prod   True    30m

The cert-manager stack will act upon ingresses that comply with a couple of conditions to generate HTTPS certificates. The following example illustrates that. Take special note of the annotations and the “secretName” property under “tls”.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    kubernetes.io/tls-acme: "true"
  name: nginx-ingress
  namespace: ingress-testing
spec:
  rules:
  - host: <public_URL>
    http:
      paths:
      - backend:
          serviceName: <service_name>
          servicePort: 80
  tls:
  - hosts:
    - <public_URL>
    secretName: <secret_name_of_your_choosing>

Further reading

Official documentation on the installation process

Information about Cluster Issuers

Supported Ingress annotations