MQTT Authentication

Intra-Cluster Communication#

The above image shows how intra-cluster communication takes place. Nodes send application status and node health reports to the cluster-service-manager. The cluster-service-manager then appropriately propagates this information to other nodes and the root-service-manager. Since the nodes report sensitive information, such as IP Addresses, we highly recommend securing the MQTT channels.

MQTTS#

MQTT supports exchanging certificates to establish a TLS-secured channel. For this, the server (MQTT Broker) and every client require a certificate-key file pair signed against the same certificate Authority (CA). The MQTT broker can be configured to only accept incoming secured connections and to identify devices by their certificate common name (CN) entry. MQTT supports the exchanging of certificates to establish a TLS secured channel. For this the server (MQTT Broker) and every client require a certificate-keyfile pair singed against the same certificate Authority (CA). The MQTT broker can be configured to only accept incoming secured connection, and to identify devices by their certificate common name (CN) entry.

Enable Mosquitto Authentication#

Configuring the Cluster Manager#

Navigate into the cluster_orchestrator directory in the oakestra repository.

1. Configure the MQTT Broker by adding the following lines to the mosquitto/mosquitto.conf file:

   cafile /certs/ca.crt
   certfile /certs/server.crt
   keyfile /certs/server.key
   
   require_certificate true
   use_identity_as_username true

2. Generate the certificates in the ./certs directory

   • MQTTS (Server):
     1. Generate CA authority key:

        openssl req -new -x509 -days <duration> -extensions v3_ca -keyout ca.key -out ca.crt

     2. Generate a server key:

        openssl genrsa -out server.key 2048

     3. Generate a certificate signing request including the URL as a SAN:

        openssl req -out server.csr -key server.key -new -addext "subjectAltName = IP:${SYSTEM_MANAGER_URL}, DNS:mqtts"

        When prompted for the CN, enter mqtts
     4. Send the CSR to the CA:

        openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days <duration> -copy_extensions copyall

     5. Grant permissions to read the server keyfile:

        chmod 0644 server.key

   • Cluster Manager (Client):
     1. Generate a client key:

        openssl genrsa -aes256 -out cluster.key 2048

     2. Generate a certificate signing request:

        openssl req -out cluster.csr -key cluster.key -new

        When prompted for the CN, enter cluster_manager
     3. Send the CSR to the CA:

        openssl x509 -req -in cluster.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out cluster.crt -days <duration>

     4. Export the keyfile password as an environment variable:

        export CLUSTER_KEYFILE_PASSWORD=<keyfile password>

   • Cluster Service Manager (Client):
     1. Generate a client key:

        openssl genrsa -aes256 -out cluster_net.key 2048

     2. Generate a certificate signing request:

        openssl req -out cluster_net.csr -key cluster_net.key -new

        When prompted for the CN, enter cluster_service_manager
     3. Send the CSR to the CA:

        openssl x509 -req -in cluster_net.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out cluster_net.crt -days <duration>

     4. Export the keyfile password as an environment variable:

        export CLUSTER_SERVICE_KEYFILE_PASSWORD=<keyfile password>

3. Deploy the cluster with the MQTTS override

   sudo -E docker compose -f docker-compose.yml -f override-mosquitto-auth.yml

Configuring a Node#

1. Copy the ca.crt and ca.key files to the worker node.
2. Generate the certificates
   1. Generate a client key:

      openssl genrsa -aes256 -out client.key 2048

   2. Generate a certificate signing request:

      openssl req -out client.csr -key client.key -new

      When prompted for the CN, enter the public IP of the machine
   3. Send the CSR to the CA:

      openssl x509 -req -in client.csr -CA <path to ca file> -CAkey <path to ca key file> -CAcreateserial -out client.crt -days <duration>

   4. Decrypt the keyfile:

      openssl rsa -in client.key -out unencrypt_client.key

   5. Tell your OS to trust the certificate authority by placing the ca.crt file in the /etc/ssl/certs/ directory
3. Run the NodeEngine:

   sudo NodeEngine -n 0 -p 10100 -a <SYSTEM_MANAGER_URL> -c <path to client.crt> -k <path to unencrypt_client.key>

4. (Optional) Configure the NetManager:
   1. Edit the /etc/netmanager/netcfg.json file so that the "MqttCert" and "MqttKey" fields specify the path to the node certificate and key files (The NetManager should use the same certificate-keyfile pair as the NodeEngine)
   2. Run the NetManager:

      sudo NetManager -p 6000

   3. Run the NodeEngine:

      sudo NodeEngine -n 6000 -p 10100 -a <SYSTEM_MANAGER_URL> -c <path to client.crt> -k <path to unencrypt_client.key>

Finishing up#

Let’s check if all the components succesfully registered with the MQTT Broker via TLS.

1. Cluster Manager: Check the docker compose logs with

   docker compose logs | grep cluster_manager
   docker compose logs | grep cluster_service_manager

   Look for the following lines:

   service_manager - INFO - MQTT - TLS configured
   cluster_manager - INFO - MQTT - TLS configured

2. Node: Both the NodeEngine and NetManager should display the following log line after execution:

   MQTT - Configuring TLS

If everything looks good then Congrats 🎉, your MQTT channels are now secured! When adding any further components, be sure to always give them a unique CN, as this is used to identify the device.