Showing posts with label NGINX Unit. Show all posts
Showing posts with label NGINX Unit. Show all posts

2024-05-02

Renewing Let's Encrypt Certificates with NGINX Unit

Recently, I moved the DjangoTricks website and started PyBazaar on servers with Nginx Unit. One thing that was left undone was SSL certificate renewals. Let's Encrypt has special certbot parameters for renewing certificates for websites on Apache or Nginx servers, but they don't work out of the box with the Nginx Unit. In this blog post, I will tell you how to do that.

The certificate bundle

Nginx Unit doesn't use the fullchain.pem and privkey.pem generated by certbot directly from the location where they were generated. Instead, one has to create a bundle (like bundle1.pem) by concatenating them and then uploading it to the Nginx Unit configuration endpoint.

The bash script

For that, I created a bash script:

#!/usr/bin/env bash
SECONDS=0
CRON_LOG_FILE=/var/webapps/pybazaar/logs/renew_certificate.log

echo "=== Renewing Letsencrypt Certificate ===" > ${CRON_LOG_FILE}
date >> ${CRON_LOG_FILE}

echo "Renewing certificate..." >> ${CRON_LOG_FILE}
certbot --renew-by-default certonly -n --webroot -w /var/www/letsencrypt/ -m hello@pybazaar.com --agree-tos --no-verify-ssl -d pybazaar.com -d www.pybazaar.com

echo "Creating bundle..." >> ${CRON_LOG_FILE}
cat /etc/letsencrypt/live/pybazaar.com/fullchain.pem /etc/letsencrypt/live/pybazaar.com/privkey.pem > /var/webapps/pybazaar/unit-config/bundle1.pem

echo "Temporarily switching the Unit configuration to a dummy one..." >> ${CRON_LOG_FILE}
curl -X PUT --data-binary @/var/webapps/pybazaar/unit-config/unit-config-pre.json --unix-socket /var/run/control.unit.sock http://localhost/config

echo "Deleting old certificate from Nginx Unit..." >> ${CRON_LOG_FILE}
curl -X DELETE --unix-socket /var/run/control.unit.sock http://localhost/certificates/certbot1

echo "Installing new certificate to Nginx Unit..." >> ${CRON_LOG_FILE}
curl -X PUT --data-binary @/var/webapps/pybazaar/unit-config/bundle1.pem --unix-socket /var/run/control.unit.sock http://localhost/certificates/certbot1

echo "Switching the Unit configuration to the correct one..." >> ${CRON_LOG_FILE}
curl -X PUT --data-binary @/var/webapps/pybazaar/unit-config/unit-config.json --unix-socket /var/run/control.unit.sock http://localhost/config

echo "Restarting Unit..." >> ${CRON_LOG_FILE}
service unit restart

echo "Finished." >> ${CRON_LOG_FILE}
duration=$SECONDS
echo "$(($duration / 60)) minutes and $(($duration % 60)) seconds elapsed." >> ${CRON_LOG_FILE}

Once you have adapted the script, you can run it manually as a root user to test it:

$ chmod +x renew_certificate.sh
$ ./renew_certificate.sh

Note that the certbot command will try to validate your website's URL by attempting to reach a temporary file that it will create on http://example.com/.well-known/acme-challenge/, so make sure that this location is accessible and serving the static files.

For more details about the Nginx Unit, check my previous blog post.

The cron job

If everything works as expected, you can add it to the root user's cron jobs to be executed weekly.

Export the current root cron jobs to a crontab.txt:

$ crontab -l > crontab.txt

Then edit it and add the weekly script to update the SSL certificate:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
SHELL=/bin/bash
MAILTO=""
@weekly /var/webapps/pybazaar/unit-config/renew_certificate.sh

Then run the following as the root user to apply it:

$ crontab crontab.txt

The good thing about not editing the cron job with crontab -e is that you can choose the editor and even put the crontab.txt under Git version control.

Happy web development with WSGI or ASGI!


Cover picture by Gotta Be Worth It

2024-02-24

Django Project on NGINX Unit

Django Project on NGINX Unit

Recently, I learned about the NGINX Unit and decided to try it on my DjangoTricks website. Unit is a web server developed by people from NGINX, with pluggable support for Python (WSGI and ASGI), Ruby, Node.js, PHP, and a few other languages. I wanted to see whether it's really easy to set it up, have it locally on my Mac and the remote Ubuntu server, and try out the ASGI features of Django, allowing real-time communication. Also, I wanted to see whether Django is faster with Unit than with NGINX and Gunicorn. This article is about my findings.

My observations

Unit service uses HTTP requests to read and update its configuration. The configuration is a single JSON file that you can upload to the Unit service via a command line from the same computer or modify its values by keys in the JSON structure.

Normally, the docs suggest using the curl command to update the configuration. However, as I am using Ansible to deploy my Django websites, I wanted to create a script I could later copy to other projects. I used Google Gemini to convert bash commands from the documentation to Ansible directives and corrected its mistakes.

The trickiest part for me was to figure out how to use Let's Encrypt certificates in the simplest way possible. The docs are extensive and comprehensible, but sometimes, they dig into technical details that are unnecessary for a common Django developer.

Also, it's worth mentioning that the Unit plugin version must match your Python version in the virtual environment. It was unexpected for me when Brew installed Python 3.12 with unit-python3 and then required my project to use Python 3.12 instead of Python 3.10 (which I used for the DjangoTricks website). So I had to recreate my virtual environment and probably will have problems later with pip-compile-multi when I prepare packages for the production server, still running Python 3.10.

Below are the instructions I used to set up the NGINX Unit with my existing DjangoTricks website on Ubuntu 22.04. For simplicity, I am writing plain Terminal commands instead of analogous Ansible directives.

1. Install Unit service to your server

Follow the installation instructions from documentation to install unit, unit-dev, unit-python3.10, and whatever other plugins you want. Make sure the service is running.

2. Prepare Let's Encrypt certificates

Create a temporary JSON configuration file /var/webapps/djangotricks/unit-config/unit-config-pre.json, which will allow Let's Encrypt certbot to access the .well-known directory for domain confirmation:

{
  "listeners": {
    "*:80": {
      "pass": "routes/acme"
    }
  },
  "routes": {
    "acme": [
      {
        "match": {
          "uri": "/.well-known/acme-challenge/*"
        },
        "action": {
          "share": "/var/www/letsencrypt/$uri"
        }
      }
    ]
  }
}

Install it to Unit:

$ curl -X PUT --data-binary @/var/webapps/djangotricks/unit-config/unit-config-pre.json \
--unix-socket /var/run/control.unit.sock http://localhost/config

If you make any mistakes in the configuration, it will be rejected with an error message and not executed.

Create Let's Encrypt certificates:

$ certbot certonly -n --webroot -w /var/www/letsencrypt/ -m hello@djangotricks.com \
--agree-tos --no-verify-ssl -d djangotricks.com -d www.djangotricks.com

Create a bundle that is required by the NGINX Unit:

cat /etc/letsencrypt/live/djangotricks.com/fullchain.pem \
/etc/letsencrypt/live/djangotricks.com/privkey.pem > \
/var/webapps/djangotricks/unit-config/bundle1.pem

Install certificate to NGINX Unit as certbot1:

curl -X PUT --data-binary @/var/webapps/djangotricks/unit-config/bundle1.pem \
--unix-socket /var/run/control.unit.sock http://localhost/certificates/certbot1

3. Install Django project configuration

Create a JSON configuration file /var/webapps/djangotricks/unit-config/unit-config.json which will use your SSL certificate and will serve your Django project:

{
  "listeners": {
    "*:80": {
      "pass": "routes/main"
    },
    "*:443": {
      "pass": "routes/main",
      "tls": {
        "certificate": "certbot1"
      }
    }
  },
  "routes": {
    "main": [
      {
        "match": {
          "host": [
            "djangotricks.com",
            "www.djangotricks.com"
          ],
          "uri": "/.well-known/acme-challenge/*"
        },
        "action": {
          "share": "/var/www/letsencrypt/$uri"
        }
      },
      {
        "match": {
          "host": [
            "djangotricks.com",
            "www.djangotricks.com"
          ],
        },
        "action": {
          "pass": "applications/django"
        }
      },
      {
        "action": {
          "return": 444
        }
      }
    ]
  },
  "applications": {
    "django": {
      "type": "python",
      "path": "/var/webapps/djangotricks/project/djangotricks",
      "home": "/var/webapps/djangotricks/venv/",
      "module": "djangotricks.wsgi",
      "environment": {
        "DJANGO_SETTINGS_MODULE": "djangotricks.settings.production"
      },
      "user": "djangotricks",
      "group": "users"
    }
  }
}

In this configuration, HTTP requests can only be used for certification validation, and HTTPS requests point to the Django project if the domain used is correct. In other cases, the status "444 - No Response" is returned. (It's for preventing access for hackers who point their domains to your IP address).

In the NGINX Unit, switching between WSGI and ASGI is literally a matter of changing one letter from "w" to "a" in the line about the Django application module, from:

"module": "djangotricks.wsgi",

to:

"module": "djangotricks.asgi",

I could have easily served the static files in this configuration here, too, but my STATIC_URL contains a dynamic part to force retrieval of new files from the server instead of the browser cache. So, I used WhiteNoise to serve the static files.

For redirection from djangotricks.com to www.djangotricks.com, I also chose to use PREPEND_WWW = True setting instead of Unit directives.

And here, finally, installing it to Unit (it will overwrite the previous configuration):

$ curl -X PUT --data-binary @/var/webapps/djangotricks/unit-config/unit-config.json \
--unix-socket /var/run/control.unit.sock http://localhost/config

How it performed

DjangoTricks is a pretty small website; therefore, I couldn't do extensive benchmarks, but I checked two cases: how a filtered list view performs with NGINX and Gunicorn vs. NGINX Unit, and how you can replace NGINX, Gunicorn, and Huey background tasks with ASGI requests using NGINX Unit.

First of all, the https://www.djangotricks.com/tricks/?categories=development&technologies=django-4-2 returned the HTML result on average in 139 ms on NGINX with Gunicorn, whereas it was on average 140 ms with NGINX Unit using WSGI and 149 ms with NGINX Unit using ASGI. So, the NGINX Unit with WSGI is 0.72% slower than NGINX with Gunicorn, and the NGINX Unit with ASGI is 7.19% slower than NGINX with Gunicorn.

However, when I checked https://www.djangotricks.com/detect-django-version/ how it performs with background tasks and continuous Ajax requests until the result is retrieved vs. asynchronous checking using ASGI, I went on average from 6.62 s to 0.75 s. Of course, it depends on the timeout of the continuous Ajax request, but generally, a real-time ASGI setup can improve the user experience significantly.

UPDATE on 2024-09-15: After using ASGI with the NGINX Unit for a while, I noticed that it crashed several times, and the server had to be restarted. It's still unclear whether the issue was due to NGINX Unit instability, Django's ASGI implementation, or simply heavy load. So use ASGI at your own risk.

Final words

Although NGINX Unit with Python is slightly (unnoticeably) slower than NGINX with Gunicorn, it allows Django developers to use asynchronous requests and implement real-time user experience. Also, you could probably have a Django website and Matomo analytics or WordPress blog on the same server. The NGINX Unit configuration is relatively easy to understand, and you can script the process for reusability.


Cover Image by Volker Meyer.