How to deploy Django project to Dokku with Docker

Django Heroku Tutorial Series:

Heroku vs AWS Which is Best for Your Django project

How to deploy Django project to Heroku using Docker

How to deploy Python project to Heroku in Gitlab CI

How to use Heroku Pipeline

Heroku Logs Tutorial

How to monitor Heroku Postgres using heroku-pg-extras

Django Dokku Tutorial Series:

How to deploy Django project to Dokku

How to deploy Django project to Dokku with Docker

Introduction

In this post, I will talk about how to deploy Django project to Dokku with Docker, it would use Postgres db and Amazon S3 to store data and media files.

After reading this tutorial, you will get:

How to install Dokku, config Dokku server
How to create, manage Dokku app.
How to test Docker image for Django project in local env.

The source code of this post can be found on Github, please give it star if it helps you.

Heroku Buildpacks and Dockerfile

By default, Dokku would use Heroku's buildpacks to deploy project.

But Dockerfile provides us a more flexible way so we can take more control, you can install any packages as you like to the OS, or also run any commands during the deployment process.

Generate SSH Key on Local

If you already have SSH key, you can skip this section.

# change to your email address
(local)$ ssh-keygen -t ed25519 -C "[email protected]"

When you're prompted to "Enter a file in which to save the key," press Enter. This accepts the default file location.
At the prompt, type a secure passphrase

If you meet problem, please check Generating a new SSH key to learn more.

/Users/michaelyin/.ssh

├── id_ed25519                        # new
├── id_ed25519.pub                    # new

id_ed25519 is private key, you should not share it to other people.
id_ed25519.pub is the public key.

Create New User

After we SSH to our server as root, let's create normal user first.

(root)-# adduser michaelyin

enter some long and complex passwd

Add user to sudo group

(root)-# usermod -a -G sudo michaelyin
(root)-# groups michaelyin

michaelyin : michaelyin sudo

Make sudo without passwd, note that this setting is not recommended if the vps requires more safety

(root)-# visudo

# only edit this line
%sudo ALL=NOPASSWD: ALL

Now we can change to new user

(root)-# su michaelyin

[email protected]:/root$
'$' denote a normal user prompt

Improve SSH Experience

# make sure we login as normal user instad of root
(server)$ whoami
michaelyin

(server)$ cd ~
(server)$ mkdir .ssh && touch .ssh/authorized_keys
(server)$ chmod 700 .ssh && chmod 600 .ssh/*

Now on local, upload the ssh public key to the vps

(local)$ cat ~/.ssh/id_ed25519.pub | ssh [email protected]  "cat >> ~/.ssh/authorized_keys"

Now on local, we can login ssh [email protected] without typing the complex password, which is convenient.

Please add code below to your ~/.ssh/config on local.

Host dokku_server
    RequestTTY yes
    User michaelyin
    Hostname 138.197.208.68
    Port 22
    ServerAliveInterval 60
    ServerAliveCountMax 60

Now we can use ssh dokku_server instead of ssh [email protected] to login to your server

Install Dokku

We can get the install script on the Dokku release page

Let's SSH to our server and then use code below to install Dokku

# Please use ssh to login to your server
(server)$ whoami
michaelyin

(server)$ cd
(server)$ wget https://raw.githubusercontent.com/dokku/dokku/v0.26.5/bootstrap.sh
(server)$ sudo DOKKU_TAG=v0.26.5 bash bootstrap.sh

The script would also install dependencies and Dokku on the server.

Setup Dokku

After we install Dokku, we will see something like this in the terminal

! Setup a user's ssh key for deployment by passing in the public ssh key as shown:

echo 'CONTENTS_OF_ID_RSA_PUB_FILE' | sudo dokku ssh-keys:add admin

Let's add our public SSH to the Dokku.

# run this command on local
# need root password
(local)$ cat ~/.ssh/id_ed25519.pub | ssh [email protected]_server dokku ssh-keys:add admin

# let's check on server
$ sudo dokku ssh-keys:list

SHA256:XXXXXX NAME="admin" SSHCOMMAND_ALLOWED_KEYS="no-agent-forwarding,no-user-rc,no-X11-forwarding,no-port-forwarding"

Config Dokku App

Next, let's create and config our Dokku app.

Please note that, the dokku project has name django-dokku-docker.

Create Dokku app

$ sudo dokku apps:create django-dokku-docker
-----> Creating django-dokku-docker...

Create Postgres DB

Here we put the Postgres db on our server, but you can also use 3-party DB server like Amazon RDS if you like.

Dokku has many plugins and here we use postgres plugin to help us.

# install the plugin
$ sudo dokku plugin:install https://github.com/dokku/dokku-postgres.git

# let's create db, I add suffix '-db'
# we can even use --image-version to specify postgres version
$ sudo dokku postgres:create django-dokku-docker-db --image "postgres" --image-version "14.1"

       Waiting for container to be ready
       Creating container database
       Securing connection to database
=====> Postgres container created: django-dokku-docker-db
=====> django-dokku-docker-db postgres service information
       Config dir:          /var/lib/dokku/services/postgres/django-dokku-docker-db/data
       Config options:
       Data dir:            /var/lib/dokku/services/postgres/django-dokku-docker-db/data
       Dsn:                 postgres://postgres:[email protected]ker-db:5432/django-dokku-docker-db
       Exposed ports:       -
       Id:                  4919a29ec4820fbf758482b7d91d53ae8fa0bcc27e9ea9bb1bbc6cc0895ca43b
       Internal ip:         172.17.0.2
       Links:               -
       Service root:        /var/lib/dokku/services/postgres/django-dokku-docker-db
       Status:              running
       Version:             postgres:14.1

Now link the db to your dokku app, and it would add a new env varialbe DATABASE_URL to the dokku app.

$ sudo dokku postgres:link django-dokku-docker-db django-dokku-docker
-----> Setting config vars
       DATABASE_URL:  postgres://postgres:[email protected]ker-db:5432/django_dokku_docker_db
-----> Restarting app django-dokku-docker
 !     App image (dokku/django-dokku-docker:latest) not found

Let's print out the env of our Dokku app.

$ sudo dokku config:show django-dokku-docker

=====> django-dokku-docker env vars
DATABASE_URL:  postgres://postgres:[email protected]ker-db:5432/django_dokku_docker_db

Config Amazon s3

Unlike Heroku, Dokku also support store media files on local disk, you can check Dokku Persistent Storage for more detail

If you have no Amazon service account, please go to Amazon S3 and click the Get started with Amazon S3 to signup.
Login AWS Management Console
In the top right, click your company name and then click My Security Credentials
Click the Access Keys section
Create New Access Key, please copy the AMAZON_S3_KEY and AMAZON_S3_SECRET to notebook.

Next, we start to create Amazon bucket on S3 Management Console, please copy Bucket name to notebook.

Bucket in Amazon S3 is like top-level container, every site should have its own bucket, and the bucket name are unique across all Amazon s3, and the url of the media files have domain like {bucket_name}.s3.amazonaws.com.

Now we add AWS_STORAGE_BUCKET_NAME, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_S3_REGION_NAME env variable to our Dokku app.

Please use the values from your notebook.

$ sudo dokku config:set --no-restart django-dokku-docker AWS_STORAGE_BUCKET_NAME='CHANGE ME'
$ sudo dokku config:set --no-restart django-dokku-docker AWS_ACCESS_KEY_ID='CHANGE ME'
$ sudo dokku config:set --no-restart django-dokku-docker AWS_SECRET_ACCESS_KEY='CHANGE ME'
$ sudo dokku config:set --no-restart django-dokku-docker AWS_S3_REGION_NAME='CHANGE ME'

By default, Dokku would restart the app if you changed env variable, here we use --no-restart to tell Dokku not restart it.

Config other env variable

There are still some env variables we need to config, please use dokku config:set --no-restart django-dokku-docker to add them to Dokku app.

DJANGO_ALLOWED_HOSTS:     *
DJANGO_SECRET_KEY:        xxxxxxxxx
DJANGO_SETTINGS_MODULE:   django_dokku_docker.settings
DJANGO_DEBUG:             False

Config domain

Next, we start to config domain.

$ sudo dokku domains:add django-dokku-docker dokku.accordbox.com
-----> Added dokku.accordbox.com to django-dokku-docker
 !     No web listeners specified for django-dokku-docker

Here we add to dokku.accordbox.com to our Dokku app django-dokku-docker, we can add more than one domain to Dokku app.

Config Django project

Now the Dokku app env is ready, before pushing code to Dokku server, let's prepare our Django project.

You would also see we need to add some config files, which are very similar with Heroku's config file, Dokku would scan and read them to decide some deployment workflow.

Environment

First, let's add django-environ to requirements.txt.

django-environ==0.8.1

And then, let's update Django settings django_dokku_docker/settings.py to use django-environ to read the env.

import environ                           # new
from pathlib import Path

env = environ.Env()                      # new


# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = env('DJANGO_SECRET_KEY', default='django-insecure-$lko+#jpt#ehi5=ms9(6s%&6fsg%r2ag2xu_2zj1ibsj$pckud')

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = env.bool("DJANGO_DEBUG", True)

ALLOWED_HOSTS = env.list("DJANGO_ALLOWED_HOSTS", default=[])

Notes:

Here we use django-environ to set SECRET_KEY, DEBUG and ALLOWED_HOSTS from the Environment variable.
And we also set default value to make it work in development without extra Env.

$ ./manage.py migrate
$ ./manage.py runserver

Now, please check on http://127.0.0.1:8000/ to make sure everything is working.

Static files

To serve static assets, we need to use a 3-party package. whitenoise.

Add whitenoise to requirements.txt

whitenoise==5.3.0

Update django_dokku_docker/settings.py

import os                                                                            # new

MIDDLEWARE = [
   'django.middleware.security.SecurityMiddleware',
   'whitenoise.middleware.WhiteNoiseMiddleware',                                     # new
   # ...
]

STATIC_URL = 'static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')                                       # new

STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"      # new

That is all the config you need.

Store media in Amazon s3

update requirements.txt.

boto3==1.16.56
django-storages==1.11.1

Add storages to INSTALLED_APPS in django_dokku_docker/settings.py

And then update django_dokku_docker/settings.py

MEDIA_ROOT = os.path.join(BASE_DIR, 'media')                                      # new
MEDIA_URL = '/media/'                                                             # new

if 'AWS_STORAGE_BUCKET_NAME' in env:                                              # new
    AWS_STORAGE_BUCKET_NAME = env('AWS_STORAGE_BUCKET_NAME')
    AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
    AWS_ACCESS_KEY_ID = env('AWS_ACCESS_KEY_ID')
    AWS_SECRET_ACCESS_KEY = env('AWS_SECRET_ACCESS_KEY')
    AWS_S3_REGION_NAME = env('AWS_S3_REGION_NAME')
    AWS_DEFAULT_ACL = 'public-read'
    AWS_S3_FILE_OVERWRITE = False

    MEDIA_URL = "https://%s/" % AWS_S3_CUSTOM_DOMAIN
    DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

AWS_S3_FILE_OVERWRITE please set it to False, so this can let the storage handle duplicate filenames problem. (I do not understand why so many blog posts did not mention this)

Please note the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY is for AWS admin, for better security, you should create AWS IAM user and then grant S3 permissions. You can check Bucket owner granting its users bucket permissions to learn more.

Postgres DB

Update django_dokku_docker/settings.py

if "DATABASE_URL" in env:
    DATABASES['default'] = env.db('DATABASE_URL')
    DATABASES["default"]["ATOMIC_REQUESTS"] = True

Here env.db would convert DATABASE_URL to Django db connection dict for us.

Do not forget to add psycopg2-binary to requirements.txt.

psycopg2-binary==2.9.2

Procfile

This file exists in root directory of our project. It contains entry command of our service.

web: gunicorn django_dokku_docker.wsgi:application

release: django-admin migrate --noinput

Here web means the web service, Dokku would use the command to run for our web service.

And release is the command which would be run in release stage, here we use the command to migrate our database in release stage.

If you used Celery worker in your project, you can add worker: ... to do that.

Dockerfile

A Dockerfile is a text document that contains all the commands a user could call on the command line to build an image.

Create Dockerfile

# Please remember to rename django_dokku_docker to your project directory name
FROM python:3.10-slim-buster

WORKDIR /app

ENV PYTHONUNBUFFERED=1 \
    PYTHONPATH=/app \
    DJANGO_SETTINGS_MODULE=django_dokku_docker.settings \
    PORT=8000 \
    WEB_CONCURRENCY=3

# Install system packages required by Wagtail and Django.
RUN apt-get update --yes --quiet && apt-get install --yes --quiet --no-install-recommends \
    build-essential curl \
    libpq-dev \
    libmariadbclient-dev \
    libjpeg62-turbo-dev \
    zlib1g-dev \
    libwebp-dev \
 && rm -rf /var/lib/apt/lists/*

RUN addgroup --system django \
    && adduser --system --ingroup django django

# Requirements are installed here to ensure they will be cached.
COPY ./requirements.txt /requirements.txt
RUN pip install -r /requirements.txt

# Copy project code
COPY . .

RUN python manage.py collectstatic --noinput --clear

# Run as non-root user
RUN chown -R django:django /app
USER django

# Run application
CMD gunicorn django_dokku_docker.wsgi:application

Please note that WORKDIR docker instruction, it sets the working directory for docker instructions (RUN, COPY, etc.) It is very like the cd command in shell, but you should not use cd in Dockerfile.
We use ENV to set the default env variable.
We added a django user and used it to run the command for security.

Add gunicorn to requirements.txt

gunicorn==20.1.0

PS: If you have not installed Docker, please check this Docker install Doc

# let's test the docker image on local
$ docker build -t django_dokku:latest .

If you have some problem in Dockerfile, you can troubleshoot in this step.

Note: If you need to build frontend assets in Dockerfile, please check How to deploy Django project to Heroku using Docker for more details.

Deploy project

Please commit code first and then keep reading.

First we add a remote branch dokku to our Git repo.

$ git remote add dokku [email protected]_server:django-dokku-docker

Notes:

dokku_server here can also be the ip address of the server (dokku_server is the hostname we config in ~/.ssh/config),
django-dokku-docker is the dokku app we just created on our server.

Then we push our code to remote master branch

$ git push dokku master:master

Below is the output, Dokku would check language we use and download relevant Heroku buildpack to deploy the project. Which is very cool!

Counting objects: 177, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (162/162), done.
Writing objects: 100% (177/177), 49.38 KiB | 3.53 MiB/s, done.
Total 177 (delta 45), reused 0 (delta 0)

=====> Application deployed:
       http://dokku.accordbox.com

To dokku_server:django-dokku-docker
 * [new branch]      master -> master

If the push command did not raise error, you should see something like this and now our app is live on http://dokku.accordbox.com

Let's check http://dokku.accordbox.com/admin to make sure Django app is running without issue.

Setup SSL

Let's make https work for our site.

(server)$ sudo dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
(server)$ sudo dokku config:set --no-restart --global DOKKU_LETSENCRYPT_EMAIL=mic[email protected]
(server)$ sudo dokku letsencrypt:enable django-dokku-docker
(server)# this would setup cron job to update letsencrypt certificate
(server)$ sudo dokku letsencrypt:cron-job --add

Please replace DOKKU_LETSENCRYPT_EMAIL with your email.

After https can work on your site, you can setup http->https to improve security, Django SECURE_SSL_REDIRECT can help.

Some Advanced issues

Run Dokku command without sudo

It is annoying to make everything dokku command have sudo

From the Dokku doc

(server)$ sudo visudo /etc/sudoers

# add following line
%dokku ALL=(ALL:ALL) NOPASSWD:SETENV: /usr/bin/dokku

This means any user in dokku group can run /usr/bin/dokku without password.

Next, let's add user michaelyin to the dokku group

$ sudo usermod -a -G dokku michaelyin

# exit and login again
$ groups
michaelyin sudo dokku

As you can see, now michaelyin is in dokku group, and we can run dokku command without sudo

$ dokku apps:list
=====> My Apps
django-dokku-docker

Troubleshoot

It is common if your Django projects return 500 error page in some cases, so how to troubleshoot?

I strongly recommend to use https://sentry.io/ to help you solve this, the free plan is enough for personal projects.
Or you can check the running logs using this command dokku logs django-dokku-docker

Customize Nginx

Sometimes, we need to customize our Nginx settings.

For example, the default Nginx settings has small client_max_body_size, so I need to change it to make upload feature work in my Django project.

Dokku generate nginx config from nginx.conf.sigil, you can first download it from Dokku repo (better download from specific Git tag you use).

And then you can modify your settings.

You can add that file to root directory of your project and Dokku would use it to generate new nginx config.

You can check Doc: Customizing the nginx configuration
In the new version, you can set client_max_body_size with dokku command, https://dokku.com/docs/configuration/nginx/#specifying-a-custom-client_max_body_size

Zero Downtime Deploy

When Dokku deploy, it would start container which has the latest code and then wait for 10 secs to make sure the service is ok to run.

To decrease the time, we can add CHECKS file and Dokku would use that file to check if our web server is ok to serve.

/admin                      Django

It tells Dokku to visit /admin url and check if the response contains Django

-----> Deploying web (count=1)
       Attempting pre-flight checks (web.1)
       CHECKS expected result: http://localhost/admin => "Django" (web.1)
       Attempt 1/5. Waiting for 5 seconds (web.1)
       All checks successful (web.1)

Run Django command

If you want to run some command on web server, for example, run Django shell, you can use command in this way

(server)$ dokku run django-dokku-docker python manage.py shell

Here Dokku would create a new container for you to use and this can avoid some risky operation.

The container is deleted when you exit.

Dokku Client

If you want to run dokku on local without SSH command, you can check Dokku official client

After setting it up, we can enter Django shell on the server:

(local)$ DOKKU_HOST=dokku_server dokku run django-dokku-docker python manage.py shell

Setup Cron job

If you want to setup cron jobs for your project, it is better to do it on host machine.

# we setup crontjob as dokku user
(server)$ sudo crontab -u dokku -e

@daily dokku run django-dokku-docker python manage.py clearsessions

As you can see, we setup clearsessions job to run each day and it helps us clean out expired sessions in db.

Conclusion

In this Dokku tutorial, I showed you how to deploy Django project to Dokku with Docker.

The source code of this post can be found on Github, please give it star if it helps you.

Django Heroku Tutorial Series:

Heroku vs AWS Which is Best for Your Django project

How to deploy Django project to Heroku using Docker

How to deploy Python project to Heroku in Gitlab CI

How to use Heroku Pipeline

Heroku Logs Tutorial

How to monitor Heroku Postgres using heroku-pg-extras

Django Dokku Tutorial Series:

How to deploy Django project to Dokku

How to deploy Django project to Dokku with Docker