How to deploy Django project to Dokku with Docker

Michael Yin

Full Stack Developer

Last updated on Feb 22 2020

Table of Contents

Introduction

This is the #2 of my Django Dokku tutorial, in last post, I talked about how to deploy Django project to Dokku using Heroku Buildpacks.

In this post, I will talk about how to deploy Django project to Dokku with Docker, it would use Postgres db and Amazon S3 to store data and media files.

After reading this tutorial, you will get:

How to install Dokku, config Dokku server
How to create, manage Dokku app.
How to test Docker image for Django project in local env.

The source code of this post can be found on Django dokku docker

Heroku Buildpacks and Dockerfile

By default, Dokku would use Heroku's buildpacks to deploy project.

But Dockerfile provides us a more flexible way so we can take more control, you can install any packages as you like to the OS, or also run any commands during the deployment process.

If you are more interested in the buildpacks way, just check How to deploy Django project to Dokku

Install Dokku

First, you can check Dokku doc for the latest system requirment. (you can ignore this step if you choose the Ubuntu LTS version)

Then you can get the install script in the Dokku release

In this tutorial, below is the script I use to install Dokku. Please make sure run this command on your server

# Please use ssh to login to your server
[email protected]:# wget https://raw.githubusercontent.com/dokku/dokku/v0.18.3/bootstrap.sh
[email protected]:# sudo DOKKU_TAG=v0.18.3 bash bootstrap.sh

The script would also instal Docker and other stuff on your server.

Config Dokku Server

After Dokku is installed on your server, now you need config SSH public key for Dokku service.

Let's say the IP of your server is 165.22.153.24, then you need to visit http://165.22.153.24 in your browser.

You need paste the ssh public key. (you can skip the hostname config here because we would do it in a bit)

TODO

After you are done, please check on your server to make sure the relevant service is not running (because this is risky)

[email protected]:# ps auxf | grep dokku-installer
root     14776  0.0  0.0  12944   920 pts/0    S+   07:34   0:00          \_ grep --color=auto dokku-installer

The install service would be terminated after you config in web browser, as you can see in the output.

Setup ssh server in local

It is annoying to type the ip address to login each time, so we can config our ssh client.

Please do this in local env

You can add the server to your local ~/.ssh/config to help you ssh in more easy way.

Please add code below to your ~/.ssh/config

Host dokku_server
    ForwardAgent yes
    Hostname 165.22.153.24
    Port 22
    ServerAliveInterval 60
    ServerAliveCountMax 60

Now you can use ssh [email protected]_server instead of ssh [email protected] to login to your server

Config Dokku App

Now we start to create and config our Dokku app.

Please note that, the dokku project has name django_dokku_docker.

Create app

[email protected]:# dokku apps:create django_dokku_docker
-----> Creating django_dokku_docker... done

Create Postgres DB

Here we put the Postgres db on our server, but you can also use 3-party DB server like Amazon RDS if you like.

Dokku has many plugins and here we use postgres plugin to help us.

# install the plugin
[email protected]:# sudo dokku plugin:install https://github.com/dokku/dokku-postgres.git

# we can also specify the Postgres db version, if you do not specify here, the latest version would be used.
[email protected]:# export POSTGRES_IMAGE="postgres"
[email protected]:# export POSTGRES_IMAGE_VERSION="10.7"

# let's create db, I add suffix '_db'
[email protected]:# sudo dokku postgres:create django_dokku_docker_db

Status: Downloaded newer image for postgres:10.7
docker.io/library/postgres:10.7
       Waiting for container to be ready
       Creating container database
       Securing connection to database
=====> Postgres container created: django_dokku_docker_db
=====> Container Information
       Config dir:          /var/lib/dokku/services/postgres/django_dokku_docker_db/config
       Data dir:            /var/lib/dokku/services/postgres/django_dokku_docker_db/data
       Dsn:                 postgres://postgres:[email protected]mple-db:5432/django_dokku_docker_db
       Exposed ports:       -
       Id:                  a92fff08e4e36f60bb250bccc5816f576bc8a97b4398a0c6e228c24749ef1397
       Internal ip:         172.17.0.2
       Links:               -
       Service root:        /var/lib/dokku/services/postgres/django_dokku_docker_db
       Status:              running
       Version:             postgres:10.7

As you can see, POSTGRES_IMAGE and POSTGRES_IMAGE_VERSION is used to download the docker image. So you can use them to specify the Postgres db version you want to use.

Now link the db to your dokku app, and it would add a new env varialbe DATABASE_URL to the dokku app.

[email protected]:~# dokku postgres:link django_dokku_docker_db django_dokku_docker
-----> Setting config vars
       DATABASE_URL:  postgres://postgres:[email protected]mple-db:5432/django_dokku_docker_db
-----> Restarting app django_dokku_docker
 !     App django_dokku_docker has not been deployed

Let's print out the env of our Dokku app.

[email protected]:~# dokku config django_dokku_docker
=====> django_dokku_docker env vars
DATABASE_URL:  postgres://postgres:[email protected]ker-db:5432/django_dokku_docker_db

Config Amazon s3

dokku support store media files in local disk, you can check Dokku Persistent Storage for more detail

If you have no Amazon service account, please go to Amazon S3 and click the Get started with Amazon S3 to signup.
Login AWS Management Console
In the top right, click your company name and then click My Security Credentials
Click the Access Keys section
Create New Access Key, please copy the AMAZON_S3_KEY and AMAZON_S3_SECRET to notebook.

If you are new to Amazon and have no idea what is IAM user, you can skip it and set permissions later.

Next, we start to create Amazon bucket on S3 Management Console, please copy Bucket name to notebook.

Bucket in Amazon S3 is like top-level container, every site should have its own bucket, and the bucket name are unique across all Amazon s3, and the url of the media files have domain like {bucket_name}.s3.amazonaws.com.

Now we add AWS_STORAGE_BUCKET_NAME, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env variable to our Dokku app.

$ dokku config:set --no-restart django_dokku_docker AWS_STORAGE_BUCKET_NAME='django-dokku-example'
$ dokku config:set --no-restart django_dokku_docker AWS_ACCESS_KEY_ID=''
$ dokku config:set --no-restart django_dokku_docker AWS_SECRET_ACCESS_KEY=''

By default, Dokku would restart the app if you changed env variable, here we use --no-restart to tell Dokku not restart it.

Config other env variable

There are still some env variables we need to config, please use dokku config:set --no-restart django_dokku_docker to add them to Dokku app.

DJANGO_ALLOWED_HOSTS:  *
DJANGO_SECRET_KEY:     {please generate new one}
DJANGO_SETTINGS_MODULE:   config.settings.production

Here we set DJANGO_ALLOWED_HOSTS to * and later we would config nginx to make domain work.

Config domain

Next, we start to config domain.

[email protected]:~# dokku domains:add django_dokku_docker dokku-docker.accordbox.com
-----> Added dokku-docker.accordbox.com to django_dokku_docker
 !     No web listeners specified for django_dokku_docker

Here we add to dokku-docker.accordbox.com to our Dokku app django_dokku_docker, we can add more than one domain to Dokku app.

Config our Django project

Now the Dokku app env is ready, before pushing code to Dokku server, let's config our Django project.

You would also see we need to add some config files, which are very similar with Heroku's config file, Dokku would scan and read them to decide some deployment workflow.

Store media in Amazon s3

Now let's config Django project to let it use Amazon s3.

# please remember to update requirements.txt or Pipfile
$ pip install boto3
$ pip install django-storages

Add storages to INSTALLED_APPS in settings/base.py

Add config below to settings/production.py

AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME')
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
AWS_S3_FILE_OVERWRITE = False

MEDIA_URL = "https://%s/" % AWS_S3_CUSTOM_DOMAIN
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

Static files

We use whitenoise to help us serve static files on server.

# please remember to update requirements.txt or Pipfile
$ pip install whitenoise

Edit MIDDLEWARE in settings/base.py, put WhiteNoiseMiddleware above all other middleware apart from Django’s SecurityMiddleware:

MIDDLEWARE = [
   'django.middleware.security.SecurityMiddleware',
   'whitenoise.middleware.WhiteNoiseMiddleware',
   # ...
]

Set in settings/production.py

STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

Postgres DB

Now we add code to settings/production.py to make it can understand DATABASE_URL value.

# please remember to update requirements.txt or Pipfile
$ pip install django-environ
$ pip install psycopg2

import environ
env = environ.Env()

if 'DATABASE_URL' in env:
    DATABASES["default"] = env.db("DATABASE_URL")  # noqa F405
    DATABASES["default"]["ATOMIC_REQUESTS"] = True  # noqa F405
    DATABASES["default"]["CONN_MAX_AGE"] = env.int("CONN_MAX_AGE", default=60)  # noqa F405

Procfile

This file exists in root directory of our project. It contains entry command of our service.

web: gunicorn config.wsgi:application

release: python manage.py migrate --noinput

Here web means the web service, Dokku would use the command to run for our web service.

And release is the command which would be run in release stage, here we use the command to migrate our databae in release stage.

If you used Celery worker in your project, you can add worker: ... to do that.

DOKKU_SCALE

This file exists in root directory of our project, it defines container number of our services.

web=1

Here we set web=1 so there would be one web container

Dockerfile

A Dockerfile is a text document that contains all the commands a user could call on the command line to build an image.

# Please remember to rename django_dokku_docker to your project directory name
FROM python:3.6-stretch

# WORKDIR sets the working directory for docker instructions, please not use cd
WORKDIR /app

EXPOSE 8000

# sets the environment variable
ENV PYTHONUNBUFFERED=1 \
    PYTHONPATH=/app \
    DJANGO_SETTINGS_MODULE=config.settings.production \
    PORT=8000 \
    WEB_CONCURRENCY=3

# Install operating system dependencies.
RUN apt-get update -y && \
    apt-get install -y apt-transport-https rsync gettext libgettextpo-dev && \
    rm -rf /var/lib/apt/lists/*

# Install Python requirements.
COPY requirements/production.txt requirements/base.txt ./
RUN pip install -r production.txt

# Copy application code.
COPY . .

# Install assets
RUN python manage.py collectstatic --noinput --clear

# Run application
CMD gunicorn config.wsgi:application

After you create Dockerfile and put it at root of your Django project, it is recommended to test it in local, this can save you time if something is wrong.

PS: If you have not installed Docker, please check this Docker install Doc

$ docker build -t django_dokku:latest .

If you have some problem in Dockerfile, you can troubleshoot in this step.

Deploy project

Please commit code first and then keep reading.

First we add a remote branch dokku to our Git repo.

$ git remote add dokku [email protected]_server:django_dokku_docker

dokku_server here can also be the ip address of the server (dokku_server is the hostname we config in ~/.ssh/config), django_dokku_docker is the dokku app we just created on our server.

Then we push our code to remote branch

$ git push dokku master

Below is the output, Dokku would check language we use and download relevant Heroku buildpack to deploy the project. Which is very cool!

$ git push dokku master
Counting objects: 201, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (186/186), done.
Writing objects: 100% (201/201), 51.00 KiB | 3.92 MiB/s, done.
Total 201 (delta 66), reused 0 (delta 0)
-----> Cleaning up...
-----> Building django_dokku_docker from dockerfile...
-----> Setting config vars
       DOKKU_DOCKERFILE_CMD:  CMD gunicorn config.wsgi:application
remote: build context to Docker daemon  200.2kB
Step 1/12 : FROM python:3.6-stretch
3.6-stretch: Pulling from library/python
db3b6004c61a: Pull complete
f8f075920295: Pull complete
6ef14aff1139: Pull complete
0bbd8b48260f: Pull complete
b35072bbc91e: Pull complete
2a114127ad2d: Pull complete
bb188ce9189c: Pull complete
0c23e11bc19b: Pull complete
Digest: sha256:2f9a31ab290cc6bd80a0ce98d8b8f41777337669c6acb33616640ecfb7af8b6f
Status: Downloaded newer image for python:3.6-stretch
 ---> 61ef22d3ce5f
Step 2/12 : WORKDIR /app
 ---> Running in 9f5285ffac29
Removing intermediate container 9f5285ffac29
 ---> 681f78f3b838
Step 3/12 : ENV PYTHONUNBUFFERED=1     PYTHONPATH=/app     DJANGO_SETTINGS_MODULE=config.settings.production     PORT=8000     WEB_CONCURRENCY=3
 ---> Running in 811a419fab59
Removing intermediate container 811a419fab59
 ---> c740375a3a91
Step 4/12 : RUN apt-get update -y &&     apt-get install -y apt-transport-https rsync gettext libgettextpo-dev &&     rm -rf /var/lib/apt/lists/*
 ---> Running in 9a72ea76cd24
Step 7/12 : COPY . .
 ---> 9d3b3b0f49aa
Step 8/12 : RUN python manage.py collectstatic --noinput --clear
 ---> Running in d471c1e98a3a

288 static files copied to '/app/staticfiles', 840 post-processed.
Removing intermediate container d471c1e98a3a
 ---> 324f62437e2b
Step 9/12 : CMD gunicorn config.wsgi:application
 ---> Running in 7245deb76c05
Removing intermediate container 7245deb76c05
 ---> 97b43412df47
Step 10/12 : LABEL dokku=
 ---> Running in e2300e0bb096
Removing intermediate container e2300e0bb096
 ---> 0cd41243e4f9
Step 11/12 : LABEL org.label-schema.schema-version=1.0
 ---> Running in 2f253d0a4051
Removing intermediate container 2f253d0a4051
 ---> 5516ca5984f5
Step 12/12 : LABEL org.label-schema.vendor=dokku
 ---> Running in 6fe424a650b9
Removing intermediate container 6fe424a650b9
 ---> a2276fb41757
Successfully built a2276fb41757
Successfully tagged dokku/django_dokku_docker:latest
-----> Releasing django_dokku_docker (dokku/django_dokku_docker:latest)...
-----> Deploying django_dokku_docker (dokku/django_dokku_docker:latest)...
 !     Release command declared: 'python manage.py migrate --noinput'
       Operations to perform:
         Apply all migrations: account, admin, auth, contenttypes, sessions, sites, socialaccount, taggit, users, wagtailadmin, wagtailcore, wagtaildocs, wagtailimages, wagtailsearch, wagtailusers
       Running migrations:
-----> App Procfile file found (/home/dokku/django_dokku_docker/DOKKU_PROCFILE)
       DOKKU_SCALE declares scale -> web=1

-----> Attempting pre-flight checks
       For more efficient zero downtime deployments, create a file CHECKS.
       See http://dokku.viewdocs.io/dokku/deployment/zero-downtime-deploys/ for examples
       CHECKS file not found in container: Running simple container check...
-----> Waiting for 10 seconds ...
-----> Default container check successful!
-----> Running post-deploy
-----> Overriding default nginx.conf with detected nginx.conf.sigil
-----> Configuring dokku-docker.accordbox.com...(using app-supplied template)
-----> Creating http nginx.conf
-----> Running nginx-pre-reload
       Reloading nginx
-----> Renaming containers
       Renaming container (64905037695d) cool_haslett to django_dokku_docker.web.1
=====> Application deployed:
       http://dokku-docker.accordbox.com

To dokku_server:django_dokku_docker
 * [new branch]      master -> master

If the push command did not raise error, you should see something like this and now our app is live on http://dokku-docker.accordbox.com

Setup SSL

Let's make https work for our site.

$ dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
$ dokku config:set --no-restart --global DOKKU_LETSENCRYPT_EMAIL=[email protected]
$ dokku letsencrypt django_dokku_docker
# this would setup cron job to update letsencrypt certificate
$ dokku letsencrypt:cron-job --add

Please replace DOKKU_LETSENCRYPT_EMAIL with your email.

After https can work on your site, you can setup http->https to improve security. Please check SECURITY section of config/settings/production.py for more detail.

Some Advanced issues

Customize Nginx

Sometimes, we need to customize our Nginx settings.

For example, the default Nginx settings has small client_max_body_size, so I need to change it to make upload feature work in my Django project.

Dokku generate nginx config from nginx.conf.sigil, you can first download it from Dokku repo (better download from specific Git tag instead of master branch).

And then you can add more modify your settings.

You can add that file to root directory of your project and Dokku would use it to generate new nginx config.

Zero Downtime Deploy

When Dokku deploy, it would start container which has latest code and then wait for 10 secs to make sure the service is ok to run.

To shorten the time, we can add add CHECKS file and Dokku would use that file to check if our web server is ok to serve.

/                       django_dokku_docker

This tell Dokku to visit / url and check if the response contains django_dokku_docker

-----> Attempting pre-flight checks
-----> Attempt 1/5 Waiting for 5 seconds ...
       CHECKS expected result:
       http://localhost/ => "django_dokku_docker"
-----> All checks successful!
=====> django_dokku_docker web container output:
       [2019-10-08 04:41:55 +0000] [9] [INFO] Starting gunicorn 19.9.0
       [2019-10-08 04:41:55 +0000] [9] [INFO] Listening at: http://0.0.0.0:5000 (9)
       [2019-10-08 04:41:55 +0000] [9] [INFO] Using worker: sync
       [2019-10-08 04:41:55 +0000] [204] [INFO] Booting worker with pid: 204
       172.17.0.1 - - [08/Oct/2019:04:42:04 +0000] "GET / HTTP/1.1" 200 3590 "-" "curl/7.47.0"

As you can see, Dokku use curl to check and if the check succeed, it would send traffic to the new container immediatly.

Run Django command

If you want to run some command on web server, for example, run Django shell, you can use command in this way

dokku --rm run django_dokku_docker python manage.py shell

Here Dokku would create a new container for you to use and this can avoid some risky operation. --rm means the container would be delted after you exit.

I always use this to run createsuperuser

Setup Cron job

If you want to setup cron jobs for your project, it is better to do it in host machine.

# we set crontjob as dokku user
$ crontab -u dokku -e

@daily dokku --rm run django_dokku_docker python manage.py clearsessions

As you can see, we setup clearsessions job to run each day and it can helps us clean out expired sessions in db.

Short reviews from Accordbox

Dokku is a very good option if you have limited budget but you still like the Heroku experience or you want to use Heroku on AWS EC2

There is no silver bullet in this world, so I will also talk about limitations of Dokku here.

Because of the design of Dokku, it is not easy to scale the app. So if your project needs HA (high availability), Dokku might not be a good choice here

Conclusion

In this Dokku tutorial, I showed you how to deploy Django project to Dokku with Docker.