Wagtail SEO Guide

Michael Yin

Last updated on December 10 2021

Table of Contents

Wagtail Tutorial Series:

To get the latest learning resource for Wagtail 4, please check Build Blog With Wagtail CMS (4.0.0)

  1. Create Wagtail Project
  2. Dockerizing Wagtail App
  3. Add Blog Models to Wagtail
  4. How to write Wagtail page template
  5. Add Bootstrap Theme to Wagtail
  6. How to use StreamField in Wagtail
  7. Wagtail Routable Page
  8. Add pagination component to Wagtail
  9. Customize Wagtail Page URL
  10. Add Full Text Search to Wagtail
  11. Add Markdown Support to Wagtail
  12. Add LaTeX Support & Code Highlight In Wagtail
  13. How to Build Form Page in Wagtail
  14. How to Create and Manage Menus in Wagtail
  15. Wagtail SEO Guide
  16. Source code: https://github.com/AccordBox/wagtail-tailwind-blog

Wagtail Tips:

  1. Wagtail Tip #1: How to replace ParentalManyToManyField with InlinePanel
  2. Wagtail Tip #2: How to Export & Restore Wagtail Site

Write style in Wagtail:

  1. How to use SCSS/SASS in your Django project (Python Way)
  2. How to use SCSS/SASS in your Django project (NPM Way)

Other Wagtail Topics:

  1. How to make Wagtail project have good coding style
  2. How to do A/B Testing in Wagtail CMS 
  3. How to build a landing page using Wagtail CMS 
  4. How to support multi-language in Wagtail CMS 

More Wagtail articles and eBooks written by me

Objective

By the end of this chapter, you should be able to:

  1. Generate meta tag with wagtail-metadata package.
  2. Generate sitemap, and robots.txt for Wagtail project.
  3. Check 404, 500 page on local env.

Meta Tag

Setup

Add wagtail-metadata to the requirements.txt

Django>=3.1,<3.2
wagtail>=2.11,<2.12
psycopg2-binary
django-extensions==3.1.0

wagtail-markdown==0.6
Pygments
wagtail-django-recaptcha==1.0
django-crispy-forms==1.10.0
wagtailmenus==3.0.2
wagtail-metadata==3.3.1

Add wagtailmetadata to the INSTALLED_APPS in wagtail_bootstrap_blog/settings/base.py

INSTALLED_APPS = [
    # code omitted for brevity

    'modelcluster',
    'taggit',
    'django_extensions',
    'wagtailmarkdown',
    'crispy_forms',
    'captcha',
    'wagtailcaptcha',
    'wagtailmenus',
    'wagtailmetadata',

    # code omitted for brevity
]
$ docker-compose up -d --build
$ docker-compose logs -f

Update blog/models.py

from wagtailmetadata.models import MetadataPageMixin


class PostPage(MetadataPageMixin, Page):

    # code omitted for brevity

Notes:

  1. We make PostPage inherit from MetadataPageMixin (please be careful about the order)

Let's migrate db

$ docker-compose run --rm web python manage.py makemigrations
$ docker-compose run --rm web python manage.py migrate

Add meta data in Wagtail admin

Now we can add meta data in promote tab for the PostPage

  1. slug, page_title, search_description come from Wagtail Page model
  2. The search_image come from by wagtailmetadata.models.MetadataPageMixin

Template

Update wagtail_bootstrap_blog/templates/base.html

<head>
    <meta charset="utf-8" />

    {% block meta_tag %}
      <title>
          {% block title %}
              {% if self.seo_title %}{{ self.seo_title }}{% else %}{{ self.title }}{% endif %}
          {% endblock %}
          {% block title_suffix %}
              {% with self.get_site.site_name as site_name %}
                  {% if site_name %}- {{ site_name }}{% endif %}
              {% endwith %}
          {% endblock %}
      </title>
      <meta name="description" content="{{ page.search_description }}" />
    {% endblock %}

    <meta name="viewport" content="width=device-width, initial-scale=1" />

    {% if page.canonical_url %}
      <link rel="canonical" href="{{ page.canonical_url }}"/>
    {% endif %}

    {# Global stylesheets #}
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.1/css/all.min.css" integrity="sha512-+4zCK9k+qNFUR5X+cKL9EIR+ZOhtIloNl9GIKS57V1MyNsYpYcUrUeQc9vNfzsWfV28IaLL3i96P9sdNyeRssA==" crossorigin="anonymous" />
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/4.5.3/css/bootstrap.min.css" integrity="sha512-oc9+XSs1H243/FRN9Rw62Fn8EtxjEYWHXRvjS43YtueEewbS6ObfXcJNyohjHqVKFPoXXUxwc+q1K7Dee6vv9g==" crossorigin="anonymous" />
    {% block extra_css %}
        {# Override this in templates to add extra stylesheets #}
    {% endblock %}
</head>

Notes:

  1. We created a meta_tag block
  2. If child template does not fill the meta_tag block, meta_tag of base.html would be used instead

Update wagtail_bootstrap_blog/templates/blog/post_page.html

{% extends "base.html" %}

{% load wagtailcore_tags wagtailimages_tags blogapp_tags static wagtailmetadata_tags %}

{% block meta_tag %}
  {% meta_tags %}
{% endblock %}

Notes:

  1. In post_page.html, we fill the meta_tag block with django template tag meta_tags from wagtailmetadata_tags
  2. When we render HTML for PostPage, it would generate HTML using meta_tags and fill the meta_tag block.

If we check the HTML source code, we will see something like this

<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="PostPage1">
<meta name="twitter:description" content="search description test">
<meta name="twitter:image" content="http://127.0.0.1:8000/media/images/image_3.original.jpg">

<meta property="og:url" content="http://127.0.0.1:8000/postpage1/" />
<meta property="og:title" content="PostPage1" />
<meta property="og:description" content="search description test" />
<meta property="og:site_name" content="" />

<meta property="og:image" content="http://127.0.0.1:8000/media/images/image_3.original.jpg" />
<meta property="og:image:width" content="1280" />
<meta property="og:image:height" content="853" />

<meta itemprop='url' content='http://127.0.0.1:8000/postpage1/'/>
<meta itemprop="name" content="PostPage1">
<meta itemprop='description' content='search description test' />
<meta itemprop='image' content='http://127.0.0.1:8000/media/images/image_3.original.jpg' />

<title>PostPage1</title>
<meta name="description" content="search description test">

SiteMap

Please add django.contrib.sitemaps to INSTALLED_APPS in wagtail_bootstrap_blog/settings/base.py

INSTALLED_APPS = [

    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.sitemaps',
]

Update wagtail_bootstrap_blog/urls.py

from django.conf import settings
from django.urls import include, path
from django.contrib import admin

from wagtail.admin import urls as wagtailadmin_urls
from wagtail.core import urls as wagtail_urls
from wagtail.documents import urls as wagtaildocs_urls
from wagtail.contrib.sitemaps.views import sitemap

urlpatterns = [
    path('django-admin/', admin.site.urls),

    path('admin/', include(wagtailadmin_urls)),
    path('documents/', include(wagtaildocs_urls)),
    path('sitemap.xml', sitemap)
]
  1. We added path('sitemap.xml', sitemap), the sitemap view come from Wagtail.
  2. If we visit http://127.0.0.1:8000/sitemap.xml, we will see sitemap is generated.

Notes:

  1. The domain and port are generated from wagtail sites setting, so remember to modify it when deploying.
  2. In Wagtail Page model, there is get_sitemap_urls method and Wagtail would call this method to get info for sitemap item.
  3. For RoutablePageMixin, we can overwrite the get_sitemap_urls method.

Update blog/models.py

class BlogPage(RoutablePageMixin, Page):

    def get_sitemap_urls(self, request=None):
        output = []
        posts = self.get_posts()
        for post in posts:
            post_date = post.post_date
            url = self.get_full_url(request) + self.reverse_subpage(
                'post_by_date_slug',
                args=(
                    post_date.year,
                    '{0:02}'.format(post_date.month),
                    '{0:02}'.format(post_date.day),
                    post.slug,
                )
            )

            output.append({
                'location': url,
                'lastmod': post.last_published_at
            })

        return output


class PostPage(MetadataPageMixin, Page):

    def get_sitemap_urls(self, request=None):
        return []

Notes:

  1. Here we use BlogPage.get_sitemap_urls to generate the date_slug_url for the PostPage
  2. To avoid http://127.0.0.1:8000/postpage4/, we return empty list in PostPage.get_sitemap_urls

The sitemap would seem like this

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://127.0.0.1:8000/2020/12/20/postpage4/</loc>
<lastmod>2020-12-21</lastmod>
</url>
<url>
<loc>http://127.0.0.1:8000/2020/12/20/postpage3/</loc>
<lastmod>2020-12-21</lastmod>
</url>
<url>
<loc>http://127.0.0.1:8000/2020/12/20/postpage2/</loc>
<lastmod>2020-12-21</lastmod>
</url>
<url>
<loc>http://127.0.0.1:8000/2020/12/20/postpage1/</loc>
<lastmod>2020-12-23</lastmod>
</url>
<url>
<loc>http://127.0.0.1:8000/contact/</loc>
<lastmod>2020-12-22</lastmod>
</url>
</urlset>

Robots.txt

Update wagtail_bootstrap_blog/urls.py

import blog.views

urlpatterns = [
    path('robots.txt', blog.views.RobotsView.as_view()),
]

Update blog/views.py

from django.views.generic import TemplateView
from wagtail.core.models import Site


class RobotsView(TemplateView):

    content_type = 'text/plain'
    template_name = 'robots.txt'

    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        request = context['view'].request
        context['wagtail_site'] = Site.find_for_request(request)
        return context

Create wagtail_bootstrap_blog/templates/robots.txt

User-Agent: *
Disallow: /admin/

# Sitemap files
Sitemap: {{ wagtail_site.root_url }}/sitemap.xml

Notes:

  1. Since the Sitemap should be fully-qualified URL so here we use this way to generate proper robots
  2. If you site does not support multi-site feature, you can write static URL and delete the RobotsView.get_context_data method.

Context

Let's review our code before going to the next section.

  1. We already add blog_page in the get_context method of BlogPage, PostPage, and FormPage
  2. We need blog_page because we need it so routablepageurl blog_page xxxx in sidebar can work.
  3. But this is tedious because if we have many page types in our project, then each page model would have code like this.

There are some solutions here:

  1. Create a BasePage class, set blog_page in the get_context, then other pages which inherit BasePage do not need to to that again.
  2. Or we can build a custom context processor, Django doc here

A context processor has a simple interface: It’s a Python function that takes one argument, an HttpRequest object, and returns a dictionary that gets added to the template context.

Here let's choose the latter solution.

Create blog/context_processors.py

from wagtail.core.models import Site
from blog.models import BlogPage


def blog_page(request):
    """
    To avoid multiple Wagtail site query in request-response cycle, you can use
    wagtail.contrib.legacy.sitemiddleware.SiteMiddleware
    """
    wagtail_site = Site.find_for_request(request)
    context = {
        'blog_page': BlogPage.objects.in_site(wagtail_site).first()
    }
    return context

The logic is simple here, in the blog_page function, we return the correct blog_page according to the request.

Update TEMPLATES in wagtail_bootstrap_blog/settings/base.py to add the blog.context_processors.blog_page

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
            os.path.join(PROJECT_DIR, 'templates'),
        ],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'wagtailmenus.context_processors.wagtailmenus',
                'blog.context_processors.blog_page'
            ],
        },
    },
]

Now remove code context['blog_page'] = self.blog_page from blog/models.py (BlogPage, PostPage, FormPage)

We will see everything can still work like a charm.

Custom 404 500 page

It is always better to have custom 404, 500 page to tell user what happened, but how to test the page on local?

Update wagtail_bootstrap_blog/urls.py

from django.views import defaults as default_views


if settings.DEBUG:
    from django.conf.urls.static import static
    from django.contrib.staticfiles.urls import staticfiles_urlpatterns

    # Serve static and media files from development server
    urlpatterns += staticfiles_urlpatterns()
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
    urlpatterns = [
        path('404/', default_views.page_not_found, kwargs={'exception': Exception("Page not Found")}),
        path('500/', default_views.server_error),
    ] + urlpatterns

Now we can visit http://127.0.0.1:8000/404/ to test

Notes:

  1. For 404 page, the sidebar widget can also work
  2. When we visits http://127.0.0.1:8000/404/, Django view is used to return response (not Wagtail)
  3. blog.context_processors.blog_page helps us make blog_page available in the template context so widgets in the sidebar can still work.

3-party tool

There are some great 3-party tools to help us and I'd like to share with you here. I would also appreciate that if you can tell me your tool!

  1. uptimerobot would send notification when our site is down, it has free plan.
  2. deadlinkchecker I'd like to use it to help check broken links for my projects

Wagtail Tutorial Series:

To get the latest learning resource for Wagtail 4, please check Build Blog With Wagtail CMS (4.0.0)

  1. Create Wagtail Project
  2. Dockerizing Wagtail App
  3. Add Blog Models to Wagtail
  4. How to write Wagtail page template
  5. Add Bootstrap Theme to Wagtail
  6. How to use StreamField in Wagtail
  7. Wagtail Routable Page
  8. Add pagination component to Wagtail
  9. Customize Wagtail Page URL
  10. Add Full Text Search to Wagtail
  11. Add Markdown Support to Wagtail
  12. Add LaTeX Support & Code Highlight In Wagtail
  13. How to Build Form Page in Wagtail
  14. How to Create and Manage Menus in Wagtail
  15. Wagtail SEO Guide
  16. Source code: https://github.com/AccordBox/wagtail-tailwind-blog

Wagtail Tips:

  1. Wagtail Tip #1: How to replace ParentalManyToManyField with InlinePanel
  2. Wagtail Tip #2: How to Export & Restore Wagtail Site

Write style in Wagtail:

  1. How to use SCSS/SASS in your Django project (Python Way)
  2. How to use SCSS/SASS in your Django project (NPM Way)

Other Wagtail Topics:

  1. How to make Wagtail project have good coding style
  2. How to do A/B Testing in Wagtail CMS 
  3. How to build a landing page using Wagtail CMS 
  4. How to support multi-language in Wagtail CMS 

More Wagtail articles and eBooks written by me

Launch Products Faster with Django

SaaS Hammer helps you launch products in faster way. It contains all the foundations you need so you can focus on your product.

Michael Yin

Michael is a Full Stack Developer from China who loves writing code, tutorials about Django, and modern frontend tech.

He has published some ebooks on leanpub and tech course on testdriven.io.

He is also the founder of the AccordBox which provides the web development services.

Table of Contents

Django SaaS Template

It aims to save your time and money building your product, developed by Michael Yin

Learn More

Build Jamstack web app with Next.js and Wagtail CMS.

Read More

Subscribe

Get notified about new great Web Development Tutorial