Wagtail SEO Guide


Michael Yin

Full Stack Developer

Last updated on Mar 30 2020


Table of Contents

Introduction

In this Wagtail tutorial, I would talk about something which can help SEO in Wagtail, after reading, you will learn

  1. How to generate meta tags with wagtail-metadata

  2. How to generate sitemap in Wagtail

  3. How to add robots.txt to Wagtail

  4. How to add custom 404, 500 page to Wagtail

Meta tag

Meta tag can better help search engines and social media understand the page content.

Wagtail's page model already has seo_title and search_description, but in most cases, we need a better way for us to generate meta tags.

We can use wagtail-metadata to help us quickly get this job done, and it also provides us a better way to customize if we need more features.

Now let's first install it pip install wagtail-metadata

Then add it to our INSTALLED_APPS

INSTALLED_APPS = [
    ...

    'wagtailmetadata',
]

In wagtailmetadata, there is a MetadataPageMixin for us to use, let's first check the source code.

class MetadataPageMixin(MetadataMixin, models.Model):
    """An implementation of MetadataMixin for Wagtail pages."""
    search_image = models.ForeignKey(
        get_image_model_string(),
        null=True,
        blank=True,
        related_name='+',
        on_delete=models.SET_NULL,
        verbose_name=ugettext_lazy('Search image')
    )

    promote_panels = [
        MultiFieldPanel([
            FieldPanel('slug'),
            FieldPanel('seo_title'),
            FieldPanel('show_in_menus'),
            FieldPanel('search_description'),
            ImageChooserPanel('search_image'),
        ], ugettext_lazy('Common page configuration')),
    ]

    def get_meta_url(self):
        return self.full_url

    def get_meta_title(self):
        return self.seo_title or self.title

    def get_meta_description(self):
        return self.search_description

    def get_meta_image(self):
        return self.search_image

    class Meta:
        abstract = True

As you can see, it give us search_image field, which can help us generate preview image in cards when sharing on FB or Twitter, it also provide some methods for us to use in the template.

Let's add it to our page models.

from wagtailmetadata.models import MetadataPageMixin


class PostPage(MetadataPageMixin, Page):

    body = RichTextField(blank=True)

    content_panels = [
        FieldPanel('body'),
    ]

Please remember to migrate db when you are done

$ python manage.py makemigrations
$ python manage.py migrate

Now please add code below to your template file just like this.

{% load wagtailmetadata_tags %}

<html>
<head>

  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">

  {% meta_tags %}

</head>

In Wagtail admin, you will see a new field search_image in promote tab

And if you set the value and submit, you will see meta tag already generated like this.

Here are some notes:

  1. wagtailmetadata is very simple, and it give us direction if we want to do more with meta tag.

  2. In my projects, I'd like to create a BasePage for this, so I do not need to add MetadataPageMixin to all my page models.

SiteMap

Sitemap can help search engine better index your web application so you better build it in right way and submit it to search engine.

Please add django.contrib.sitemaps to INSTALLED_APPS

INSTALLED_APPS = [
    ...

    "django.contrib.sitemaps",
]

In urls.py add a url for sitemap

from wagtail.contrib.sitemaps.views import sitemap

urlpatterns = [
    url(r'^sitemap\.xml$', sitemap),
    url(r'', include(wagtail_urls)),
]

If you visit http://127.0.0.1:8000/sitemap.xml, you will see Wagtail would automatically build sitemap from the page trees.

There are some points here.

  1. The domain and port are generated from wagtail sites setting, so remember to config it when deploying.

  2. In Wagtail Page model, there is get_sitemap_urls method and Wagtail would call this method to get info for sitemap item.

  3. If you use RoutablePageMixin in your project, you can overwrite get_sitemap_urls. (This is not required in most cases, but you can take a look for better understanding)

For example

class BlogPage(MetadataPageMixin, RoutablePageMixin, Page):

    def get_sitemap_urls(self, request=None):
        output = []
        posts = self.get_posts()
        for post in posts:
            post_date = post.date
            url = self.get_full_url(request)+ self.reverse_subpage(
                'post_by_date_slug',
                args=(
                    post_date.year,
                    '{0:02}'.format(post_date.month),
                    '{0:02}'.format(post_date.day),
                    post.slug,
                )
            )

            output.append({
                'location': url,
                'lastmod': post.last_published_at
            })

        return output

As you can see, I use this method to generate links for the children page and the new URL has date information.

Now if you check http://127.0.0.1:8000/sitemap.xml, you will see both type of URL exists in sitemap

/blog/welcome/
/blog/2018/01/30/welcome/

You can make PostPage.get_sitemap_urls return empty list then this issue would be resolved.

Robots.txt in Wagtail

In urls.py

urlpatterns = [
    url(r'^robots\.txt$', home.views.RobotsView.as_view(), name='robots'),
    url(r'^sitemap\.xml$', sitemap),
    url(r'', include(wagtail_urls)),
]

I'd like to store robots.txt in templates to make it simple to edit, in some projects, I would create two robots.txt for the production site and staging site.

import os

from django.views.generic import TemplateView

env = os.environ.copy()


class RobotsView(TemplateView):

    content_type = 'text/plain'

    def get_template_names(self):
        return 'robots.txt'

In robots.txt, we have something like this

User-Agent: *
Disallow: /admin/

# Sitemap files
Sitemap: {{ request.site.root_url }}/sitemap.xml

Since the Sitemap should be fully-qualified URL so here we use this way to generate proper robots and if you site does not support multi-site feature, you can write static URL here

Custom 404 500 page

It is always better to have custom 404, 500 page to not let user close the window, but how to test the page?

In urls.py

from django.views import defaults as default_views

if settings.DEBUG:
    from django.conf.urls.static import static
    from django.contrib.staticfiles.urls import staticfiles_urlpatterns
    import debug_toolbar

    # Serve static and media files from development server
    urlpatterns += staticfiles_urlpatterns()
    urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
    urlpatterns = [
        url(r'^__debug__/', include(debug_toolbar.urls)),
        url(r'^404/$', default_views.page_not_found, kwargs={'exception': Exception("Page not Found")}),
        url(r'^500/$', default_views.server_error),
    ] + urlpatterns

Now you can visit http://127.0.0.1:8000/500/ or http://127.0.0.1:8000/404/ to test

3-party tool

There are some great 3-party tools to help us and I'd like to share with you here. I would also appreciate that if you can tell me your tool!

  1. uptimerobot would send notification when our site is down, it has free plan.

  2. deadlinkchecker I'd like to use it to help check broken links for my projects

Conclusion

In this Wagtail tutorial, I talked about how to do better SEO with Wagtail.

But I still want to say, if you want web page rank higher, you should focus more on the page content, which can bring valuable things to your reader.

Table of Contents

Subscribe to get notified about new great blog posts about Web Development

Let’s Work on
Contact Us

Subscribe

Get notified about new great Web Development Tutorial