How do I configure Django to serve my Robots.txt file ?

Within this article we will look at how to configure Django to serve your 'robots.txt' file.

Though there are a number of methods available to achieve this, this example we will only look at the Django method. In that only the the file within your Django project will need to be changed.


Below is an example of the file. The additional lines that are required are displayed via the plus symbol.

Based on this configuration the robots.txt file would need to be placed directly within your templates directory.

  from django.conf.urls.defaults import patterns, include, url
+ from django.views.generic.simple import direct_to_template

  urlpatterns = patterns('',
      url(r'^example_home/$', 'example.views.home'),
+     url(r'^robots\.txt$', direct_to_template, {'template': 'robots.txt', 'mimetype': 'text/plain'}),

Tags: Django

About the Author


R Donato

Rick Donato is the Founder and Chief Editor of He currently works as an SDN/NFV Solutions Architect and has a keen interest in automation and the cloud.

You can find Rick on Twitter @f3lix001