How do I configure Django to serve my Robots.txt file ?

Within this article we will look at how to configure Django to serve your 'robots.txt' file.

Though there are a number of methods available to achieve this, this example we will only look at the Django method. In that only the the file within your Django project will need to be changed.


Below is an example of the file. The additional lines that are required are displayed via the plus symbol.

Based on this configuration the robots.txt file would need to be placed directly within your templates directory.

  from django.conf.urls.defaults import patterns, include, url
+ from django.views.generic.simple import direct_to_template

  urlpatterns = patterns('',
      url(r'^example_home/$', 'example.views.home'),
+     url(r'^robots\.txt$', direct_to_template, {'template': 'robots.txt', 'mimetype': 'text/plain'}),

Tags: Django