2014-11-04

Win Free Copies of “Web Development with Django Cookbook”

Readers would be pleased to know that I have teamed up with Packt Publishing to organize a Giveaway of my book Web Development with Django Cookbook.

Three lucky winners stand a chance to win an e-copy of the book. Keep reading to find out how you can be one of the lucky ones.

Book Overview

  • Improve your skills by developing models, forms, views, and templates
  • Create a rich user experience using Ajax and other JavaScript techniques
  • A practical guide to writing and using APIs to import or export data

How to Enter?

All you need to do is head on over to the book page and look through the product description of the book and drop a line via the comments below this post to let us know what interests you the most about this book. It’s that simple.

Prize

Winners will get an e-copy of the Book.

Deadline

The contest will close on the 1st of December, 2014 at 00:00 GMT. Winners will be contacted by email, so be sure to use your real email address when you comment!


...And the winners are...

Out of the 21 participants I finally chose these winners: Mark Phillips, Faria Chowdhury, and Mario Gudelj. The guys from Packt Publishing will contact you by email and will give you the free copies of "Web Development with Django Cookbook". Have a nice reading!

2014-10-27

Contributing Back to the Community - Django Cookbook

In the early beginning of year 2014, the IT book publishing company "Packt Publishing" contacted me with an interesting offer: to share my Django experience in a form of a book. I thought it might be a good challenge for me and also value for the Django community, as I had been working with Django for 7 years or so, and during that time there was quite a lot of knowledge gathered and used practically. So for the next 9 months on late evenings and weekends I was adapting some of the most useful code snippets and describing them in the book. The great staff from the Packt Publishing helped me to structure the content, make everything clear and understandable, and get the grammar correct. Finally, the book was released and it's called "Web Development with Django Cookbook".

Word cloud of the Web Development with Django Cookbook

This book is written for intermediate or advanced Django web developers. When writing the book, my own purpose was not to fully cover every possible web development task, but rather to have enough useful bits of knowledge for those who seek information about web development with Django. The book was written in the format of recipes. There are over 70 recipes giving you instructions how to deal with different challenges of web development. The code mentioned in the book is optimized for Django 1.6, but most of it should also work with older Django versions as well as with Django 1.7.

The cookbook consists of 10 chapters:

  1. Getting started with Django 1.6. This chapter will guide you through the basic configuration which is necessary to start any Django project. It will cover topics like virtual environment, version control, and project settings.
  2. Database Structure. When you create a new app, the first thing to do is to define your models. In this chapter you will learn how to write reusable pieces of code to use in your models. You will learn how to deal with multilingual data in your database. Also you will be told how to manage database schema changes using South migrations.
  3. Forms and Views. To show your data or let one create it, you need views and forms. This chapter will show you some patterns for creating them.
  4. Templates and JavaScript. Information is presented to the user by rendered templates. In modern websites, JavaScript is a must for richer user experience. This chapter shows practical examples of using templates and JavaScript together.
  5. Custom Template Filters and Tags. The default Django template system is quite extensive, but there are more things to add for different cases. This chapter shows you how to create and use own template tags and filters.
  6. Model Administration. Django framework comes with a handy pre-build administration. This chapter shows how to extend the default administration with your own functionality.
  7. Django CMS. Django CMS is the most popular open source content management system made in Django. This chapter deals with best practices using Django CMS and extending it for your needs.
  8. Hierarchical Structures. When you need to create a tree-like structure in Django, django-mptt module comes to hand. This chapter shows you how to use it and how to set administration for hierarchical structures.
  9. Data Import and Export. Very often there are cases when you need to transfer data from and to different formats, retrieve it from and provide it to different sources. This chapter deals with management commands for data import and also APIs for data export.
  10. Bells and Whistles. There is more to Django. This chapter shows additional snippets and tricks useful in web development with Django. Also you will learn about deployment of Django projects.

Get a copy of this book at Packt Publishing and tell me what you think about it in the comments below. Which recipes did you find the most useful? What would you like to read more in this blog or another edition of the cookbook?

Web Development with Django Cookbook

2013-12-18

How to Export Your Data as CSV, XLS, or XLSX

There are times, when you need to export the data from your database to different formats. For example, you want to create some diagrams in Office program for a presentation. In this post I will show you how to create admin actions which export selected items as files for a spreadsheet application (like MS Excel, OpenOffice Calc, LibreOffice Calc, Gnumeric, or Numbers). I will cover the mostly used formats: Comma Separated Values (CSV), Binary Excel (XLS), and Office Open XML (XLSX).

First of all, have a look at the model we will be dealing with. It's a simple model with title, description, and - of course - the id.

# models.py
from django.db import models
class MyModel(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField(blank=True)

In the admininstration options, we'll define three admin actions: export_csv, export_xls, and export_xlsx.

# admin.py
from django.contrib import admin
from models import MyModel

# ... export functions will go here ...

class MyModelAdmin(admin.Admin):
    actions = [export_csv, export_xls, export_xlsx]

admin.site.register(MyModel, MyModelAdmin)

Now let's create functions for each of those actions!

Comma-Separated Values Format

CSV is the most common import and export format for spreadsheets and databases. It's a textual format which one could easily create or parse himself, but there is also a python built-in library csv for handy data manipulation.

def export_csv(modeladmin, request, queryset):
    import csv
    from django.utils.encoding import smart_str
    response = HttpResponse(mimetype='text/csv')
    response['Content-Disposition'] = 'attachment; filename=mymodel.csv'
    writer = csv.writer(response, csv.excel)
    response.write(u'\ufeff'.encode('utf8')) # BOM (optional...Excel needs it to open UTF-8 file properly)
    writer.writerow([
        smart_str(u"ID"),
        smart_str(u"Title"),
        smart_str(u"Description"),
    ])
    for obj in queryset:
        writer.writerow([
            smart_str(obj.pk),
            smart_str(obj.title),
            smart_str(obj.description),
        ])
    return response
export_csv.short_description = u"Export CSV"

As you can see, HttpResponse is a file-like object and we used it to write data to.

Excel Binary File Format

XLS is the main spreadsheet format which holds data in worksheets, charts, and macros. We are going to use xlwt library to create a spreadsheet. There is analogous library xlrd to read XLS files. Note, that this format allows to have only 256 columns.

def export_xls(modeladmin, request, queryset):
    import xlwt
    response = HttpResponse(mimetype='application/ms-excel')
    response['Content-Disposition'] = 'attachment; filename=mymodel.xls'
    wb = xlwt.Workbook(encoding='utf-8')
    ws = wb.add_sheet("MyModel")
    
    row_num = 0
    
    columns = [
        (u"ID", 2000),
        (u"Title", 6000),
        (u"Description", 8000),
    ]

    font_style = xlwt.XFStyle()
    font_style.font.bold = True

    for col_num in xrange(len(columns)):
        ws.write(row_num, col_num, columns[col_num][0], font_style)
        # set column width
        ws.col(col_num).width = columns[col_num][1]

    font_style = xlwt.XFStyle()
    font_style.alignment.wrap = 1
    
    for obj in queryset:
        row_num += 1
        row = [
            obj.pk,
            obj.title,
            obj.description,
        ]
        for col_num in xrange(len(row)):
            ws.write(row_num, col_num, row[col_num], font_style)
            
    wb.save(response)
    return response
    
export_xls.short_description = u"Export XLS"

Here we created one worksheet, filled it with data, marked the first row in bold, and made the lines in the other cells wrapped. Also we set the width for each column. We'll do the same in the next format too.

Office Open XML Format

XLSX (a.k.a. OOXML or OpenXML) is a zipped, XML-based file format developed by Microsoft. It is fully supported by Microsoft Office 2007 and newer versions. OpenOffice 4.0, for example, can only read it. There is a python library openpyxl for reading and writing those files. This format is great when you need more than 256 columns and text formatting options.

def export_xlsx(modeladmin, request, queryset):
    import openpyxl
    from openpyxl.cell import get_column_letter
    response = HttpResponse(mimetype='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')
    response['Content-Disposition'] = 'attachment; filename=mymodel.xlsx'
    wb = openpyxl.Workbook()
    ws = wb.get_active_sheet()
    ws.title = "MyModel"

    row_num = 0

    columns = [
        (u"ID", 15),
        (u"Title", 70),
        (u"Description", 70),
    ]

    for col_num in xrange(len(columns)):
        c = ws.cell(row=row_num + 1, column=col_num + 1)
        c.value = columns[col_num][0]
        c.style.font.bold = True
        # set column width
        ws.column_dimensions[get_column_letter(col_num+1)].width = columns[col_num][1]

    for obj in queryset:
        row_num += 1
        row = [
            obj.pk,
            obj.title,
            obj.description,
        ]
        for col_num in xrange(len(row)):
            c = ws.cell(row=row_num + 1, column=col_num + 1)
            c.value = row[col_num]
            c.style.alignment.wrap_text = True

    wb.save(response)
    return response

export_xlsx.short_description = u"Export XLSX"

Conclusion

So whenever you need to get your Django project data to some spreadsheet application, there are several ways to do that. If you are planning to import the data to some other database, CSV is probably the best, as it is simple, straightforward, and requires no third-party libraries. However, if you need your data with nice formatting and maybe some statistical formulas, you should export XLS or XLSX format. The maximum amount of columns in XLS format is limited to 256, whereas XLSX format allows more columns, but is not fully supported by all spreadsheet applications.

2013-12-07

How to Store Your Media Files in Amazon S3 Bucket

In this article, I will show you how to use Amazon Simple Storage Service (S3) to store your media files in the cloud. S3 is known and widely used for its scalability, reliability, and relatively cheap price. It is free to join and you only pay the hosting and bandwidth costs as you use it. The service is provided by Amazon.com. S3 tends to be attractive for start-up companies looking to minimize costs.

S3 uses a concept of buckets which is like a storage database. Each bucket has its own url. Inside the buckets you have folders and under that you have files. In fact, directories don't actually exist within S3 buckets. The entire file structure is actually just one flat single-level container of files. The illusion of directories is actually created based on having the file names like dirA/dirB/file.

If you want to browse the files in a folder-like structure, you can use Transmit FTP client on Mac OS X. It supports S3 services. Amazon browser-based console also has interface for browsing or uploading files.

OK. Now let's have a look how to set up a Django project which will use S3 for media files.

1. Create a bucket

At first you will need to create a bucket at S3 and make it accessible for all visitors. Login to your Amazon Web Services console. Click on "Services" in the menu, then on the "S3". Click on the button "Create bucket" and enter your bucket name like "mywebsite.media". Choose a region there which is the closest to your target audience, for example, if your website is for Europeans, choose "Ireland". Go to the properties of the bucket, expand "Permissions" section, click on the "add bucket policy" button and enter the following:

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "AllowPublicRead",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::mywebsite.media/*"
        }
    ]
}

2. Install boto and django-storages

Amazon Web Services provide a python library called boto for accessing the API. There is a django app called django-storages which allows to use AWS S3 as the main storage. So your next step is to activate your virtual environment and install latest versions of boto and django-storages.

pip install boto==2.19.0
pip install django-storages==1.1.8

3. Set up django-storages for your project

Add the following to the settings.py:

INSTALLED_APPS = [
    # ...
    'storages'
    # ...
]
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_S3_SECURE_URLS = False       # use http instead of https
AWS_QUERYSTRING_AUTH = False     # don't add complex authentication-related query parameters for requests
AWS_S3_ACCESS_KEY_ID = '...'     # enter your access key id
AWS_S3_SECRET_ACCESS_KEY = '...' # enter your secret access key
AWS_STORAGE_BUCKET_NAME = 'mywebsite.media'

4. Create your models with FileField or ImageField

Let's create a Profile model with avatar field.

def upload_avatar_to(instance, filename):
    import os
    from django.utils.timezone import now
    filename_base, filename_ext = os.path.splitext(filename)
    return 'profiles/%s%s' % (
        now().strftime("%Y%m%d%H%M%S"),
        filename_ext.lower(),
    )

class Profile(models.Model):
    # ...
    avatar = models.ImageField(_("Avatar"), upload_to=upload_avatar_to, blank=True)
    # ...

Whenever you save an instance of the Profile with the new avatar picture, avatar will be uploaded to S3 bucket. To show it in a template, you will need something like <img src="{{ profile.avatar.url }}" alt="" /> where the image source will look like "http://mywebsite.media.s3.amazonaws.com/profiles/20140214203012.jpg".

5. Use the storage to manipulate file versions

If you need to create a thumbnail version of your image, it's probably best to overwrite the save method of the model and trigger the generation of the thumbs there. Let's add some methods to the Profile class:

class Profile(models.Model):
    # ...

    def save(self, *args, **kwargs):
        super(Profile, self).save(*args, **kwargs)
        self.create_avatar_thumb()

    def create_avatar_thumb(self):
        import os
        from PIL import Image
        from django.core.files.storage import default_storage as storage
        if not self.avatar:
            return ""
        file_path = self.avatar.name
        filename_base, filename_ext = os.path.splitext(file_path)
        thumb_file_path = "%s_thumb.jpg" % filename_base
        if storage.exists(thumb_file_path):
            return "exists"
        try:
            # resize the original image and return url path of the thumbnail
            f = storage.open(file_path, 'r')
            image = Image.open(f)
            width, height = image.size

            if width > height:
                delta = width - height
                left = int(delta/2)
                upper = 0
                right = height + left
                lower = height
            else:
                delta = height - width
                left = 0
                upper = int(delta/2)
                right = width
                lower = width + upper

            image = image.crop((left, upper, right, lower))
            image = image.resize((50, 50), Image.ANTIALIAS)

            f_thumb = storage.open(thumb_file_path, "w")
            image.save(f_thumb, "JPEG")
            f_thumb.close()
            return "success"
        except:
            return "error"

    def get_avatar_thumb_url(self):
        import os
        from django.core.files.storage import default_storage as storage
        if not self.avatar:
            return ""
        file_path = self.avatar.name
        filename_base, filename_ext = os.path.splitext(file_path)
        thumb_file_path = "%s_thumb.jpg" % filename_base
        if storage.exists(thumb_file_path):
            return storage.url(thumb_file_path)
        return ""

As you might have guessed, the avatar can be placed in the template using something like

{% if profile.get_avatar_thumb_url %}
    <img src="{{ profile.get_avatar_thumb_url }}" alt="" />
{% endif %}

where the image source will look like "http://mywebsite.media.s3.amazonaws.com/profiles/20140214203012_thumb.jpg".

Conclusion

When you have a basic overview about Amazon Simple Storage Service, it is quite easy to use it in Django projects with existing third-party libraries. For flexibility, if you need to modify uploaded files, storage object should be used instead of the default os methods. That way, you can simply switch to the default file storage for local development.

2013-09-22

How to Create a File Upload Widget

Ajaxified file uploads are becoming de facto standard on the web. People want to see what they chose right after selecting a file instead of during submit of the form. Also if a form has validation errors, nobody wants to select the files again; the selection made should be still available in the form. Therefore, we need a solution that works by Ajax.

Luckily there is a Django app that can help. It's called django-ajax-uploader. Ajax uploader allows to create asynchronous single and multiple file uploads. All uploads are done into the uploads directory. In addition to the default functionality we need a possibility to show the preview of the temporary uploaded file, translate all messages and buttons, delete a file, move a file to a specific directory. I created a demo project which implements all that.

If you have a ModelForm with a FileField or ImageField, you need to not show this field in the form, but instead to add a hidden field for uploaded file path. When you choose a file, it will be uploaded to a temporary uploads directory and its path will be set in the hidden field. Also you need a hidden BooleanField for the state of file deletion. When you submit the form, the file will be moved from the temporary location to the correct location defined by the FileField or ImageField.

In the given example, besides django-ajax-uploader, I am using django-crispy-forms with bootstrap 3.0.0 support. It allows you to define the structure for the forms without creating repetitive HTML markup.

This would be a quick workflow of integrating ajax uploads into your project:

  • Install django-ajax-uploader and django-crispy-forms.
  • Put 'crispy_forms' and 'ajaxuploader' into INSTALLED_APPS.
  • Define CRISPY_TEMPLATE_PACK = 'bootstrap3' in the settings.
  • Define the url rule for ajax uploader
    from ajaxuploader.views import AjaxFileUploader
    uploader = AjaxFileUploader()
    urlpatterns = patterns('',
        # ...
        url(r'^helper/ajax-upload/$', uploader, name="ajax_uploader"),
        # ...
    )
    
  • Create a form for your model which will have the ajax uploader.
  • Create the view which will handle your model and the chosen files and call it in your urls.
  • Create the template with appropriate javascript.

For a working example, please check image uploads project.

2013-09-05

7 Powerful Features of PyCharm Editor I Enjoy Using Daily

Not long ago I noticed PyCharm editor's advertisement at Stack Overflow. As it was targeted to Python and Django developers, I got very curious. I tried the 30-day trial and after years of using jEdit, I finally switched to this new editor. Here is a list of features I am using every day and enjoy very much.

1. Separation by project

Every project can be opened in a different window. PyCharm supports and recognizes virtual environments and python paths. In addition, you can mark some directories as source roots to put them under python path. This is useful when you have some dynamical modifications of python paths. When you open a file of a project, it will be shown in a tab. You can split the window vertically or horizontally to organize tabs in groups. It is especially useful, when you need one file for reference (like models.py) when you are editing another file (like detail template). You can open additional files which don't belong to the project for reference too; their tabs will be marked in different color.

2. Syntax highlighting

This is the core feature of any coder's text editor. PyCharm supports syntax highlighting for all different formats that might be used in Django development: Python, HTML templates, CSS, JavaScript, SQL, and many more. As expected, Django template comments are grayed out. Anywhere in the comments "TODO:" and "FIXME:" lines are marked in different color. At the bottom of the editor you can browse through different files with those TODO lines. PyCharm also marks unused imports and undefined variables with the different color. This editor also encourages PEP 8 standard which makes the coding style uniform and easier to understand and modify for different developers.

3. Browsing

Like many other IDEs, PyCharm has a side-bar file browser in a tree structure. For the clear list view, the *.pyc files are not shown. Edited files are marked in different color. You can drag and drop files to different directories there. You can create new files or new python packages (directories with __init__.py files) quickly. Version control systems like Git, Subversion, Mercurial and CVS are supported (although for this purpose, I am using SourceTree). Also one feature I like is "Scroll from Source" which scrolls and activates the file of the active tab. This is very useful when you need to open another file of the same Django app. By middle-clicking on a variable/class/function, you get to the definition/implementation of it. By ⌘⇧O you can search for specific file. Analogously by ⌘O you can go to specific class implementation. When you have simple views, you can go from the view to the template by clicking on an icon and back (unfortunately, this didn't work in a more complex situation, where I used wrapping view functions).

4. Autocompletion

PyCharm tracks python paths and autocompletes almost everywhere: variables, classes, or functions defined or imported in the current file, imports themselves, Django templates, Django shell, Django management commands. Just don't forget to fire Ctrl + Space for autocompletion.

5. Editing shortcuts

Different shortcuts make the coding faster. Press ⌥→ to move the cursor right by one word. Press ⌥⇧→ to select one word from the right. Press ⌥↑ to select one block of code (multiple times to select containing blocks, e.g. from method declaration to class). Press ⇥ to indent the block of code or ⇧⇥ to unindent it. Press ⌘/ to comment out or uncomment a block of code. Press ⌘R to activate the replace dialog. (These where examples of keyboard shortcuts on Mac OS X. Shortcuts on Windows and Linux will/may differ).

6. Code refactoring.

PyCharm has a powerful feature of renaming classes, functions, variables, or modules. I tried it a couple of times in straightforward situations, but I am still a little bit suspicious about its accuracy for large projects. Although it is worth mentioning that in complex situations it shows you the occurrences of classes, functions, variables, or modules in a list for you to confirm.

7. Django management commands

Running schema migration previously required a lot of typing in the Terminal, i.e. changing the directory to the specific virtual environment, activating the virtual environment, changing directory to Django project, running

python manage.py schemamigration appname modificationdone --auto
Now it is as fast as pressing ⌥R, typing "sc", pressing ⇧⏎, typing "appname modificationdone --auto", pressing ⏎ just in the PyCharm IDE. Running development server or restarting can now be done just with one click. Django shell can also be used from within the IDE. So you don't get distracted by different windows and user interfaces while developing.

I am still quite new to PyCharm and I am discovering new useful features every day. In my opinion, the price of PyCharm is worth every Euro, because my coding, I would say, is now 30 to 40% more effective. Finally, there is some official video introduction to the editor from the developers "JetBrains":

2012-11-07

How to create a virtual environment that also uses global modules

Last weekend I upgraded my Mac to Mountain Lion. I installed Python 2.7 with PIL and MySQLdb and virtualenv using MacPorts.

When I created and activated a virtual environment with

cd myproject
virtualenv .
. bin/activate

the PIL and MySQLdb modules were unrecognizible in that environment. What I needed was just different Django versions in my virtual environments, but MySQLdb and PIL libraries had to be shared.

After a quick search I found an additional parameter for the creation of virtual environment:

virtualenv --system-site-packages .

With this parameter I could use my global modules within the virtual environment.

2012-10-09

How to Set Hijax with Django and jQuery

The main principle of progressive enhancement is that the markup, styling, and Javascript are separated. The site should be functional and browsable without Javascript, but when Javascript is activated, it can add additional functionality to the elements of a page. For example, it can hijack the default behavior of normal links replacing it with Ajax loading of just a specific part of the page. This type of progressive enhancement was called Hijax by Jeremy Keit.

Hijax has the following benefits:

  • Visitors can browse the content on any browser and on any platform.
  • The content is indexable by search engines for each page.
  • Open graph lets you share each page on Facebook with specific title, description and image.

Let me show you an example of Hijax. Recently during my spare time I developed a simple collective storytelling game 1000 Words. Every participant at any point of a story has a chance to choose the continuation previously entered by someone else or to continue by his own words. It's like threaded comments where you see only one thread at a time.

All continuations are browsable without Javascript and each of them has Open Graph set, so Google can index all variations of the stories and any page can be shared on Facebook with custom title, description, and image. When Javascript is enabled (which is the usual case for human visitors), clicking on one of the given continuation choices doesn't refresh the full page, but rather loads the piece of the story and further possible continuations by Ajax and injects into a proper place. This way, I don't need to load all the history with each request and the visitor has more pleasant experience.

Django HttpRequest has a method to check if the call to a page was made by Ajax or by the normal browsing. It's the request.is_ajax() method. Depending on the call, the view of a page can provide different templates: one containing the full HTML with head and body, and another one just with the required snippet of HTML.

There are two ways to organize your templates for Hijax in Django without repeating yourself. One way is to include the Ajax snippet into the template of non-Ajax version.

# views.py
def my_view(request, ...):
    # ...
    template = "myapp/story.html"
    if request.is_ajax():
        template = "myapp/story_ajax.html"
    # ...   
    return render(request, template, context_dict)
{# story.html #}
{% extends "base.html" %}

{% block content %}
    {% include "myapp/story_ajax.html" %}
{% endblock %}
{# story_ajax.html #}
<div class="dynamic-content">
    ...
</div>

The other way to organize your templates for Hijax is extend different bases for normal and Ajax cases. The normal base would have all usual HTML and body, whereas the base Ajax template would just have empty block.

# views.py
def my_view(request, ...):
    # ...
    if request.is_ajax():
        context_dict['base_template'] = "base_ajax.html"
    # ...   
    return render(request, "myapp/story.html", context_dict)
{# story.html #}
{% extends base_template|default:"base.html" %}

{% block content %}
    <div class="dynamic-content">
        ...
    </div>
{% endblock %}
{# base_ajax.html #}
{% block content %}
{% endblock %}

Last but not least, it's time to add unobtrusive Javascript which replaces the default behavior of the links with Ajax calls. With jQuery it should be something like this:

// hijax.html
(function($, undefined) {
    $(document).ready(function() {
        // hijack all links with the css class "dynamic"
        $('a.dynamic').live('click', function() {
            $('#content').load($(this).attr('href'));
            return false;
        });
    });
})(jQuery);

This is not the exact code from the game, but it illustrates the concept of progressive enhancement very well.

2012-10-05

Quick Tip on Setuptools

You are probably used to the default usage of Setuptools, for example the following command installs the latest Django version into your python installation or currently active virtual environment:

easy_install django

Sometimes you might need to install a specific version of a module. This is an example of installing Django 1.3.3:

easy_install django==1.3.3
If you want to upgrade the existing installation to the latest version, you can do that with:
easy_install -U django

2012-09-27

Initial Data Using South Migrations

Usually when starting a new project with commonly used apps, you will need some default data for the database, for example, default categories. Django provides fixtures for that. A fixture is a collection of data that Django knows how to import into a database.

They say:

If you create a fixture named initial_data.[xml/yaml/json], that fixture will be loaded every time you run syncdb.

There is one problem with this approach. It is very likely that you will need to run syncdb in the future when adding new apps (without south migrations). In such cases the modified data will be overwritten with the initial data from fixtures.

But there is a solution for that problem. You need to name your fixtures differently and then import them by south migrations. If you don't use south, you should start doing that, because it is a very handy way to modify database schema and data automatically. :)

So this is how you can prepare south migration with the default fixture:

1. Create a fixture from existing models.

python manage.py dumpdata --format=json --indent=4 myapp.Category > /path/to/myapp/fixtures/myapp_categories.json

2. Create data migration

python manage.py datamigration myapp load_default_data

3. Edit data migration

class Migration(DataMigration):
    
    def forwards(self, orm):
        from django.core.management import call_command
        call_command("loaddata", "myapp_categories.json")
    
    def backwards(self, orm):
        pass

4. Migrate

python manage.py migrate myapp