How to Use Semantic Versioning for Shared Django Apps

When you are building websites with Django, there is no need to track their software versions. You can just have a stable branch in Git repository and update the production environment whenever it makes sense. However, when you create a Django app or package shared among various websites and maybe with multiple people, it is vital to keep track of the package versions.

The Benefits

Versioning allows you to identify a specific state of your package and has these benefits:

• Developers can be aware of which package version works with their websites together flawlessly.
• You can track which versions had which bugs or certain features when communicating with open-source communities or technical support.
• In the documentation, you can clearly see which version of the software it is referring to.
• When fixing bugs of a particular version, developers of the versioned package have a narrower scope of code to check at version control commits.
• Just from the version number, it's clear if the upgrade will only fix the bugs or if it requires more attention to make your software compatible.
• When talking to other developers about a particular package, you can clearly state what you are talking about (yes, developers talk about software versions from time to time 😅).

Semantic Versioning

The most popular versioning scheme is called semantic versioning. The version consists of three numbers separated by dots. Django framework is using it too. For example, the latest stable version at the time of writing is 3.2.9. In semantic versioning, the first number is the major version, the second is the minor version, and the third is a patch version: MAJOR.MINOR.PATCH.

1. MAJOR version introduces backward incompatible features.
2. MINOR version adds functionality that is backward compatible.
3. PATCH version is for backward-compatible bug fixes.

There can also be some versioning outliers for alpha, beta, release candidates rc1, rc2, etc., but we will not cover them in this post.

By convention, the package's version is saved in its myapp/__init__.py file as __version__ variable, for example:

__version__ = "0.2.4"

In addition, it is saved in setup.py, README.md, CHANGELOG.md, and probably some more files.

Changelog

When you develop a Django app or another Python package that you will share with other developers, it is also recommended to have a changelog. Usually, it's a markdown-based document with the list of features, bugs, and deprecations that were worked on between specific versions of your package.

A good changelog starts with the "Unreleased" section and is followed by sections for each version in reverse order. In each of those sections, there are lists of changes grouped into categories:

• Added
• Changed
• Deprecated
• Removed
• Fixed
• Security

This could be the starting template for CHANGELOG.md:

Changelog
=========

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).


[Unreleased]
------------


<!--
### Added
### Changed
### Deprecated
### Removed
### Fixed
### Security
-->

As the new version is released, it will replace the Unreleased section while creating a new Unreleased section above it. For example, this:

[Unreleased]
------------

### Added

- Logging the cookie consent choices in the database because of the legal requirements.

### Fixed

- Disabling the buttons while saving the Cookie Choices so that they are not triggered more than once with slow Internet connections.

would become this:

[Unreleased]
------------

[v0.2.0] - 2021-10-27
------------------

### Added

- Logging the cookie consent choices in the database because of the legal requirements.

### Fixed

- Disabling the buttons while saving the Cookie Choices so that they are not triggered more than once with slow Internet connections.

To keep track of the versions manually would be pretty tedious work, with the likelihood of forgetting one or more files or mismatching the version numbers. Gladly, there is a utility tool to do that for you, which is called bump2version.

Using bump2version

Installation

Install the bump2version utility to your virtual environment the standard way with pip:

(env)$ pip install bump2version

Preparation

In your package's __init__.py file, set the version to 0.0.0:

__version__ = "0.0.0"

Set the version to 0.0.0 in all other files, where the version needs to be mentioned, for example, README.md and setup.py.

Then create setup.cfg with the following content:

[bumpversion]
current_version = 0.0.0
commit = True
tag = True

[bumpversion:file:setup.py]
search = version="{current_version}"
replace = version="{new_version}"

[bumpversion:file:myapp/__init__.py]
search = __version__ = "{current_version}"
replace = __version__ = "{new_version}"

[bumpversion:file:README.md]
search = django_myapp-{current_version}-py2.py3-none-any.whl
replace = django_myapp-{new_version}-py2.py3-none-any.whl

[bumpversion:file:CHANGELOG.md]
search = 
    [Unreleased]
    ------------
replace = 
    [Unreleased]
    ------------
    
    [v{new_version}] - {utcnow:%%Y-%%m-%%d}
    ------------------

[bdist_wheel]
universal = 1

Replace "django_myapp" and "myapp" with your Django app there. If you have more files, mentioning the package version, make sure to have analogous version replacements in the setup.cfg.

Commit and push your changes to the Git repository.

Usage

Every time you want to create a new version, type this in the shell:

$ bump2version patch

or

$ bump2version minor

or

$ bump2version major

followed by the command to build the package:

$ python3 setup.py sdist bdist_wheel

As mentioned before, patch is for the bug fixes, minor is for backward-compatible changes, and major is for backward-incompatible changes.

The bump2version command will use the configuration at setup.cfg and will do these things:

• It will increment the current version number depending on the parameter passed to it.
• It will replace the old version with the new one in the setup.py, myapp/__init__.py, and README.md.
• It will take care of correct versioning in the CHANGELOG.md.
• Then, it will commit the changes and create a tag according to the pattern vMAJOR.MINOR.PATCH there.

Some further details

There are two things to note there.

First, in Markdown, there are two ways of setting the headings:

Changelog
=========

[Unreleased]
------------

is identical to

# Changelog

## [Unreleased]

Usually, I would use the second format for Markdown, but the replacement function in setup.cfg treats lines with the # symbol as comments, and escaping doesn't work either.

For example, this wouldn't work:

[bumpversion:file:CHANGELOG.md]
search = 
    ## [Unreleased]
replace = 
    ## [Unreleased]
    
    ## [v{new_version}] - {utcnow:%%Y-%%m-%%d}

If anybody knows or finds a simple solution with the shorter Markdown variation, let me know.

Second, instead of using setup.cfg, you can use .bumpversion.cfg, but that file is hidden, so I recommend sticking with setup.cfg.

Final words

You can see this type of semantic versioning configuration in the paid Django GDPR Cookie Consent app I recently published on Gumroad.

Happy programming!

---

Cover photo by Jacob Campbell

Deploying a Django Website on Heroku

Once you have a working project, you have to host it somewhere. One of the most popular deployment platforms nowadays is Heroku. Heroku belongs to a Platform as a Service (PaaS) category of cloud computing services. Every Django project you host on Heroku is running inside a smart container in a fully managed runtime environment. Your project can scale horizontally (adding more computing machines) and you pay for what you use starting with a free tier. Moreover, you won't need much of system administrator's skills to do the deployment - once you do the initial setup, the further deployment is as simple as pushing Git repository to a special heroku remote.

However, there are some gotchas to know before choosing Heroku for your Django project:

• One uses PostgreSQL database with your project. MySQL is not an option.
• You cannot store your static and media files on Heroku. One should use Amazon S3 or some other storage for that.
• There is no mailing server associated with Heroku. One can use third-party SendGrid plugin with additional costs, GMail SMTP server with sent email amount limitations, or some other SMTP server.
• The Django project must be version-controlled under Git.
• Heroku works with Python 2.7. Python 3 is not yet supported.

Recently I deployed a small Django project on Heroku. To have a quick reference for the future, I summed up the process here providing instructions how to do that for future reference.

1. Install Heroku Toolbelt

Sign up for a Heroku account. Then install Heroku tools for doing all the deployment work in the shell.

To connect your shell with Heroku, type:

$ heroku login

When asked, enter your Heroku account's email and password.

2. Prepare Pip Requirements

Activate your project's virtual environment and install Python packages required for Heroku:

(myproject_env)$ pip install django-toolbelt

This will install django, psycopg2, gunicorn, dj-database-url, static3, and dj-static to your virtual environment.

Install boto and Django Storages to be able to store static and media files on an S3 bucket:

(myproject_env)$ pip install boto
(myproject_env)$ pip install django-storages

Go to your project's directory and create the pip requirements that Heroku will use in the cloud for your project:

(myproject_env)$ pip freeze -l > requirements.txt

3. Create Heroku-specific Files

You will need two files to tell Heroku what Python version to use and how to start a webserver.

In your project's root directory create a file named runtime.txt with the following content:

python-2.7.11

Then at the same location create a file named Procfile with the following content:

web: gunicorn myproject.wsgi --log-file -

4. Configure the Settings

As mentioned in the "Web Development with Django Cookbook - Second Edition", we keep the developmnent and production settings in separate files both importing the common settings from a base file.

Basically we have myproject/conf/base.py with the settings common for all environments.

Then myproject/conf/dev.py contains the local database and dummy email configuration as follows:

# -*- coding: UTF-8 -*-
from __future__ import unicode_literals
from .base import *

DATABASES = {
    "default": {
        "CONN_MAX_AGE": 0,
        "ENGINE": "django.db.backends.postgresql",
        "HOST": "localhost",
        "NAME": "myproject",
        "PASSWORD": "",
        "PORT": "",
        "USER": "postgres"
    }
}

EMAIL_BACKEND = "django.core.mail.backends.console.EmailBackend"

Lastly for the production settings we need myproject/conf/prod.py with special database configuration, non-debug mode, and unrestrictive allowed hosts as follows:

# -*- coding: UTF-8 -*-
from __future__ import unicode_literals
from .base import *
import dj_database_url

DATABASES = {
    "default": dj_database_url.config()
}

ALLOWED_HOSTS = ["*"]

DEBUG = False

Now let's open myproject/settings.py and add the following content:

# -*- coding: UTF-8 -*-
from __future__ import unicode_literals
from .conf.dev import *

Finally, open the myproject/wsgi.py and change the location of the DJANGO_SETTINGS_MODULE there:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.conf.prod")

5. Set Up Amazon S3 for Static and Media Files

Create an Amazon S3 bucket myproject.media at the AWS Console (web interface for Amazon Web Services). Go to the properties of the bucket, expand "Permissions" section, click on the "add bucket policy" button and enter the following:

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "AllowPublicRead",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::myproject.media/*"
        }
    ]
}

This ensures that files on the S3 bucket will be accessible publicly without any API keys.

Go back to your Django project and add storages to the INSTALLED_APPS in myproject/conf/base.py:

INSTALLED_APPS = [
    # ...
    "storages",
]

Media files and static files will be stored on different paths under S3 bucket. To implement that, we need to create two Python classes under a new file myproject/s3utils.py as follows:

# -*- coding: UTF-8 -*-
from __future__ import unicode_literals
from storages.backends.s3boto import S3BotoStorage

class StaticS3BotoStorage(S3BotoStorage):
    """
    Storage for static files.
    """

    def __init__(self, *args, **kwargs):
        kwargs['location'] = 'static'
        super(StaticS3BotoStorage, self).__init__(*args, **kwargs)


class MediaS3BotoStorage(S3BotoStorage):
    """
    Storage for uploaded media files.
    """

    def __init__(self, *args, **kwargs):
        kwargs['location'] = 'media'
        super(MediaS3BotoStorage, self).__init__(*args, **kwargs)

Finally, let's edit the myproject/conf/base.py and add AWS settings:

AWS_S3_SECURE_URLS = False       # use http instead of https
AWS_QUERYSTRING_AUTH = False                # don't add complex authentication-related query parameters for requests
AWS_S3_ACCESS_KEY_ID = "..."                # Your S3 Access Key
AWS_S3_SECRET_ACCESS_KEY = "..."            # Your S3 Secret
AWS_STORAGE_BUCKET_NAME = "myproject.media"
AWS_S3_HOST = "s3-eu-west-1.amazonaws.com"  # Change to the media center you chose when creating the bucket

STATICFILES_STORAGE = "myproject.s3utils.StaticS3BotoStorage"
DEFAULT_FILE_STORAGE = "myproject.s3utils.MediaS3BotoStorage"

# the next monkey patch is necessary to allow dots in the bucket names
import ssl
if hasattr(ssl, '_create_unverified_context'):
   ssl._create_default_https_context = ssl._create_unverified_context

Collect static files to the S3 bucket:

(myproject_env)$ python manage.py collectstatic --noinput

6. Set Up Gmail to Send Emails

Open myproject/conf/prod.py and add the following settings:

EMAIL_USE_TLS = True
EMAIL_HOST = "smtp.gmail.com"
EMAIL_HOST_USER = "myproject@gmail.com"
EMAIL_HOST_PASSWORD = "mygmailpassword"
EMAIL_PORT = 587

7. Push to Heroku

Commit and push all the changes to your Git origin remote. Personally I prefer using SourceTree to do that, but you can also do that in the command line, PyCharm, or another software.

In your project directory type the following:

(myproject_env)$ heroku create my-unique-project

This will create a Git remote called "heroku", and a new Heroku project "my-unique-project" which can be later accessed at http://my-unique-project.herokuapp.com.

Push the changes to heroku remote:

(myproject_env)$ git push heroku master

8. Transfer Your Local Postgres Database To Heroku

Create local database dump:

(myproject_env)$ PGPASSWORD=mypassword pg_dump -Fc --no-acl --no-owner -h localhost -U myuser mydb > mydb.dump

Upload the database dump temporarily to some server, for example, S3 bucket: http://myproject.media.s3-eu-west-1.amazonaws.com/mydb.dump. Then import that dump into the Heroku database:

(myproject_env)$ heroku pg:backups restore 'http://myproject.media.s3-eu-west-1.amazonaws.com/mydb.dump' DATABASE_URL

Remove the database dump from S3 server.

9. Set Environment Variables

If your Git repository is not private, put your secret values in environment variables rather than in the Git repository directly.

(myproject_env)$ heroku config:set AWS_S3_ACCESS_KEY_ID=ABCDEFG123
$ heroku config:set AWS_S3_SECRET_ACCESS_KEY=aBcDeFg123

To read out the environment variables you can type:

(myproject_env)$ heroku config

To read out the environment variables in the Python code open myproject/conf/base.py and type:

import os
AWS_S3_ACCESS_KEY_ID = os.environ.get("AWS_S3_ACCESS_KEY_ID", "")
AWS_S3_SECRET_ACCESS_KEY = os.environ.get("AWS_S3_SECRET_ACCESS_KEY", "")

10. Set DNS Settings

Open your domain settings and set CNAME to "my-unique-project.herokuapp.com".

At last, you are done! Drop in the comments if I missed some part. For the new updates, see the next section.

*. Update Production

Push the changes to heroku remote:

(myproject_env)$ git push heroku master

If you have changed something in the static files, collect them again:

(myproject_env)$ python manage.py collectstatic --noinput

Collecting static files to S3 bucket takes quite a long time, so I do not recommend to do that automatically every time when you want to deploy to Heroku.

Further Reading

You can read more about Django on Heroku in the following resources:

• "Getting Started on Heroku with Python" at Heroku Dev Center
• "Deploy to Heroku Chapter" at Ultimate Django
• "Migrating Your Django Project to Heroku" at Real Python

---

Cover photo by Frances Gunn

7 Powerful Features of PyCharm Editor I Enjoy Using Daily

Not long ago I noticed PyCharm editor's advertisement at Stack Overflow. As it was targeted to Python and Django developers, I got very curious. I tried the 30-day trial and after years of using jEdit, I finally switched to this new editor. Here is a list of features I am using every day and enjoy very much.

1. Separation by project

Every project can be opened in a different window. PyCharm supports and recognizes virtual environments and python paths. In addition, you can mark some directories as source roots to put them under python path. This is useful when you have some dynamical modifications of python paths. When you open a file of a project, it will be shown in a tab. You can split the window vertically or horizontally to organize tabs in groups. It is especially useful, when you need one file for reference (like models.py) when you are editing another file (like detail template). You can open additional files which don't belong to the project for reference too; their tabs will be marked in different color.

2. Syntax highlighting

This is the core feature of any coder's text editor. PyCharm supports syntax highlighting for all different formats that might be used in Django development: Python, HTML templates, CSS, JavaScript, SQL, and many more. As expected, Django template comments are grayed out. Anywhere in the comments "TODO:" and "FIXME:" lines are marked in different color. At the bottom of the editor you can browse through different files with those TODO lines. PyCharm also marks unused imports and undefined variables with the different color. This editor also encourages PEP 8 standard which makes the coding style uniform and easier to understand and modify for different developers.

3. Browsing

Like many other IDEs, PyCharm has a side-bar file browser in a tree structure. For the clear list view, the *.pyc files are not shown. Edited files are marked in different color. You can drag and drop files to different directories there. You can create new files or new python packages (directories with __init__.py files) quickly. Version control systems like Git, Subversion, Mercurial and CVS are supported (although for this purpose, I am using SourceTree). Also one feature I like is "Scroll from Source" which scrolls and activates the file of the active tab. This is very useful when you need to open another file of the same Django app. By middle-clicking on a variable/class/function, you get to the definition/implementation of it. By ⌘⇧O you can search for specific file. Analogously by ⌘O you can go to specific class implementation. When you have simple views, you can go from the view to the template by clicking on an icon and back (unfortunately, this didn't work in a more complex situation, where I used wrapping view functions).

4. Autocompletion

PyCharm tracks python paths and autocompletes almost everywhere: variables, classes, or functions defined or imported in the current file, imports themselves, Django templates, Django shell, Django management commands. Just don't forget to fire Ctrl + Space for autocompletion.

5. Editing shortcuts

Different shortcuts make the coding faster. Press ⌥→ to move the cursor right by one word. Press ⌥⇧→ to select one word from the right. Press ⌥↑ to select one block of code (multiple times to select containing blocks, e.g. from method declaration to class). Press ⇥ to indent the block of code or ⇧⇥ to unindent it. Press ⌘/ to comment out or uncomment a block of code. Press ⌘R to activate the replace dialog. (These where examples of keyboard shortcuts on Mac OS X. Shortcuts on Windows and Linux will/may differ).

6. Code refactoring.

PyCharm has a powerful feature of renaming classes, functions, variables, or modules. I tried it a couple of times in straightforward situations, but I am still a little bit suspicious about its accuracy for large projects. Although it is worth mentioning that in complex situations it shows you the occurrences of classes, functions, variables, or modules in a list for you to confirm.

7. Django management commands

Running schema migration previously required a lot of typing in the Terminal, i.e. changing the directory to the specific virtual environment, activating the virtual environment, changing directory to Django project, running

python manage.py schemamigration appname modificationdone --auto

Now it is as fast as pressing ⌥R, typing "sc", pressing ⇧⏎, typing "appname modificationdone --auto", pressing ⏎ just in the PyCharm IDE. Running development server or restarting can now be done just with one click. Django shell can also be used from within the IDE. So you don't get distracted by different windows and user interfaces while developing.

I am still quite new to PyCharm and I am discovering new useful features every day. In my opinion, the price of PyCharm is worth every Euro, because my coding, I would say, is now 30 to 40% more effective. Finally, there is some official video introduction to the editor from the developers "JetBrains":

Weather App on Github

As promised, I put the code of the climate_change app online. For that reason, I created an account on github trying to use Git for the first time. This is also the first time I am using a distributed version control system, i.e. you can create branches of the project, develop the forks separately on different computers, and merge them in any order. There is no main centralized repository. The code on github that I uploaded is just one initial copy.

I am still new to Git commands and workflows. You can share your knowledge with me. For example, maybe you know some good cheat sheet or writings about Git. Or you can try to fork the project I committed, change it somehow and then push for merging (not sure if I'm using correct jargon). I will share with you what I find interesting and useful about Git while learning it.