Django has a great system for maintaining your database structure, for keeping your RDMS (typically MySQL, PostgreSQL or SQLite) in line with your Django-defined Models, and within your version control system.

I recently hit a snag with a live client site, which already has 100s of registered users, significant numbers in a small market. Something to handle with extreme caution. Time to dive a bit deeper into Django’s migrations.

Note: migrations were introduced in Django 1.7

Models

Instead of working directly on the database, Django expects you to define the database structure in Python code. This powers many of Django’s features. For instance, the admin system gives  administrators an easy way to maintain the data, and forms can be quickly created.

Once you’ve defined the models and configured the database connection (username, password, etc), Django creates the table structures for you.

Generate the initial database structure by running
>> python manage.py makemigrations

This generates the ‘migration’ path (see below) from an empty database to the initial database

Then apply this to the actual database
>> python manage.py migrate

If you’ve used Django 1.6 or earlier, note that syncdb has been deprecated and replaced by migrate

Migrations

But what happens when you change a table, e.g. when you delete or create a field? And how do you track the changes on your development machine, and make sure the same changes are applied to your test and live systems?

Every time you run
>> python manage.py makemigrations
Django compares the last known Model structure (by running through all past migrations) to the current Model structure and generates a new migrations file, one for each Django app with a changed Model structure. These files are stored in <app>/migrations and are numbered (starting with 0001).

Running
>> python manage.py migrate
applies the most recent migrations. If the database is empty, then Django will start from 0001_initial.py. Otherwise, it checks the migrations table to see, for each app, which migration is the latest one which has been applied to the database, and then runs any later migrations

Migrations and version control

Because the migration files are ordinary Python (text) files, stored in an app’s sub directory, your version control system will manage them for you. When you update the test or live system with the latest code changes, the latest migrations will also be copied across.

Just make sure to run
>> python manage.py migrate
after updating the source code through your version control system

 What went wrong

Normally Django’s migration system works beautifully, making sure the models and database structure stay in sync, between each other and across the development, test and live systems.

Due to some complicated reasons involving tight deadlines and budgets, holidays, and more, something went wrong with the migrations.

The client requested some extra database fields. I added the fields to the model and ran makemigrations.

First nothing happened. Just to be sure I ran
>> python manage.py makemigrations <app name>
and Django told me it had created migration 0001_initial, instead of an incremental change. Checking the database, no changes were made.

Diving into git, I could see a series of past migration files. I could probably roll back the changes to them (i.e. which deleted them), but decided to try something a bit simpler first.

 And how I fixed it

I deleted all existing migrations, from <app name>/migrations and commented out the new database fields in the model.

Running
>> python manage.py makemigrations
created a few 0001_initial migration file, which matches the original model and the current database structure

I removed the comments, to re-instate the new database fields, and re-ran
>> python manage.py makemigrations
to create a migrations file which adds the new fields to the database

Finally
>> python manage.py migrate
applied the changes to the database, adding the new fields

So far so good

Using PHPMyAdmin I can now see the new database fields in the MySQL database

However, I have yet to apply the changes to the live system. I’ll do this once the changes are made and tested. But first I’ll check the revision history on the server and I’ll backup the database, just in case