In this post, we will go through how to add a non-nullable field and handle the existing rows. Instead of providing a default value or forcing the column to accept null values, we will automatically add values to existing rows, where each value is specific to that row.
If you’re still building confidence handling migrations, I recommend reading my beginner’s guide to migrations, where you will learn how to avoid database errors.
I wanted to add a slug field to an existing model.
It’s important that I don’t allow blank values, as the slug will be used for URLs.
But when I tried running
python manage.py makemigrations , I got this error:
It is impossible to add a non-nullable field 'slug' to movie without specifying a default. This is because the database needs something to populate existing rows. Please select a fix: 1) Provide a one-off default now (will be set on all existing rows with a null value for this column) 2) Quit and manually define a default value in models.py.
These are my options:
1. Assign an arbitrary default
I’d rather not as I’d be deliberately assigning invalid values to my existing data. I could run a script after to assign the correct slug to each existing movie, but I’d rather not have the extra step. It creates confusion if you’re trying to set the project up on another machine.
2. Allow blank values
I’ve done this a few times in personal projects as a quick fix but it’s not ideal. If I see
blank=True in a model, then that tells me that blank values are acceptable. Slugs are used to construct URLs, so blank values are definitely not acceptable, in this case.
3. Drop the database
This is something to avoid as much as possible. Deleting the
db.sqlite3 file, rerunning your migrations, and recreate all your data can seem tempting when you’re in a mess, but it takes so much effort to get your database contents to the same state it was before. Once your product has been deployed, dropping the database will never be an option, so I say learn to avoid this sooner rather than later.
4. Customise your migration
This is where you add code to your migration to assign the correct value for each existing row. If you’re working with another developer or on multiple machines, this approach will get the results you need in one step, without running additional scripts.
How do we do this?
We are going to temporarily assign an arbitrary default value.
Then we’re going to add an operation to our migration that will go through each of our existing rows and update the new slug field with the correct value.
Let’s get started.
How to customise a migration
1. Create your migration
Start by running
python manage.py makemigration
When you get the following prompt, press 1:
Please select a fix: 1) Provide a one-off default now (will be set on all existing rows with a null value for this column) 2) Quit and manually define a default value in models.py.
Provide any value. As the column I’m adding is a string, I’ve chosen
“abc”. This will give Django what it needs to create the migration. We’re going to customise the migration before we run it. Our Python code will overwrite any column with the
2. Customise the Migration
For now, I’ve set a default of
“abc” that I don’t want applied to the database. I’ve done this to appease Django to create the migration, but we’ll make sure
“abc” doesn’t get applied.
Django migrations contain a list called
operations. The migration is created with a single operation:
migrations.AlterField. What we’re going to do is add a second operation that will go through each of our movies, calculate the correct slug from the movie’s title and update the movie. Operations are executed in order, so we’ll update each movie’s slug after the field has been created.
Write the custom code
We need to write two functions: one that will update the movie with our desired slug and one that will restore the slug to the default value.
We do this because migrations should always be reversible. When we run the migration,
forwards will be called. Should we ever need to reverse the migration,
backwards is called.
Add the operation
Add the following to
See the gist below for the full code of the migration:
3. Run the Migration
The migration is now ready to run. Run
python manage.py migrate in your terminal to run the migration.
Go into admin to check that the correct value for the new field has been applied to existing rows.
We have shown that you don’t have to drop your database or compromise on defaults when you want to add a non-nullable field to one of your models.
By understanding how Django migrations execute operations in order, we can use
migrations.RunPython(forwards, backwards) to add custom code. This is useful when we want to add a field, and give each existing row a custom value.