How to make a Foreign Key field non-nullable with a default

To make a foreign key field in Django non-nullable, you need to provide a default value for the existing rows. You might want to also provide a default value for new rows. Adding a default is easy when the value is a string or a date, but not when you have to provide the ID of another object.

Why this isn't easy

1. Your default category must be available on all environments

All environments including your local development environment and your Live site must have the default category available in the database. You don't have to add the default object manually every time you set up a new environment.

We will create a custom migration that will automatically create the category. We also wouldn't want our application to break if an administrator deleted the category, so we will use get_or_create when fetching the category.

2. Your default category can have different IDs in different environments

The simplest method to add a default is to supply the ID to the default attribute like so (don't do this):

category = models.ForeignKey(to=Category, related_name="posts", on_delete=models.SET_DEFAULT, default="1")

This will only work if you're adding the Category model at the same time. If you already have the Category model, then this could result in different categories being used as the default on different machines.

What we will do instead is hard-code the name of the default category. If that category has a different ID on different machine, it will still work.

Our strategy

This tutorial will go through how to set a default Foreign Key using a custom migration, in a way that will ensure consistent results across multiple environments.

These are the steps we are going to to take:

Write a custom migration that will create the default category in the database and handle existing rows with null values
Write a function that will fetch the default category
Update our Post model

The finished code for this tutorial is available in this GitHub repository.

Example: Adding Categories to Posts

I recently wrote a tutorial about how to add a Foreign Key to your Django project. I added a Category model to my blogging application and linked it with my Post model using a Foreign Key field.

Our example is adding categories to posts. In the tutorial, I allowed null values to make it work. Now, I need to populate existing rows with a value and provide a value for new rows.

The models

This is my Category model. It just has a name and a slug for URLs. I've used django-autoslug to automatically populate the slug value (tutorial).

# models.py

class Category(models.Model):
    class Meta:
        verbose_name_plural = "categories"

    name = models.CharField(max_length=100, unique=True)
    slug = AutoSlugField(populate_from="name", unique=True)

    def __str__(self):
        return self.name

The Post model contains our Foreign Key:

category = models.ForeignKey(
        to=Category,
        related_name="posts",
        on_delete=models.SET_NULL,
        blank=True,
        null=True,
    )

Currently, null is set to True, but I want to change that to False.

Attempting to set null=False without a valid default returns an error

If I try to set blank and null to False, I get this error when making migrations:

It is impossible to change a nullable field 'category' on post to non-nullable without providing a default.

We don’t need to drop our database to make this work. We just need to write a custom migration.

Step 1: Write a Custom Migration

We are going to write a custom migration to create a default category. In this case, this will be a category with the name “Uncategorised”.

Why do we not just create a category in the database?

You can do that and it will work on your local environment. However, if you ever need to set up your repository on another machine; you’re collaborating with another developer, or you’re deploying your application, then you will have to remember to create the object.

Creating your default category in a migration will ensure all environments get the same object automatically.

Create the migration file

We will start by creating an empty migration. Run this from your terminal:

$ python manage.py makemigrations blog --empty

# Result

Migrations for 'blog':
  blog/migrations/0007_auto_20220629_2033.py

I like to rename my migrations to make it clearer what it’s for (don’t change the number!): 0007_make_post_category_nonnullable.py

This is the empty migration. Django has automatically added the correct dependency. The dependency is there to ensure migrations are run in the correct order.

Our job is to write an operation for the operations list on Line 12.

Update the operations list with the following:

# 0007_make_post_category_nonnullable.py (Line 12)

operations = [migrations.RunPython(forwards, backwards)]

We will add the functions forwards and backwards to the migration in the next step.

When the migration is run, it will run add_default_category. When the migration is reversed, it will run remove_default_category .

It is important that you include a function for the reverse migration. The ability to reverse migrations is critical to avoid getting into a migrations mess.

Write the forwards migration

As stated in the Django documentation, functions called by migrations.RunPython have two arguments: apps and schema_editor. We will use apps but not the schema_editor, so I have replaced it with an underscore in my snippet.

The function needs to get the category model and create the default category. It also needs to check the existing rows of the Post table for any rows where the category is not set. For posts with no category, the new default category should be used.

This is my function:

# 0007_make_post_category_nonnullable.py

DEFAULT_CATEGORY = "Uncategorised"


def forwards(apps, _):

    category_model = apps.get_model(app_label="blog", model_name="Category")

    category, _ = category_model.objects.get_or_create(name=DEFAULT_CATEGORY)

    post_model = apps.get_model(app_label="blog", model_name="Post")

    posts = post_model.objects.all()

    for post in posts:
        if not post.category:
            post.category = category
            post.save()

It gets the model using apps and then creates a category with the name “Uncategorised”. I have used get_or_create because my category has unique=True set against the category. I don’t want to get an error if I run this migration on an environment that already has that category in the database.

Write the reverse migration

The reverse migration looks for a category with the name “Uncategorised”. If it finds one, it will delete it.

def backwards(apps, _):
    """
    Removes the default category.

    The on_delete rule will be used for any posts that used the default category (models.SET_NULL in this case)
    """
    category_model = apps.get_model(app_label="blog", model_name="Category")

    category = category_model.objects.filter(name=DEFAULT_CATEGORY).first()

    if category:
        category.delete()

I haven’t used objects.get because I don’t want the migration to raise an Error if the category was deleted prior to running the migration.

I also haven't iterated through the posts, removing the default category. This is because I've set on_delete on the Foreign Key field which provides instructions about what to do with the posts when a category is deleted. In this case, reversing the migration will set the category to a null value.

This is the full code of the migration:

# 0007_make_post_category_nonnullable.py
# Generated by Django 4.0.5 on 2022-06-29 20:33

from django.db import migrations

DEFAULT_CATEGORY = "Uncategorised"


def forwards(apps, _):
    """
    Adds a category to use as the default.

    Iterates through existing posts. Assigns the default to posts without a category.
    """
    category_model = apps.get_model(app_label="blog", model_name="Category")

    category, _ = category_model.objects.get_or_create(name=DEFAULT_CATEGORY)

    post_model = apps.get_model(app_label="blog", model_name="Post")

    posts = post_model.objects.all()

    for post in posts:
        if not post.category:
            post.category = category
            post.save()

def backwards(apps, _):
    """
    Removes the default category.

    The on_delete rule will be used for any posts that used the default category (models.SET_NULL in this case)
    """
    category_model = apps.get_model(app_label="blog", model_name="Category")

    category = category_model.objects.filter(name=DEFAULT_CATEGORY).first()

    if category:
        category.delete()

class Migration(migrations.Migration):

    dependencies = [
        ("blog", "0006_category_post_category"),
    ]

    operations = [migrations.RunPython(forwards, backwards)]

Run the migration using:

$ python manage.py migrate

If you need to, you can reverse the migration by specifying the number of the previous migration. E.g.

$ python manage.py migrate blog 0006

Step 2: Write a function that will fetch the default value

Now that our default category exists in the database, we can write a function that will fetch the default category.

Write a function that will get default category

To set a default on a Foreign Key, you need to supply the ID. Even with the migration, you can’t guarantee that the ID of the default category will be 1, so we will need to query the database for the category ID.

# models.py

def get_category_id():
    category, _ = Category.objects.get_or_create(name=Category.DEFAULT_NAME)
    return category.id

Notice how I have used get_or_create here. This is so the application doesn't break if the default category is deleted- it will just create another one.

Step 3: Update the Post model

Finally, we can update our Post model.

I’ve used get_category_id to return the ID of the default category.

When a category is deleted, I want any posts assigned to that category to use the default, so I have changed on_delete to models.SET_DEFAULT.

# models.py

category = models.ForeignKey(
    to=Category,
    related_name="posts",
    default=get_category_id,
    on_delete=models.SET_DEFAULT,
    blank=False,
    null=False,
)

Make and run migrations

Now that a default has been assigned to the Foreign Key field, we can now make migrations.

$ python manage.py makemigrations

# result
Migrations for 'blog':
  blog/migrations/0008_alter_post_category.py
    - Alter field category on post

Conclusion

We have shown how you can add a Foreign Key to an existing database without allowing the value to not be set.

Custom migrations allow you to change the content of databases as well as the structure. When your code is being run on multiple machines, custom migrations will avoid the need for manual changes and ensure consistency across environments.

To make sure the correct object is set as a default, we query the database rather than hard-code the ID of the default, as we cannot guarantee the ID will be the same on all machines.

The full code for this tutorial can be found here.

CTRL+Z - a blog about Django