Write a reversible data migration

Approach

Use RunPython with historical models from apps.get_model (never import current models), process rows in batches, and provide a reverse function so the migration can be rolled back.

Solution

# shop/migrations/0012_backfill_slug.py
from django.db import migrations
from django.utils.text import slugify


def forwards(apps, schema_editor):
    Product = apps.get_model("shop", "Product")
    batch = []
    for product in Product.objects.filter(slug="").only("id", "name").iterator(chunk_size=1000):
        product.slug = slugify(product.name)[:50] or f"product-{product.id}"
        batch.append(product)
        if len(batch) >= 1000:
            Product.objects.bulk_update(batch, ["slug"])
            batch.clear()
    Product.objects.bulk_update(batch, ["slug"])


def backwards(apps, schema_editor):
    apps.get_model("shop", "Product").objects.update(slug="")


class Migration(migrations.Migration):
    dependencies = [("shop", "0011_product_slug")]
    operations = [migrations.RunPython(forwards, backwards)]

Pitfalls

• Importing from shop.models import Product in a migration; it breaks when the model changes later.
• Loading every row into memory with .all() on large tables.
• Combining a long data migration with schema changes on databases without transactional DDL.

Variations

• For very large tables, backfill with a management command in batches after deploying, then add the constraint.
• Use migrations.RunPython.noop as the reverse when there's nothing to undo.

Further reading: Data migrations