--- title: "Smooth Database Changes in Blue-Green Deployments" description: "Let's explore how to make smooth blue-green deployments in Django using advanced migration tools" --- # Smooth Database Changes in Blue-Green Deployments ![Multiple blue-green jellyfishes swimming up in a dark blue background.](https://fly.io/django-beats/smooth-database-changes-in-blue-green-deployments/assets/smooth-database-changes-in-blue-green-deployments-cover.webp) [Annie Ruygt](https://annieruygtillustration.com/) Mariusz Felisiak, a Django and Python contributor and a Django Fellow, explores how to make smooth blue-green deployments using advanced migration tools. Django on Fly.io is pretty sweet! Check it out: [you can be up and running on Fly.io in just minutes.](https://docs.fly.io/django) Blue-green deployment is a technique of releasing changes to a project by gradually transferring users to a new version. It provides an isolation between the current *“blue”* environment and the new *“green”* environment. Blue nodes are sequentially upgraded to the new *“green”* version, so for the entire time of deployment we have both environments working simultaneously. It’s quite a challenge to keep both versions running smoothly and to keep downtime as short as possible, with getting close to the mythical zero-downtime. Changes in an app logic are causing changes in Django models and in the database structure. Therefore, it’s really important to use the right strategy to make the new version backward compatible from the database perspective. Luckily for us, [the migrations framework](https://docs.djangoproject.com/en/stable/topics/migrations/) makes it possible in Django. This article will discuss: - How migrations work? - How to use advanced migration tools to make database transitions in the most efficient way? - How does migration fit into the blue-green strategy? Let’s start with a brief introduction to the migrations framework. ## Migrations framework [The migrations framework](https://docs.djangoproject.com/en/stable/topics/migrations/) is a way of propagating changes in Django models to the database. It makes web development with Django more accessible because, as developers, we no longer need to know, write, and maintain SQL statements with data definitions (known as *DDL* statements *- Data Definition Language* ). Believe me, maintaining SQL statements can be really painful. Luckily for us, we don’t have to do that anymore. Let’s explore how it works. ### Workflow The standard workflow for changing our data definitions has three steps: - First, make a change in a model definition. For example, we can add a new field called `price` to the `Book` model: ``` # bookstore/models.py from django.db import models class Font(models.Model): name = models.CharField(max_length=255) class Author(models.Model): name = models.CharField(max_length=255) class Book(models.Model): title = models.CharField(max_length=1023) description = models.TextField() pages = models.IntegerField() isbn = models.IntegerField() authors = models.ManyToManyField("Author") font = models.ForeignKey( "Font", on_delete=models.CASCADE, null=True ) # ↓ New field ↓ price = models.DecimalField( max_digits=8, decimal_places=2, null=True ) ``` - Second, run the `makemigrations` command to generate a new migration file: ``` python3 manage.py makemigrations ``` ``` Migrations for 'bookstore': bookstore/migrations/0002_book_price.py - Add field price to book ``` Migration files describe what kind of changes are needed in our database structure and what kind of changes has been made in model definitions. That’s what it looks for adding a new field: ``` # bookstore/migrations/0002_book_price.py from django.db import migrations, models class Migration(migrations.Migration):    dependencies = [        ("bookstore", "0001_initial"),    ]    operations = [        migrations.AddField(            model_name="book",            name="price",            field=models.DecimalField( decimal_places=2, max_digits=8, null=True ),        ),    ] ``` 1. `dependencies` - list of migrations it depends on, e.g. the previous migration name. 2. `operations` - list of operations to perform, in our case it is `AddField()`. [The Django documentation](https://docs.djangoproject.com/en/stable/ref/migration-operations/) describes all kind of core operations. 3. `initial` - whether the migration is the first initial migration of an app. - The last step is to propagate changes into our database by running the `migrate` command. ``` python3 manage.py migrate ``` ``` Operations to perform:  Apply all migrations: admin, auth, bookstore, contenttypes, sessions Running migrations:  Applying bookstore.0002_book_price... OK ``` We now have a new column in the `Book` table 🚀 . Fortunately, there are many database operations that are backward compatible and don’t require any special treatment during the blue-green deployments. Django handles them really efficiently in [the standard migrations flow](https://fly.io/django-beats/smooth-database-changes-in-blue-green-deployments/#workflow), it includes - adding a model (table in the database), - adding a nullable field (column in the database), - removing an index, or - removing a constraint. We are not discussing a database locks here, which may be required for some of the database structure changes. Unfortunately, some database transformations are backward incompatible and we need to choose the right strategy and tools to keep them smooth from deployment perspective, it includes - removing a field, - removing a model, or - adding a non-nullable field. Let’s check what advanced tools are built into Django and how we can use them. ## Separate database and project state What we want to do is to separate model definitions from the corresponding database structure during the blue-green deployments. We want to ensure that removed fields (or models) are no longer used on new “ *green* ” nodes, while at the same time the required database columns (or tables) are available for old “blue” nodes. Django’s answer to this problem is a special and extremely powerful migration operation called `SeparateDatabaseAndState()`. It allows to separate database and project state, so to separate changes that should be made in the database structure from the changes that are recognized by Django as being made in the state of models. It allows us to persuade Django that project operations were made as it expects and in the same time no SQL statements are issued on the database. ``` SeparateDatabaseAndState( database_operations=[...], state_operations=[...], ) ``` It accepts two lists of operations: - `database_operations` - list of operations to apply to the database, - `state_operations` - list of operations to apply to the project state. Let’s find out how we can use it in practice for removing a field and a model. ### Full example We need an existing or new Django project. Here are some great resources for [getting started with Django](https://www.djangoproject.com/start/) or [deploying your Django app to Fly.io](https://docs.fly.io/django/getting-started#initial-set-up). With a project ready, let’s get started! Suppose that we have a bookstore site, that provides various information about available books. In our example, we’ll use the same models (`Font`, `Author`, and `Book`) as defined in the [Workflow paragraph](https://fly.io/django-beats/smooth-database-changes-in-blue-green-deployments/#workflow). What if, at some point, we realize that collecting information about fonts used in books are difficult and unnecessary. It’s not important for our clients, so we would like to simplify our models definition by removing the `Font` model and the `Book.font` field. The following steps will show you how we can perform this potentially backward incompatible transition in a blue-green deployment by using the `SeparateDatabaseAndState()` operation. - First step is to remove all logic related to the `Font` from our project. - Secondly, remove the `Font` model and the `Book.font` field: ``` --- a/bookstore/models.py +++ b/bookstore/models.py @@ -1,10 +1,6 @@ from django.db import models -class Font(models.Model): - name = models.CharField(max_length=255) - - class Author(models.Model): name = models.CharField(max_length=255) @@ -15,9 +11,6 @@ class Book(models.Model): pages = models.IntegerField() isbn = models.IntegerField() authors = models.ManyToManyField("Author") - font = models.ForeignKey( - "Font", on_delete=models.CASCADE, null=True - ) price = models.DecimalField( max_digits=8, decimal_places=2, null=True ) ``` - Next, run the `makemigrations` command to generate a new migration file: ``` python3 manage.py makemigrations ``` ``` Migrations for 'bookstore': bookstore/migrations/0003_remove_book_font_delete_font.py - Remove field font from book - Delete model Font ``` `makemigrations` generates a new migration with two operations: `RemoveField()` and `DeleteModel()`: ``` # bookstore/migrations/0003_remove_book_font_delete_font.py from django.db import migrations class Migration(migrations.Migration): dependencies = [ ("bookstore", "0002_book_price"), ] operations = [ migrations.RemoveField( model_name="book", name="font", ), migrations.DeleteModel(name="Font"), ] ``` Both operations are backward incompatible and deploying this migration as it is would break the old *“blue”* environments, because the underlying table and column would be deleted. Using the `sqlmigrate` command we can check SQL statements that will be issued (output for PostgreSQL): ``` python3 manage.py sqlmigrate bookstore 0003_remove_book_font_delete_font ``` ``` BEGIN; -- -- Remove field font from book -- ALTER TABLE "bookstore_book" DROP COLUMN "font_id" CASCADE; -- -- Delete model Font -- DROP TABLE "bookstore_font" CASCADE; COMMIT; ``` These operations describe changes that we made in models, so we can wrap them with the `SeparateDatabaseAndState()` operation by moving to the `state_operations`, at the same time leaving `database_operations` empty to avoid changes in the database structure, like so: ``` # bookstore/migrations/0003_remove_book_font_delete_font.py from django.db import migrations class Migration(migrations.Migration): dependencies = [ ("bookstore", "0002_book_price"), ] operations = [ migrations.SeparateDatabaseAndState( # Operations in the project state. state_operations=[ migrations.RemoveField( model_name="book", name="font", ), migrations.DeleteModel(name="Font"), ], # No changes in the database. database_operations=[], ), ] ``` Using the `sqlmigrate` command again confirms that no SQL statements will be issued: ``` python3 manage.py sqlmigrate bookstore 0003_remove_book_font_delete_font ``` ``` BEGIN; -- -- Custom state/database change combination -- -- (no-op) COMMIT; ``` We can now safely deploy our new “green” version to all instances in a fully backward compatible manner from a database perspective, as nothing has changed in the database structure 🎉 . Check out the [`bluegreen`](https://docs.fly.io/launch/deploy#deployment-strategy) deployment strategy on Fly.io! 💙 → 💚 Once the new version is deployed to all instances and when we are sure that the underlying table and column are unused, we can actually remove them from the database. To do this, we need a new blank migration file that will be structured like other migration files. To generate such a file, use the `--empty` flag to the `makemigrations` command. In order to follow good practice, and to avoid migration names based on timestamp, I’d recommend to also pass the `--name` flag, e.g. `"remove_book_font_delete_font_from_db"` which makes it clear what this migrations will do: ``` python3 manage.py makemigrations bookstore \ --empty \ --name remove_book_font_delete_font_from_db ``` ``` Migrations for 'bookstore': bookstore/migrations/0004_remove_book_font_delete_font_from_db.py ``` The `dependencies` are already filled and we have an empty list of operations ready to be use: ``` # bookstore/migrations/0004_remove_book_font_delete_font_from_db.py from django.db import migrations class Migration(migrations.Migration): dependencies = [ ("bookstore", "0003_remove_book_font_delete_font"), ] operations = [] ``` We want to perform the same operations, but this time in the database, not in the project state. Therefore, we will use `SeparateDatabaseAndState()` again but with SQL statements that describe `RemoveField()` and `DeleteModel()` operations in the `database_operations` and an empty list of `state_operations`. The modified migration file `0004_remove_book_font_delete_font_from_db.py` should look like this: ``` # bookstore/migrations/0004_remove_book_font_delete_font_from_db.py from django.db import migrations class Migration(migrations.Migration): dependencies = [ ("bookstore", "0003_remove_book_font_delete_font"), ] operations = [ migrations.SeparateDatabaseAndState( # No changes in the project. state_operations=[], # Operations to be performed on the database. database_operations=[ # SQL statement for RemoveField(). migrations.RunSQL( sql=( 'ALTER TABLE "bookstore_book" ' 'DROP COLUMN "font_id" CASCADE;' ), ), # SQL statement for DeleteModel(). migrations.RunSQL( sql='DROP TABLE "bookstore_font" CASCADE;', ), ], ), ] ``` We can now safely deploy our latest “green” version to all instances in a fully backward compatible manner as the deleted table and column are no longer used by any node after the previous deployment 💙 → 💚. Voilà ✨ ## Closing Thoughts Separating database and project states gives us highly flexible pattern for propagating database changes. It is not only handy for blue-green deployments, but it also allows for describing potentially not feasible transitions in the most efficient way. Frequently without any data migration, without any intermediate steps, atomic, and even reversible. Some use cases are: - [changing a `ManyToManyField` to use a `through` model](https://docs.djangoproject.com/en/stable/howto/writing-migrations/#changing-a-manytomanyfield-to-use-a-through-model), - changing a custom `through` model to auto-generated intermediate table, - moving models between apps, etc. Try it and share! Remember to check out the [new and shiny `bluegreen` deployment strategy](https://docs.fly.io/launch/deploy#deployment-strategy) on Fly.io!