Django Unit Testing taking a very long time to create test database

The final solution that fixes my problem is to force Django to disable migration during testing, which can be done from the settings like this

TESTING = 'test' in sys.argv[1:]
if TESTING:
    print('=========================')
    print('In TEST Mode - Disableling Migrations')
    print('=========================')

    class DisableMigrations(object):

        def __contains__(self, item):
            return True

        def __getitem__(self, item):
            return None

    MIGRATION_MODULES = DisableMigrations()

or use https://pypi.python.org/pypi/django-test-without-migrations

My whole test now takes about 1 minute and a small app takes 5 seconds.

In my case, migrations are not needed for testing as I update tests as I migrate, and don't use migrations to add data. This won't work for everybody


Summary

Use pytest !

Operations

  1. pip install pytest-django
  2. pytest --nomigrations instead of ./manage.py test

Result

  • ./manage.py test costs 2 min 11.86 sec
  • pytest --nomigrations costs 2.18 sec

Hints

  • You can create a file called pytest.ini in your project root directory, and specify default command line options and/or Django settings there.

    # content of pytest.ini
    [pytest]
    addopts = --nomigrations
    DJANGO_SETTINGS_MODULE = yourproject.settings
    

    Now you can simply run tests with pytest and save you a bit of typing.

  • You can speed up the subsequent tests even further by adding --reuse-db to the default command line options.

    [pytest]
    addopts = --nomigrations --reuse-db
    

    However, as soon as your database model is changed, you must run pytest --create-db once to force re-creation of the test database.

  • If you need to enable gevent monkey patching during testing, you can create a file called pytest in your project root directory with the following content, cast the execution bit to it (chmod +x pytest) and run ./pytest for testing instead of pytest:

    #!/usr/bin/env python
    # -*- coding: utf-8 -*-
    # content of pytest
    from gevent import monkey
    
    monkey.patch_all()
    
    import os
    
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "yourproject.settings")
    
    from django.db import connection
    
    connection.allow_thread_sharing = True
    
    import re
    import sys
    
    from pytest import main
    
    if __name__ == '__main__':
        sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
        sys.exit(main())
    

    You can create a test_gevent.py file for testing whether gevent monkey patching is successful:

    # -*- coding: utf-8 -*-
    # content of test_gevent.py
    import time
    from django.test import TestCase
    from django.db import connection
    import gevent
    
    
    def f(n):
        cur = connection.cursor()
        cur.execute("SELECT SLEEP(%s)", (n,))
        cur.execute("SELECT %s", (n,))
        cur.fetchall()
        connection.close()
    
    
    class GeventTestCase(TestCase):
        longMessage = True
    
        def test_gevent_spawn(self):
            timer = time.time()
            d1, d2, d3 = 1, 2, 3
            t1 = gevent.spawn(f, d1)
            t2 = gevent.spawn(f, d2)
            t3 = gevent.spawn(f, d3)
            gevent.joinall([t1, t2, t3])
            cost = time.time() - timer
            self.assertAlmostEqual(cost, max(d1, d2, d3), delta=1.0,
                                   msg='gevent spawn not working as expected')
    

References

  • pytest-django documentation
  • pytest documentation

use ./manage.py test --keepdb when there are no changes in the migration files