Django Unit Testing taking a very long time to create test database
The final solution that fixes my problem is to force Django to disable migration during testing, which can be done from the settings like this
TESTING = 'test' in sys.argv[1:]
if TESTING:
print('=========================')
print('In TEST Mode - Disableling Migrations')
print('=========================')
class DisableMigrations(object):
def __contains__(self, item):
return True
def __getitem__(self, item):
return None
MIGRATION_MODULES = DisableMigrations()
or use https://pypi.python.org/pypi/django-test-without-migrations
My whole test now takes about 1 minute and a small app takes 5 seconds.
In my case, migrations are not needed for testing as I update tests as I migrate, and don't use migrations to add data. This won't work for everybody
Summary
Use pytest
!
Operations
pip install pytest-django
pytest --nomigrations
instead of./manage.py test
Result
./manage.py test
costs 2 min 11.86 secpytest --nomigrations
costs 2.18 sec
Hints
You can create a file called
pytest.ini
in your project root directory, and specify default command line options and/or Django settings there.# content of pytest.ini [pytest] addopts = --nomigrations DJANGO_SETTINGS_MODULE = yourproject.settings
Now you can simply run tests with
pytest
and save you a bit of typing.You can speed up the subsequent tests even further by adding
--reuse-db
to the default command line options.[pytest] addopts = --nomigrations --reuse-db
However, as soon as your database model is changed, you must run
pytest --create-db
once to force re-creation of the test database.If you need to enable gevent monkey patching during testing, you can create a file called
pytest
in your project root directory with the following content, cast the execution bit to it (chmod +x pytest
) and run./pytest
for testing instead ofpytest
:#!/usr/bin/env python # -*- coding: utf-8 -*- # content of pytest from gevent import monkey monkey.patch_all() import os os.environ.setdefault("DJANGO_SETTINGS_MODULE", "yourproject.settings") from django.db import connection connection.allow_thread_sharing = True import re import sys from pytest import main if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) sys.exit(main())
You can create a
test_gevent.py
file for testing whether gevent monkey patching is successful:# -*- coding: utf-8 -*- # content of test_gevent.py import time from django.test import TestCase from django.db import connection import gevent def f(n): cur = connection.cursor() cur.execute("SELECT SLEEP(%s)", (n,)) cur.execute("SELECT %s", (n,)) cur.fetchall() connection.close() class GeventTestCase(TestCase): longMessage = True def test_gevent_spawn(self): timer = time.time() d1, d2, d3 = 1, 2, 3 t1 = gevent.spawn(f, d1) t2 = gevent.spawn(f, d2) t3 = gevent.spawn(f, d3) gevent.joinall([t1, t2, t3]) cost = time.time() - timer self.assertAlmostEqual(cost, max(d1, d2, d3), delta=1.0, msg='gevent spawn not working as expected')
References
- pytest-django documentation
- pytest documentation
use ./manage.py test --keepdb when there are no changes in the migration files