Run / Debug a Django application's UnitTests from the mouse right click context menu in PyCharm Community Edition?
1. Background info
- I am only working with Django for ~3 months
- Regarding PyCharm, I worked with it for some years, but only as an IDE (like PyCharm for dummies), so I didn't get into its advanced stuff
Considering the above, some (or all) parts of the solution might seem cumbersome / stupid for some advanced users, so please bear with me. I will incorporate any possible comment that adds value into the solution.
Back to the question: I did my tests / research on a project that consists of Django Tutorial ([DjangoProject]: Writing your first Django app) + some parts from Django Rest Framework Tutorial ([DRF]: Quickstart). As an example, I'm going to attempt running polls/tests.py: QuestionViewTests.test_index_view_with_no_questions()
As a note, setting DJANGO_SETTINGS_MODULE as the exception instructs, triggers another one, and so on ...
2. Creating a Python configuration
Although this is not an answer to the question (it's only remotely related), I'm posting it anyway (I'm sure that many people already did it):
- Click on the menu Run -> Edit Configurations...
- On the Run/Debug Configurations dialog:
- Add a new configuration having the type: Python
- Set the Working directory to the root path of your project (for me it is "E:\Work\Dev\Django\Tutorials\proj0\src"). By default, this will also add the path in the Python's modules search paths
- Set the Script to your Django project startup script (manage.py)
- Set the Script parameters to the test parameters (
test QuestionViewTests.test_index_view_with_no_questions) - Give your configuration a name (optional) and click OK. Now, you will be able to run this test
Of course, having to do this for every test case (and their methods) is not the way to go (it is truly annoying), so this approach is not scalable.
3. Adjusting PyCharm to do what we want
Just to be noted that I don't see this as a true solution, it's more like a (lame) workaround (gainarie), and it's also intrusive.
Let's start by looking what happens when we RClick on a test (I'm going to use this term in general - it might mean Test Case or method or whole test file, unless specified otherwise). For me, it is running the following command:
"E:\Work\Dev\VEnvs\py2713x64-django\Scripts\python.exe" "C:\Install\PyCharm Community Edition\2016.3.2\helpers\pycharm\utrunner.py" E:\Work\Dev\Django\Tutorials\proj0\src\polls\tests.py::QuestionViewTests::test_index_view_with_no_questions true
As you can see, it's launching "C:\Install\PyCharm Community Edition\2016.3.2\helpers\pycharm\utrunner.py" (I'm going to refer to it as utrunner) with a bunch of arguments (the 1st matters to us, since it's the test specification). utrunner uses a test run framework which does not care about Django (actually there is some Django handling code, but that's not helping us).
A few words on PyCharm`s Run/Debug configurations:
- When RClick-ing on a test, PyCharm automatically creates a new Run configuration (that you will be able to save), just like you would from the Run/Debug Configurations dialog. An important thing to note is the configuration type which is Python tests/Unittests (which automatically fires utrunner)
- When creating a Run configuration in general, PyCharm "copies" the settings from that configuration type Defaults (can be viewed in the Run/Debug Configurations dialog), into the new configuration, and fills the others with specific data. One important thing about Default configurations is that they are project based: they reside in the .idea folder (workspace.xml) of the project, so modifying them would not impact other projects (as I first feared)
With the above in mind, let's proceed:
First thing you need to do is: from the Run/Debug Configurations dialog (menu: Run -> Edit Configurations...), edit the Defaults/Python tests/Unittests settings:
- Set the Working directory just like in the previous approach
- In the Environment variables add a new one named DJANGO_TEST_MODE_GAINARIE and set it to any string (other than empty/null)
Second thing and the trickier one (also involving intrusion): patching utrunner.
utrunner.patch:
--- utrunner.py.orig 2016-12-28 19:06:22.000000000 +0200
+++ utrunner.py 2017-03-23 15:20:13.643084400 +0200
@@ -113,7 +113,74 @@
except:
pass
-if __name__ == "__main__":
+
+def fileToMod(filePath, basePath):
+ if os.path.exists(filePath) and filePath.startswith(basePath):
+ modList = filePath[len(basePath):].split(os.path.sep)
+ mods = ".".join([os.path.splitext(item)[0] for item in modList if item])
+ return mods
+ else:
+ return None
+
+
+def utrunnerArgToDjangoTest(arg, basePath):
+ if arg.strip() and not arg.startswith("--"):
+ testData = arg.split("::")
+ mods = fileToMod(testData[0], basePath)
+ if mods:
+ testData[0] = mods
+ return ".".join(testData)
+ else:
+ return None
+ else:
+ return None
+
+
+def flushBuffers():
+ sys.stdout.write(os.linesep)
+ sys.stdout.flush()
+ sys.stderr.write(os.linesep)
+ sys.stderr.flush()
+
+
+def runModAsMain(argv, codeGlobals):
+ with open(argv[0]) as f:
+ codeStr = f.read()
+ sys.argv = argv
+ code = compile(codeStr, os.path.basename(argv[0]), "exec")
+ codeGlobals.update({
+ "__name__": "__main__",
+ "__file__": argv[0]
+ })
+ exec(code, codeGlobals)
+
+
+def djangoMain():
+ djangoTests = list()
+ basePath = os.getcwd()
+ for arg in sys.argv[1: -1]:
+ djangoTest = utrunnerArgToDjangoTest(arg, basePath)
+ if djangoTest:
+ djangoTests.append(djangoTest)
+ if not djangoTests:
+ debug("/ [DJANGO MODE] Invalid arguments: " + sys.argv[1: -1])
+ startupTestArgs = [item for item in os.getenv("DJANGO_STARTUP_TEST_ARGS", "").split(" ") if item]
+ startupFullName = os.path.join(basePath, os.getenv("DJANGO_STARTUP_NAME", "manage.py"))
+ if not os.path.isfile(startupFullName):
+ debug("/ [DJANGO MODE] Invalid startup file: " + startupFullName)
+ return
+ djangoStartupArgs = [startupFullName, "test"]
+ djangoStartupArgs.extend(startupTestArgs)
+ djangoStartupArgs.extend(djangoTests)
+ additionalGlobalsStr = os.getenv("DJANGO_STARTUP_ADDITIONAL_GLOBALS", "{}")
+ import ast
+ additionalGlobals = ast.literal_eval(additionalGlobalsStr)
+ flushBuffers()
+ runModAsMain(djangoStartupArgs, additionalGlobals)
+ flushBuffers()
+
+
+def main():
arg = sys.argv[-1]
if arg == "true":
import unittest
@@ -186,3 +253,10 @@
debug("/ Loaded " + str(all.countTestCases()) + " tests")
TeamcityTestRunner().run(all, **options)
+
+
+if __name__ == "__main__":
+ if os.getenv("DJANGO_TEST_MODE_GAINARIE"):
+ djangoMain()
+ else:
+ main()
The above is a diff ([man7]: DIFF(1)) (or a patch - the names can be used conjunctively - I preffer (and will use) patch): it shows the differences between utrunner.py.orig (the original file - that I saved before starting modifying, you don't need to do it) and utrunner.py (the current version containing the changes). The command that I used is diff --binary -uN utrunner.py.orig utrunner.py (obviously, in utrunner's folder). As a personal remark, patch is the preferred form of altering 3rd party source code (to keep changes under control, and separate).
What the code in the patch does (it's probably harder to follow than plain Python code):
- Everything under the main block (
if __name__ == "__main__":or the current behavior) has been moved into a function called main (to keep it separate and avoid altering it by mistake) - The main block was modified, so that if the env var DJANGO_TEST_MODE_GAINARIE is defined (and not empty), it will follow the new implementation (djangoMain function), otherwise it will act normally. The new implementation:
- fileToMod subtracts basePath from filePath and converts the difference into Python package style. Ex:
fileToMod("E:\Work\Dev\Django\Tutorials\proj0\src\polls\tests.py", "E:\Work\Dev\Django\Tutorials\proj0\src"), will returnpolls.tests - utrunnerArgToDjangoTest: uses the previous function and then adds the class name (QuestionViewTests) and (optionally) the method name (test_index_view_with_no_questions), so at the end it converts the test specification from utrunner format (
E:\Work\Dev\Django\Tutorials\proj0\src\polls\tests.py::QuestionViewTests::test_index_view_with_no_questions) to manage.py format (polls.tests.QuestionViewTests.test_index_view_with_no_questions) - flushBuffers: writes an eoln char and flushes the stdout and stderr buffers (this is needed because I noticed that sometimes the outputs from PyCharm and Django are interleaved, and the final result is messed up)
- runModAsMain: typically, all the relevant manage.py code is under
if __name__ == "__main__":. This function "tricks" Python making it believe that manage.py was run as its 1st argument
- fileToMod subtracts basePath from filePath and converts the difference into Python package style. Ex:
Patching utrunner:
- I did these modifications on my own (I didn't search for versions having Django integration and inspire from there)
- utrunner is part of PyCharm. It's obvious why JetBrains guys didn't include any Django integration in the Community Edition: to make people buy the Professional Edition. This kinda steps on their toes. I'm not aware of the legal implications of modifying utrunner, but anyway if you patch it, you're doing it on your own responsibility and risk
- Coding style: it sucks (at least from naming / indenting PoV), but it's consistent with the rest of the file (the only case when coding style should be allowed to suck). [Python]: PEP 8 -- Style Guide for Python Code contains the coding style guidelines for Python
- The patch is applied on the original file (utrunner.py), with the following properties (still valid for v2019.2.3 (last checked: 20190930)):
- size: 5865
- sha256sum: db98d1043125ce2af9a9c49a1f933969678470bd863f791c2460fe090c2948a0
- Applying the patch:
- utrunner is located in "${PYCHARM_INSTALL_DIR}/helpers/pycharm"
- Typically, ${PYCHARM_INSTALL_DIR} points to:
- Nix: /usr/lib/pycharm-community
- Win: "C:\Program Files (x86)\JetBrains\PyCharm 2016.3" (adapt to your version number)
- Save the patch content (in a file called e.g. utrunner.patch, let's assume it's under /tmp)
- Nix - things are easy, just (cd to utrunner's folder and) run
patch -i /tmp/utrunner.patch. [man7]: PATCH(1) is an utility that is installed by default (part of patch dpkg in Ubtu). Note that since utrunner.py is owned by root, for this step you would need sudo - Win - similar steps to be followed, but things are trickier since there's no native patch utility. However, there are workarounds:
- Use Cygwin. As in Nix (Lnx) case, patch utility is available, but it doesn't get installed by default. The patch pkg must be explicitly installed from Cygwin setup. I tried this and it works
- There are alternatives (I didn't try them):
- [SourceForge.GnuWin32]: Patch for Windows
- In theory, [RedBean]: svn patch(any client) should be able to apply a patch, but I'm not sure if the file should be part of a working copy.
- Applying the patch manually (a less desired option :) )
- As in Nix's case, patching the file would (most likely) have to be done by one of the Administrators. Also, watch out for file paths, make sure to (dbl)quote them if they contain spaces
- Reverting the patch:
- Backups are not harmful (except from the free disk space's PoV, or when they start to pile up, managing them becomes a pain). There's no need for them in our case. In order to revert the changes, just run the command on the modified file:
patch -Ri /tmp/utrunner.patch, and it will switch it back to its original content (it will also create an utrunner.py.orig file with the modified content; it will actually switch the .py and .py.orig files).
Nevertheless always back 3rd-party files up before modifying them (especially if they're being tracked by some tools / installers), so that if something goes wrong while modifying them, there's always a way to restore the original state
- Backups are not harmful (except from the free disk space's PoV, or when they start to pile up, managing them becomes a pain). There's no need for them in our case. In order to revert the changes, just run the command on the modified file:
- Although not the case here, but if the changes are in another form, like the file with the patch applied (e.g. on GitHub), you can obviously get the entire file (if there are many files, tracking all of them down could become a pain) and overwrite yours. But again, back it (them) up first!
Couple of words about this approach:
The code can handle (optional) env vars (other than DJANGO_TEST_MODE_GAINARIE - which is mandatory):
- DJANGO_STARTUP_NAME: in case that manage.py has other name (for whatever reason?), or is located in another folder than the Working directory. An important thing here: when specifying file paths, use the platform specific path separator: slash (/) for Nix, bkslash (\) for Win
- DJANGO_STARTUP_TEST_ARGS: additional arguments that
manage.py testaccepts (runmanage.py test --helpto get the whole list). Here, I have to insist on -k / --keepdb which preserves the test database (test_${REGULAR_DB_NAME} by default or set in settings under the TEST dictionary) between runs. When running a single test, creating the DB (and applying all the migrations) and destroying it can be be time consuming (and very annoying as well). This flag ensures that the DB is not deleted at the end and will be reused at the next test run - DJANGO_STARTUP_ADDITIONAL_GLOBALS: this must have the string representation of a Python dict. Any values that for some reason are required by manage.py to be present in the
globals()dictionary, should be placed here
When modifying a Default configuration, all previously created configurations that inherit it, won't be updated, so they have to be manually removed (and will be automatically recreated by new RClicks on their tests)
RClick on the same test (after deleting its previous configuration :d), and voilà:
E:\Work\Dev\VEnvs\py2713x64-django\Scripts\python.exe "C:\Install\PyCharm Community Edition\2016.3.2\helpers\pycharm\utrunner.py" E:\Work\Dev\Django\Tutorials\proj0\src\polls\tests.py::QuestionViewTests::test_index_view_with_no_questions true Testing started at 01:38 ... Using existing test database for alias 'default'... . ---------------------------------------------------------------------- Ran 1 test in 0.390s OK Preserving test database for alias 'default'... Process finished with exit code 0
Debugging also works (breakpoints, and so on ...).
Caveats (so far I identified 2 of them):
- This is benign, it's only an UI issue: utrunner (most likely) has some initialization that PyCharm expects to take place, which obviously doesn't in our case. So, even if the test ended successfully, from PyCharm's PoV they didn't and therefore the Output window will contain a warning: "Test framework quit unexpectedly"
- This is a nasty one, and I wasn't able to get to the bottom of it (yet). Apparently, in utrunner any
input(raw_input) call is not handled very well; the prompt text: "Type 'yes' if you would like to try deleting the test database 'test_tut-proj0', or 'no' to cancel:" (which appears if the previous test run crashed, and its DB was not destroyed at the end) is not being displayed and the program freezes (this doesn't happen outside utrunner), without letting the user to input text (maybe there are threads in the mix?). The only way to recover is stopping the test run, deleting the DB and running the test again. Again, I have to promote themanage.py test -kflag which will get around this problem
I've worked/tested on the following environments:
- Nix (Lnx):
- Ubtu 16.04 x64
- PyCharm Community Edition 2016.3.3
- Python 3.4.4 (VEnv)
- Django 1.9.5
- Win:
- W10 x64
- PyCharm Community Edition 2016.3.2
- Python 2.7.13 (VEnv)
- Django 1.10.6
Notes:
- I will continue investigating the current issues (at least the 2nd one)
- A clean solution would be to override somehow in PyCharm the Unit Test running default settings (what I did from code), but I couldn't find any config files (probably it's in the PyCharm jars?)
- I noticed a lot of files/folders that are specific to Django in the helpers (utrunner's parent) folder, maybe those can be used too, will have to check
As I stated at the beginning, any suggestion is more than welcome!
@EDIT0:
- As I replied to @Udi's comment, this is an alternative for people who can't afford (or companies that aren't willing) to pay the PyCharm Professional Edition license fee (on a quick browse it looks like it's ~100$-200$ / year for each instance)