Customizing pytest junitxml failure reports
EDIT: Since I needed access to the test item's funcargs (and test result) for my reporting, I was able to move the logic to the pytest_runtest_makereport(item, __multicall__)
hook. The trick is to execute the multicall, which returns the report object:
@pytest.mark.tryfirst
def pytest_runtest_makereport(item, call, __multicall__):
report = __multicall__.execute()
# then I was able to manipulate report and get the same results as below
Bruno's answer gave me the motivation I needed to analyze this feature more thoroughly :)
So here's how it works:
def pytest_runtest_logreport(report):
if report.failed:
report.longrepr.sections.append(("Header", "Message", "-"))
report.sections.append(("Captured stdout", "This is added to stdout"))
report.sections.append(("Captured stderr", "This is added to stderr"))
report.sections.append(("Custom Section", "This can only be seen in the console - the xml won't have it."))
The longrepr
attribute is only available in case of failures. It takes a 3-tuple, the last value being a character used to decorate the decorate/surround the header. It will appear in the "failure" section of the report:
----------------------------------- Header ------------------------------------
Message
Custom sections will create additional result sections to be printed out to the console. But they won't make it to junitxml:
------------------------------- Custom Section --------------------------------
This can only be seen in the console - the xml won't have it.
The junitxml report only has 2 sections: out and err. To add custom text to it, you must create sections called "Captured std" and only those will make it to the xml file. Any other name will result in a custom section that will only be seen in the console.
Here's the resulting junitxml using the code above, with some reformatting for the sake of this post:
<?xml version="1.0" encoding="utf-8" ?>
<testsuite errors="0" failures="1" name="pytest" skips="0" tests="1" time="0.646">
<testcase classname="test_reporting" name="test_fail" time="0.000999927520752">
<failure message="test failure">
@ut def test_fail(): > assert 0, "It failed"
E AssertionError: It failed
E assert 0 test_reporting.py:346: AssertionError
----------------------------------- Header ------------------------------------
Message
</failure>
<system-out>This is added to stdout</system-out>
<system-err>This is added to stderr</system-err>
</testcase>
</testsuite>
To add information to a test report (XML, console or otherwise) please take a look at reporting hooks, more specifically at pytest_runtest_logreport.