pytest文檔35-Hooks函數之統計測試結果(pytest_terminal_summary)

前言

用例執行完成后，我們希望能獲取到執行的結果，這樣方便我們快速統計用例的執行情況。
也可以把獲取到的結果當成總結報告，發郵件的時候可以先統計測試結果，再加上html的報告。

pytest_terminal_summary

關於TerminalReporter類可以在_pytest.terminal中查看到

from _pytest import terminal

pytest_terminal_summary(terminalreporter, exitstatus, config)
最后的結果匯總,可以拿到所有的執行結果
參數：
- terminalreporter (_pytest.terminal.TerminalReporter) – 內部使用的終端測試報告對象
- exitstatus (int) – 返回給操作系統的返回碼
- config(_pytest.config.Config) - pytest config對象

TerminalReporter部分代碼

class TerminalReporter(object):
    def __init__(self, config, file=None):
        import _pytest.config

        self.config = config
        self._numcollected = 0
        self._session = None
        self._showfspath = None

        self.stats = {}
        self.startdir = config.invocation_dir


    def report_collect(self, final=False):
        if self.config.option.verbose < 0:
            return

        if not final:
            # Only write "collecting" report every 0.5s.
            t = time.time()
            if (
                self._collect_report_last_write is not None
                and self._collect_report_last_write > t - REPORT_COLLECTING_RESOLUTION
            ):
                return
            self._collect_report_last_write = t

        errors = len(self.stats.get("error", []))
        skipped = len(self.stats.get("skipped", []))
        deselected = len(self.stats.get("deselected", []))
        selected = self._numcollected - errors - skipped - deselected
        if final:
            line = "collected "
        else:
            line = "collecting "
        line += (
            str(self._numcollected) + " item" + ("" if self._numcollected == 1 else "s")
        )
        if errors:
            line += " / %d errors" % errors
        if deselected:
            line += " / %d deselected" % deselected
        if skipped:
            line += " / %d skipped" % skipped
        if self._numcollected > selected > 0:
            line += " / %d selected" % selected
        if self.isatty:
            self.rewrite(line, bold=True, erase=True)
            if final:
                self.write("\n")
        else:
            self.write_line(line)

案例參考

先在test_a.py寫幾個用例

# test_a.py
import pytest
# 上海-悠悠

def test_1():
    print("測試用例1111")
    assert 1 == 1


@pytest.mark.skip("跳過")
def test_2():
    print("測試用例22222")
    assert 1 == 1


def test_3():
    print("測試用例3333")


def test_4():
    print("測試用例44444444")
    assert 1 == 2

test_b.py用例參考

# test_b.py
import time
# 上海-悠悠

def test_5():
    print("測試用例55555555")
    time.sleep(3)


def test_6():
    print("測試用例66666666")
    time.sleep(3)
    assert 1 == 2

於是在conftest.py中寫個 pytest_terminal_summary 函數收集測試結果

import time
from _pytest import terminal
# 上海-悠悠


def pytest_terminal_summary(terminalreporter, exitstatus, config):
    '''收集測試結果'''
    print(terminalreporter.stats)
    print("total:", terminalreporter._numcollected)
    print('passed:', len(terminalreporter.stats.get('passed', [])))
    print('failed:', len(terminalreporter.stats.get('failed', [])))
    print('error:', len(terminalreporter.stats.get('error', [])))
    print('skipped:', len(terminalreporter.stats.get('skipped', [])))
    # terminalreporter._sessionstarttime 會話開始時間
    duration = time.time() - terminalreporter._sessionstarttime
    print('total times:', duration, 'seconds')

運行結果

D:\soft\pytest_xuexi_demo>pytest
============================= test session starts =============================
platform win32 -- Python 3.6.0, pytest-4.5.0, py-1.5.4, pluggy-0.13.1
rootdir: D:\soft\pytest_xuexi_demo
plugins: allure-pytest-2.8.6, PyTestReport-0.1.9.3, forked-0.2, html-1.19.0, metadata-1.7.0, repeat-0.7.0, rerunfailures-8.0, xdist-1.23.2
collected 6 items

test_a.py .s.F                                                           [ 66%]
test_b.py .F                                                             [100%]

================================== FAILURES ===================================
___________________________________ test_4 ____________________________________

    def test_4():
        print("測試用例44444444")
>       assert 1==2
E       assert 1 == 2

test_a.py:21: AssertionError
---------------------------- Captured stdout call -----------------------------
測試用例44444444
___________________________________ test_6 ____________________________________

    def test_6():
        print("測試用例66666666")
        time.sleep(3)
>       assert 1 == 2
E       assert 1 == 2

test_b.py:18: AssertionError
---------------------------- Captured stdout call -----------------------------
測試用例66666666
{'': [<TestReport 'test_a.py::test_1' when='setup' outcome='passed'>, <TestReport 'test_a.py::test_1' when='teardown' outcome='passed'>, <TestReport 'test_a.py::test_2' when='teardown' outcome='passed'>, <TestReport 'test_a.py::test_3' when='setup' outcome='passed'>, <TestReport 'test_a.py::test_3' when='teardown' outcome='passed'>, <TestReport 'test_a.py::test_4' when='setup' outcome='passed'>, <TestReport 'test_a.py::test_4' when='teardown' outcome='passed'>, <TestReport 'test_b.py::test_5' when='setup' outcome='passed'>, <TestReport 'test_b.py::test_5' when='teardown' outcome='passed'>, <TestReport 'test_b.py::test_6' when='setup' outcome='passed'>, <TestReport 'test_b.py::test_6' when='teardown' outcome='passed'>], 'passed': [<TestReport 'test_a.py::test_1' when='call' outcome='passed'>, <TestReport 'test_a.py::test_3' when='call' outcome='passed'>, <TestReport 'test_b.py::test_5' when='call' outcome='passed'>], 'skipped': [<TestReport 'test_a.py::test_2' when='setup' outcome='skipped'>], 'failed': [<TestReport 'test_a.py::test_4' when='call' outcome='failed'>, <TestReport 'test_b.py::test_6' when='call' outcome='failed'>]}
total: 6
passed: 3
failed: 2
error: 0
skipped: 1
total times: 6.150860786437988 seconds
================ 2 failed, 3 passed, 1 skipped in 6.15 seconds ================

setup和teardown異常情況

如果setup出現異常，test_b.py代碼修改下

# test_b.py
import time
import pytest
# 上海-悠悠


@pytest.fixture(scope="function")
def setup_demo():
    raise TypeError("ERROR!")


def test_5(setup_demo):
    print("測試用例55555555")
    time.sleep(3)


def test_6():
    print("測試用例66666666")
    time.sleep(3)
    assert 1 == 2

重新運行用例,結果如下

total: 6
passed: 2
failed: 2
error: 1
skipped: 1
成功率：33.33%
total times: 3.1817877292633057 seconds
=========== 2 failed, 2 passed, 1 skipped, 1 error in 3.18 seconds ============

此時統計結果沒什么問題，接下來看teardown異常情況

# test_b.py
import time
import pytest
# 上海-悠悠


@pytest.fixture(scope="function")
def setup_demo():
    yield 
    raise TypeError("ERROR!")


def test_5(setup_demo):
    print("測試用例55555555")
    time.sleep(3)


def test_6():
    print("測試用例66666666")
    time.sleep(3)
    assert 1 == 2

運行結果

{'': [<TestReport 'test_a.py::test_1' when='setup' outcome='passed'>, <TestReport 'test_a.py::test_1' when='teardown' outcome='passed'>, <TestReport 'test_a.py::test_2' when='teardown' outcome='passed'>, <TestReport 'test_a.py::test_3' when='setup' outcome='passed'>, <TestReport 'test_a.py::test_3' when='teardown' outcome='passed'>, <TestReport 'test_a.py::test_4' when='setup' outcome='passed'>, <TestReport 'test_a.py::test_4' when='teardown' outcome='passed'>, <TestReport 'test_b.py::test_5' when='setup' outcome='passed'>, <TestReport 'test_b.py::test_6' when='setup' outcome='passed'>, <TestReport 'test_b.py::test_6' when='teardown' outcome='passed'>], 'passed': [<TestReport 'test_a.py::test_1' when='call' outcome='passed'>, <TestReport 'test_a.py::test_3' when='call' outcome='passed'>, <TestReport 'test_b.py::test_5' when='call' outcome='passed'>], 'skipped': [<TestReport 'test_a.py::test_2' when='setup' outcome='skipped'>], 'failed': [<TestReport 'test_a.py::test_4' when='call' outcome='failed'>, <TestReport 'test_b.py::test_6' when='call' outcome='failed'>], 'error': [<TestReport 'test_b.py::test_5' when='teardown' outcome='failed'>]}
total: 6
passed: 3
failed: 2
error: 1
skipped: 1
成功率：50.00%
total times: 6.18759298324585 seconds
=========== 2 failed, 3 passed, 1 skipped, 1 error in 6.19 seconds ============

這個時候總用例是6，但是2 failed, 3 passed, 1 skipped, 1 error加起來的個數是7，這個為什么？

從 terminalreporter.stats 可以看出 passed 里面 when='call' 時候統計了一次 test_5 用例

<TestReport 'test_b.py::test_5' when='call' outcome='passed'>

error 里面 when='teardown' 又統計了一次 test_5 用例

'error': [<TestReport 'test_b.py::test_5' when='teardown' outcome='failed'>]

when='teardown' 是測試用例的后置操作，一般用於數據的清理，報錯了的話不影響測試用例的執行結果，所以可以忽略掉

修改后的最終代碼如下

import time
from _pytest import terminal
# 上海-悠悠


def pytest_terminal_summary(terminalreporter, exitstatus, config):
    '''收集測試結果'''
    # print(terminalreporter.stats)
    print("total:", terminalreporter._numcollected)
    print('passed:', len([i for i in terminalreporter.stats.get('passed', []) if i.when != 'teardown']))
    print('failed:', len([i for i in terminalreporter.stats.get('failed', []) if i.when != 'teardown']))
    print('error:', len([i for i in terminalreporter.stats.get('error', []) if i.when != 'teardown']))
    print('skipped:', len([i for i in terminalreporter.stats.get('skipped', []) if i.when != 'teardown']))
    print('成功率：%.2f' % (len(terminalreporter.stats.get('passed', []))/terminalreporter._numcollected*100)+'%')

    # terminalreporter._sessionstarttime 會話開始時間
    duration = time.time() - terminalreporter._sessionstarttime
    print('total times:', duration, 'seconds')

運行后的結果

total: 6
passed: 3
failed: 2
error: 0
skipped: 1
成功率：50.00%
total times: 6.20133113861084 seconds
=========== 2 failed, 3 passed, 1 skipped, 1 error in 6.20 seconds ============

拿到測試結果

很多小伙伴問到如何拿到測試結果，這里我把測試結果保存為txt文件，你們也可以保存json文件

import time
from _pytest import terminal
# 作者-上海悠悠 QQ交流群:717225969
# blog地址 https://www.cnblogs.com/yoyoketang/


def pytest_terminal_summary(terminalreporter, exitstatus, config):
    '''收集測試結果'''
    # print(terminalreporter.stats)
    total = terminalreporter._numcollected
    passed= len([i for i in terminalreporter.stats.get('passed', []) if i.when != 'teardown'])
    failed=len([i for i in terminalreporter.stats.get('failed', []) if i.when != 'teardown'])
    error=len([i for i in terminalreporter.stats.get('error', []) if i.when != 'teardown'])
    skipped=len([i for i in terminalreporter.stats.get('skipped', []) if i.when != 'teardown'])
    successful = len(terminalreporter.stats.get('passed', []))/terminalreporter._numcollected*100
    # terminalreporter._sessionstarttime 會話開始時間
    duration = time.time() - terminalreporter._sessionstarttime
    print('total times: %.2f' % duration, 'seconds')

    with open("result.txt", "w") as fp:
        fp.write("TOTAL=%s" % total+"\n")
        fp.write("PASSED=%s" % passed+"\n")
        fp.write("FAILED=%s" % failed+"\n")
        fp.write("ERROR=%s" % error+"\n")
        fp.write("SKIPPED=%s" % skipped+"\n")
        fp.write("SUCCESSFUL=%.2f%%" % successful+"\n")
        fp.write("TOTAL_TIMES=%.2fs" % duration)

郵件發送

郵件發送是配合jenkins發送郵件，可以參考這篇https://www.cnblogs.com/yoyoketang/p/14956235.html