Pytest usage in DejaGNU?

[Martin Liška]

Hello.

GCOV tests suffer from tests that would cover the intermediate format.
It's a JSON format and and I'm attaching an example of its output.

I would really like to use Python to make more complex tests:

$ cat test_json.py
import pytest
import json

def test_gcov_output():
    data = json.load(open('gcov.json'))
    assert len(data['files']) == 1
    f0 = data['files'][0]
    assert f0['file'] == 'gcov-lambda.C'
    assert len(f0['functions']) == 3

    fns = {}
    for fn in f0['functions']:
        fns[fn['name']] = fn
    lines = f0['lines']

    for line in lines:
        lineno = line['line_number']
        linefn = line['function_name']
        assert linefn in fns
        fn = fns[linefn]
        assert fn['start_line'] <= lineno and lineno <= fn['end_line']

I see it pretty complicated to do the same in DejaGNU. Mainly due the missing
JSON parser.

Would it be possible to make optional Python tests in our testsuite?
I can imagine a simple pytest wrapper that will do something like:

+proc pytest-execute { dgargs } {
+    verbose "dg-pytest-execute: ${dgargs}" 2
+    set script [lindex $dgargs 0]
+    verbose "  script: ${script}" 2
+
+    spawn -noecho pytest -rA -s --tb=no $script
+
+    expect {
+      -re "FAILED .*" {
+       fail "pytest $expect_out(0,string)"
+      }
+      -re "PASSED .*" {
+       pass "pytest $expect_out(0,string)"
+      }
+    }
+}

as Pytest can provide a reasonable close output:

===========================================================================================================================
 short test summary info 
============================================================================================================================
PASSED test_json.py::test_gcov_output
PASSED test_json.py::test_gcov_output
PASSED test_json.py::test_gcov_output

Thoughts?
Martin

gcov.json
Description: application/json