[Cython] Cython coverage and multiple projects
I'd asked this of Robert Bradshaw earlier and was pointed in this direction (specifically towards Stefan), so here goes! I'm having trouble with getting Cython to play nicely with the coverage.py tool via the Cython.Coverage plug-in. We have tests that are running from a different setup.py project than the one which contains the Cython files (the projects respectively being for the duration of this discussion the 'tests' project and the 'source' project). The generated coverage file contains the line data for individual .pyx files, but the paths of the .pyx files themselves *within* the .coverage file appear to be rooted in the tests project rather than the site-packages folder in which the source project's files were installed (i.e. paths that never exist throughout the edit-build-test cycle). After editing the .coverage file manually to point to the source project directory, coverage.py reports that the .pyx files are not python files and errors out... but the .coveragerc file contains the plug-in module via `[run]\n plugins = Cython.Coverage`, so I'd have expected the Cython coverage.py plug-in to catch those files and handle them instead of letting the rest of coverage.py get confused over them. Furthermore, this is against the 4.0 coverage.py release rather than the 4.0b2 coverage.py release (which appears to be the version against which the latest edit to the Cython code base was made). I don't see any API differences between the two in the plug-ins, but I might have missed something so maybe that's relevant. A secondary issue: the .pyx files are never copied over to the installation directory, so even if the paths were 'correct' (with respect to being copied into the site-packages folder or being accessible via developer mode pip-installs), they wouldn't be found. This seems like it wouldn't necessarily be an issue from a cursory glance at the source code as the file tracer in the Cython coverage.py plug-in appears to spit out blank lines in such a scenario. Complicating matters further, we're running the tests through pytest. That said, I've given up on trying to get pytest-cov to play nicely with this until I can get a manual invocation of `coverage report ...` to produce output describing the .pyx files. Thanks for reading this far. Got any pointers? Context: https://github.com/soltanmm/grpc/tree/cy-wip/src/python Basic steps to run the tests: - cd $GRPC_REPOSITORY_ROOT - make - CONFIG=opt ./tools/run_tests/build_python.sh 2.7 - CONFIG=opt PYVER=2.7 ./run_tests/run_python.sh ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython coverage and multiple projects
Given that we have control over how and where source files get used by the virtualenv, your suggestion seems within reach by doing a bit of temporary-directory dancing. I've been hitting some issues with that (unrelated to Cython as far as I can tell), and will attempt to update this thread after digging. Some more background: we'd been trying to avoid running tests directly within the source directory and in a separate package to 'cleanly' capture how a user of the library would see the code (from grabbing it e.g. on PyPI to using it) and to keep the user free of the test modules on installation, respectively. The ideals are gradually getting, uh, *influenced* by trying to keep a variety of tools happy. It may be that the ideals simply aren't meant to be (though certainly the separate-package reasoning may be solvable via a host of other means, e.g. mucking around with the manifest if the install->test reasoning is scrapped). At the risk of bleeding frustration onto the page: a myriad of bugs, 'features', and such across multiple tools have turned this into an exhausting grid search across axes among our desiderata (e.g. parallel test execution), test tools, test setups, coverage setups, project organization, test plugins, and varying degrees of separation of responsibility between some of our shim scripts and the tools themselves. Certainly this is not a problem originating from Cython, but I do wish there'd been some complete blessed example project demonstrating the basic coverage+testing setup along the way for Cython, possibly with some glue to e.g. py.test or nosetests or nose2 or even a custom distutils/setuptools test command etc.. Thanks for the response! On Sat, Oct 10, 2015 at 8:27 AM, Stefan Behnel wrote: > Masood Malekghassemi via cython-devel schrieb am 08.10.2015 um 01:06: > > I'm having trouble with getting Cython to play nicely with the > coverage.py > > tool via the Cython.Coverage plug-in. > > > > We have tests that are running from a different setup.py project than the > > one which contains the Cython files (the projects respectively being for > > the duration of this discussion the 'tests' project and the 'source' > > project). The generated coverage file contains the line data for > individual > > .pyx files, but the paths of the .pyx files themselves *within* the > > .coverage file appear to be rooted in the tests project rather than the > > site-packages folder in which the source project's files were installed > > (i.e. paths that never exist throughout the edit-build-test cycle). > > This seems to be a rather unusual setup. The "expected" setting is that > tests live somewhere within the source checkout and are run on the > non-installed sources, usually as part of the CI build process, sometimes > even just as part of a separate reporting process. Why do you want to > create coverage reports after the installation? > > > > A secondary issue: the .pyx files are never copied over to the > installation > > directory, so even if the paths were 'correct' (with respect to being > > copied into the site-packages folder or being accessible via developer > mode > > pip-installs), they wouldn't be found. > > Sure, that's a difference from .py/.pyc files which are getting executed > directly. For Cython code, there is an additional compile/build step that > generates the executable code from the sources, which are then no longer > needed. That's another reason why support for coverage reporting after the > installation seems foreign here and wouldn't be of much use. > > So, my suggestion would be to run the coverage reports within the source > tree after building it, when all source files are readily available. But > I'm obviously missing a lot of information about your actual project setup > and the reasons behind what you are doing here. > > Stefan > > ___ > cython-devel mailing list > cython-devel@python.org > https://mail.python.org/mailman/listinfo/cython-devel > ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] Cython coverage and multiple projects
After reorganizing the code such that all the tests are located back in the same project, setting up the running of tests in-tree, and using the unittest module directly, there's been much success! Automatically suffixed .coverage files are generated per test case and merged at the end (keeping open the possibility of process-level parallelism later). The coverage report at the end after invoking `coverage combine` on the command line is as expected. There's a strange caveat, though: using the coverage.py API and calling 'coverage.Coverage().combine()' following the test runs erases the Cython coverage data from the .coverage files. I haven't poked at it much yet, as this is only a papercut for us, but if it's an issue with the Cython plug-in, I figure y'all ought to know. Thanks again! ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
[Cython] Cython coverage and having the package root in a child directory of the coverage working directory
tl;dr: How ought one fix file tracing paths for Cython coverage? Hello y'all, I'm currently trying to integrate C code into the GRPC Python project wherein said code is outside of my control, and belonging to the umbrella GRPC project. The Python+Cython code exists in a subdirectory (i.e. the package_dirs for setup.py includes an entry '': 'src/python/grpcio'). Everything works great, *except* for the Cython coverage plugin. For some strange reason, when the _c_files_map of the Plugin object is created, it appears to assume that the files it's reading are coming directly off of the invocation site of coverage.py as opposed to where the files actually are. Going a bit deeper: in the generated .c file for the .pyx, the package directory stem 'src/python/grpcio' is missing from comments on __Pyx_TraceLine(...): /* "grpc/_cython/cygrpc.pyx":53 * grpc_shutdown() * * _module_state = _ModuleState() # << * */ This doesn't appear to be a problem for .pxi files, however: /* "src/python/grpcio/grpc/_cython/_cygrpc/server.pyx.pxi":32 * cimport cpython * * import time # << * * */ We cannot simply move the Cython files around to being relative to umbrella root, as that would involve changing the umbrella project structure beyond the parameters of our current desiderata. I'm currently working around this problem by just ignoring the .pyx file (it contains little code anyway), and keeping only the .pxi files. I'd like to fix this (it doesn't seem difficult), but haven't looked into where one ought to start yet. An aside: we're using .pxi files instead of .pxds now due to some inexplicable behavior wherein a .pxd induces an expectation that a module exists when it doesn't. We're not splitting the extension out into multiple modules because there must now be only one copy of the statically linked gRPC core (wherein static linkage is among recent desiderata for distribution purposes). Context: https://github.com/grpc/grpc/tree/350345ffc70c93295ad5b78b306f9668a6332589 (steps to see what's happening: make; CONFIG=opt ./tools/run_tests/run_python.sh. The relevant C file may be examined under src/python/grpcio/grpc/_cython following the build). Thanks for reading! ___ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
