19

I want to get coverage information of a Cython module using some (unit) tests written in Python. What I have right now is coverage of the tests themselves, i.e. which lines of the tests are executed by running py.test. While nice to look at, I would rather get coverage of the .pyx file, i.e. which lines of the C/Python interface are covered by my tests.

I found some info already but wasn't able to get it running for my project:

http://blog.behnel.de/posts/coverage-analysis-for-cython-modules.html

https://medium.com/@dfdeshom/better-test-coverage-workflow-for-cython-modules-631615eb197a

How to use coverage analysis with Cython

This is the code in question: https://github.com/SCIP-Interfaces/PySCIPOpt/tree/coverage

Community
  • 1
  • 1
mattmilten
  • 6,242
  • 3
  • 35
  • 65
  • 4
    It would be nice to see a minimal example: a cython function, an unit test, you setup-file, the way you install the extension and how you run the tests. “Wasn’t able to get it running” can mean a lot of things – ead Feb 01 '18 at 17:52

1 Answers1

13

The current answer here has two parts:

  • pytest configuration
  • coverage.py configuration

Let's start with the pytest configuration. Currently on your coverage branch there is the .travis.yml file with the following line: py.test --cov tests, this tells pytest-cov to show you the coverage for the tests directory, which you don't want. You can run py.test tests --cov instead or change your setup.cfg and not pass the tests at all.

You would then have the setup.cfg file with:

[tool:pytest]
testpaths = 
    tests

In the .coverage file you need:

[run]
plugins = Cython.Coverage
source = src/pyscipopt
omit =
    tests
    *__init__.py

Specifying the paths separately is obsolete and I have removed it. This way coverage.py knows that it should only check coverage for the src/pyscipopt directory.

After these changes the output I got is:

---------- coverage: platform darwin, python 3.6.4-final-0 -----------
Name                          Stmts   Miss Branch BrPart  Cover
---------------------------------------------------------------
src/pyscipopt/Multidict.py       17     17     10      0     0%
src/pyscipopt/conshdlr.pxi      301    129      0      0    57%
src/pyscipopt/expr.pxi          168     57      0      0    66%
src/pyscipopt/heuristic.pxi      44     18      0      0    59%
src/pyscipopt/lp.pxi            235    177      0      0    25%
src/pyscipopt/pricer.pxi         52     24      0      0    54%
---------------------------------------------------------------
TOTAL                           817    422     10      0    48%

This looks like the proper output you want, but I do see the scip.pyx file is missing. You can also look at a related issue on github. I am not investigating further because it answers your question and I think you will be able to go from here with whatever is missing.

Ania Warzecha
  • 1,796
  • 13
  • 26
  • Unfortunately, I had to revert the code back to the previous state (covering only the tests), because I could not generate an XML report for codecov. I did get the same table showing coverage results for the different .`pxi` files, though. https://github.com/SCIP-Interfaces/PySCIPOpt/pull/128 – mattmilten Feb 12 '18 at 13:14
  • Did you get any errors or it just didn't work? That's so strange, it should work properly at this stage. – Ania Warzecha Feb 12 '18 at 15:59
  • Yes, I get this error: `TypeError: 'NoneType' object is not iterable` from within `Cython/Coverage.py`. Were you able to run `coverage xml` right after generating the coverage report with `py.test --cov`? I also get the same error if I run `py.test --cov --cov-report xml`. – mattmilten Feb 12 '18 at 16:53
  • I didn't really try that. You could try with `py.test --cov --junitxml=C:\path\to\out_report.xml` – Ania Warzecha Feb 12 '18 at 17:33
  • I tried that just now, but it seems that codecov can't handle these xml files. It complains that no files were found in the report although the `coverage.xml` was uploaded. – mattmilten Feb 13 '18 at 13:51
  • Shouldn't it be .coveragerc instead of .coverage? – normanius Dec 06 '20 at 23:59
  • @AniaWarzecha When I run coverage.py on the src folder, the coverage % is too less while for the tests folder (the test files in the tests folder are linked to the src files) it is higher and better as it should be. So basically, in general we run the coverage for the tests folder for calculating the overall code coverage, right ? Or is that not the case ? I'm a beginner in unit testing and required some help. I had pinged you the same on FB messenger. Kindly ignore that. – Siddharth Singh Mar 09 '23 at 08:18