Test coverage in teamcity with Django

Question

I have got teamcity working and it builds and runs a custom testrunner (http://pypi.python.org/pypi/teamcity-messages)

I loosly foolowed this post: TeamCity for Python/Django continuous integration

My run_suite method looks like this:

from teamcity import underTeamcity
from teamcity.unittestpy import TeamcityTestRunner
return TeamcityTestRunner().run(suite)

I currently use django_coverage with coverage.py and I would like teamcity to get the test coverage data.

I am not bound to teamcity but I prefere using it as a CI server but I can change to another if it is easier.

How can I get the data that ./manage.py test_coverage prints in teamcity?

Answer 1

TeamCity Coverage

In TeamCity I cover Django in the following way

Create coverage report by call make ci_test command use Makefile.

VENV_PATH := $(HOME)/venv/bin
PROJ_NAME := my_awesome_project

# ...

ci_test: cover_test cover_report

cover_test:
    $(VENV_PATH)/coverage run --source=$(PROJ_NAME) manage.py test -v 2 --noinput

cover_report:
    $(VENV_PATH)/coverage report -m
    $(VENV_PATH)/coverage html
    $(VENV_PATH)/coverage-badge > htmlcov/coverage.svg

The cover_test command runs the Django tests, and measures the coverage of the code. The cover_report command prints a cover report to the console, and also generates an html report and, using the coverage-badge utility, generates a beautiful badge with the badge code coverage status .

After that, artifacts are collected in the TeamCity (tab General Settings)

They are collected in a tab Artifacts

And available on CI server by path:

• /repository/download/%teamcity.project.id%/%teamcity.build.id%:id/htmlcov /index.html
• /repository/download/%teamcity.project.id%/.lastFinished/htmlcov/index.html

---

GitHub report coverage

Finally push GitHub hook to display build coverage state in repo:

Coverage Pending (before build)

OWNER="<GITHUB OWNER>";
REPO="<REPO NAME>";
SHA="%build.vcs.number%";

curl "https://api.github.com/repos/$OWNER/$REPO/statuses/$SHA" \
    -X POST \
    -H "Content-Type: application/json" \
    -H "Authorization: token <GITHUB API TOKEN>" \
    -d '{
        "state": "pending",
        "description": "Coverage pending.",
        "context": "continuous-integration/coverage"
    }'

Coverage badge copy

BADGE="/path/to/public/dir/badges/%teamcity.project.id%/%teamcity.build.branch%-coverage.svg"
DIR=$(dirname "${BADGE}")
mkdir -p $DIR
cp -f htmlcov/coverage.svg $BADGE

Coverage Finish hook

OWNER="<GITHUB OWNER>";
REPO="<REPO NAME>";
SHA="%build.vcs.number%";

REPORT_URL="http://<YOU TEAMCITY DOMAIN>/repository/download/%teamcity.project.id%/%teamcity.build.id%:id/htmlcov/index.html";

COVERAGE=$(cat ./htmlcov/index.html | grep '<span class="pc_cov">' | grep -o '[0-9]\+');

if [ "$COVERAGE" -ge "85" ]; then
    STATUS='success';
else
    STATUS='failure';
fi

curl "https://api.github.com/repos/$OWNER/$REPO/statuses/$SHA" \
    -X POST \
    -H "Content-Type: application/json" \
    -H "Authorization: token <GITHUB API TOKEN>" \
    -d '{
        "state": "'$STATUS'",
        "target_url": "'$REPORT_URL'",
        "description": "Coverage '$COVERAGE'%",
        "context": "continuous-integration/coverage"
    }'

Result in github:

Blog post about this ru: https://maks.live/articles/drugoe/otchety-coverage-v-teamcity/

Answer 2

I use teamcity-nose with the following configuration in settings.py:

TEST_RUNNER = 'django_nose.NoseTestSuiteRunner'
NOSE_ARGS = ['-v', '-s', '--rednose', '--with-selenium',]

if os.getenv('TEAMCITY_PROJECT_NAME') is not None:
   # whatever special teamcity settings you might have go here

My build step that does the testing looks like this:

. /opt/teamcity/virtualenvs/myproj/bin/activate
 dropdb test_myproj-teamcity &> /dev/null # bug that is not destroying database
 manage.py test

My project's manage.py is on the path (I install in the virtualenv bin via setup.py) so you will have to add the path if you do otherwise.

I never managed to add the coverage in the test itself as there are problems with package versioning, so with the latest coverage package i Just added it in an extra build step:

. /opt/teamcity/virtualenvs/myproj/bin/activate
 coverage html --include=myproj/*.*
 cloc . --out=./htmlcov/cloc.txt

You can then add a tab that includes the coverage html if you add this to your artifact:

./htmlcov/

I also add a tab with the line counter too, you will need to have cloc or the linecounter of your choice installed.

I also have an extra build configuration for deploying to staging server once a night via fab (just activate and fab as usual), and an extra build for automatically installing pip requirements if the pip files change, by adding this to the triggering rules for the "pip install -r requirements.pip" build:

+:**.pip

And I add this to my test build so that it doesn't run when pip and some other files change that don't effect the build test:

+:.
-:**.pip
-:*fabfile.py
-:*myproj/conf/*
+:*myproj/conf/teamcity/*