Running pytest test functions inside a jupyter notebook

Question

I'm working on a presentation about python testing options, and one of the technologies I want to demo is pytest. I'm planning to do the presentation from an jupyter/ipython notebook. Ideally I'd like to be able to define a test function in a cell and then use pytest to run just that function; that way I can demonstrate individual features of pytest, etc.

Is there an existing way to do this? None of the interactive features I saw in the pytest api seem to fit the bill, but I'm certainly no pytest expert. I'd be happy to write a plugin to do it, so any advice on the best approach in that direction would be great.

I did see pytest-ipynb, but it does not seem to do what I need. If that assessment is wrong, advice on how to use that would be great, too.

Just to add to the data here, but not actually answer the question, there is https://github.com/computationalmodelling/nbval — Russel Winder, Jan 30 '17 at 10:16

Jean-Francois T. · Answer 1 · 2023-08-16T07:43:25.010

30

There is a similar module that looks very mature: https://github.com/chmp/ipytest

It is referenced in the module developed by @akaiola and is more active and regularly updated / maintained.

In particular, the tests could use some functions defined in other cells, which is the whole purpose of performing tests inside the Notebook (see example below).

To use it:

Put in a cell an import and configuration of ipytest:

import ipytest
ipytest.autoconfig()

Write some tests in cell(s)
Either run cells with test with %%ipytest magic for each cell with test ... or put a ipytest.run() in a cell after the tests

You can check the official example: https://github.com/chmp/ipytest/blob/main/Example.ipynb

Or an example I did:

And another example:

edited Aug 16 '23 at 07:43

answered Dec 20 '18 at 07:10

Jean-Francois T.

11,549
7
68
107

Also - one can use the `%%writefile` magic, then call `!pytest` - demonstrated in [this talk by Florian Bruhin](https://bruhin.software/ins-pytest/). – Yinon Ehrlich Aug 15 '23 at 07:16
@YinonEhrlich Interesting addition. However, this would be limited if we want to test functions defined in other cells of your notebook... unless you duplicate writing of this code as well. – Jean-Francois T. Aug 16 '23 at 07:22

score 22 · Accepted Answer · answered Dec 25 '16 at 20:32

22

I created ipython_pytest for this purpose and used it in my PyCon Helsinki 2016 presentation.

You would use the tool like this:

%%pytest

def test_my_stuff():
    assert 42 == 42

Unfortunately I couldn't find a way to make functions and classes defined in the notebook available to test functions in the test cell. I'd appreciate if someone comes up with a solution for that.

answered Dec 25 '16 at 20:32

akaihola

26,309
7
59
69

That looks perfect! That's what I get for not coming to the conference. – abingham Dec 26 '16 at 01:28
Oops, sorry, thought I had. – abingham Dec 26 '16 at 13:45
@akaihola found your talk. Would you like to post it too? ) – pylang Dec 28 '16 at 20:57
@pylang I'm planning to turn the slides into a series of blog posts. I seem to need to be kicked a bit though... Oh, and the talk seems to be now on Youtube, too: https://www.youtube.com/watch?v=jOvkYnSBKeI – akaihola Dec 28 '16 at 21:32
@akaihola I offer a friendly kick then )) This a a great contribution to notebooks. – pylang Dec 28 '16 at 23:45
1

could someone find a solution to reach objects out of the test cell ? – clockworks Dec 30 '17 at 21:08

score 6 · Answer 3 · answered Oct 21 '20 at 00:00

6

My 2 cents here since I've already solved this issue for myself 2x and both times come back to this site. For my future self:

For a project structured in the following way

prod_folder:
* data
* code
    * run_notebook.ipynb
* tests
   * test_some_test.py

If you want to run pytest from within run_notebook.ipynb the following cell-magic worked for my purposes:

!pytest ../tests

answered Oct 21 '20 at 00:00

Aus_10

670
7
15

What kernel/tool do you refer to here? – Adam Hunyadi Dec 18 '20 at 12:44
a jupyter python kernel cell – Aus_10 Dec 22 '20 at 01:26
1

The `!` makes is a shell call. You are instantiating a new Python session from the shell and running pytest. – Joshua Cook Mar 01 '21 at 04:53
That's a great point. Hadn't considered that thanks. Mine was very much "it seems to work and go with it" – Aus_10 Mar 01 '21 at 18:49
I also often come back to this site and ends up in this page where there is my answer ;) – Jean-Francois T. Oct 25 '21 at 11:10

score 3 · Answer 4 · answered Dec 09 '22 at 16:01

In Google Colab I usually do the following procedure:

Create a file name for the respective cell:

Here's an example:

%%file test_list.py

# find the mismatched elements in list `l1` which are not in list `l2`

def solution(l1: list, l2: list) -> list:
    return [
        e
        for e in l1
        if e not in [e for e in l2]
    ]

import pytest

a = [1,2,3]
b = [3,4,None,None]
TEST_CASES = [[a,b]]

class TestCase:
    @pytest.mark.parametrize("test_case", TEST_CASES)
    def test_events(self, test_case):
        assert solution(test_case[0], test_case[1]) == [1,2]

Run pytest in another cell:

!python -m pytest test_list.py

Out:

============================= test session starts ==============================
platform linux -- Python 3.8.16, pytest-3.6.4, py-1.11.0, pluggy-0.7.1
rootdir: /content, inifile:
plugins: typeguard-2.7.1
collected 1 item                                                               

test_list.py .                                                           [100%]

=========================== 1 passed in 0.02 seconds ===========================

Running pytest test functions inside a jupyter notebook

4 Answers4

Linked