8

I have some code below that I'm using to take an input of files, open and process, and then output some data. I've gotten the functionality working and I'm unit testing it now, below is an example of the code.

def foo(dir):
    path_to_search = join(dir, "/baz/foo")

   if isdir(path_to_search): 
   #path exists so do stuff...

        for fname in listdir(path_to_search):
            do_stuff()

   else:
       print "path doesn't exist"

I've been able to create a test where the past doesn't exist easily enough, but as you can see above I assert that the "/baz/foo" portion of the directory structure exists (in production the directory structure must have this file, in some cases it won't and we won't need to process it.)

I've tried to create a temporary directory structure using TempDir and join, but the code always kicks out saying the path doesn't exists.

Is it possible to mock the output of os.listdir such that I won't need to create a temporary directory structure that follows the needed /baz/foo convention?

openingceremony
  • 181
  • 1
  • 3
  • You could use Labix Mocker and [`mocker.replace()`](http://labix.org/mocker#CA-a25a2c16ecea818439349b5f0b23eda994ab34bf_010) for this. – Lukas Graf Oct 17 '14 at 20:54
  • @LukasGraf: better to use the stdlib `unittest.mock` or it's backport, `mock`. – Martijn Pieters Oct 17 '14 at 20:58
  • @MartijnPieters I personally prefer Labix Mocker mainly for the reason that they seem to be the only ones that got patching right. See [Where to Patch](http://www.voidspace.org.uk/python/mock/patch.html#where-to-patch). Labix mocker takes care of that for you by actually gathering references to objects it patches, and patching and resetting those as well, and it just works. – Lukas Graf Oct 17 '14 at 21:24
  • @LukasGraf: I actually dislike such magic. I want to be in control as to what code is patched, not have it magically apply it *everywhere*. – Martijn Pieters Oct 17 '14 at 21:49
  • @MartijnPieters in real code, definitely. In tests, I welcome anything that makes my life easier. As for being in control: If a simple refactoring like changing `import a; a.SomeName` to `from a import SomeName` can break your tests in subtle ways, that's a loss of control as well. – Lukas Graf Oct 17 '14 at 22:03

1 Answers1

11

You don't need to create a fake directory structure, all you need to do is mock the isdir() and listdir() functions.

Using the unittest.mock library (or the external mock library, which is the exact same thing for Python versions < 3.3):

try:
    # Python >= 3.3 
    from unittest import mock
except ImportError:
    # Python < 3.3
    import mock

with mock.patch('yourmodule.isdir') as mocked_isdir, \
        mock.patch('yourmodule.listdir') as mocked_listdir:
    mocked_isdir.return_value = True
    mocked_listdir.return_value = ['filename1', 'filename2']

    yourmodule.foo('/spam/eggs')

    mocked_isdir.assert_called_with('/spam/eggs/baz/foo')
    mocked_listdir.assert_called_with('/spam/eggs/baz/foo')
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Hmm, could you explain this a little bit more, it doesn't seem to be working when I give it 'spam/eggs' as input. I see we are saying that isdir should result to true, and that list dir should return these files names, now are these file names "real" or do I need to temp create them so that they can be read? – openingceremony Oct 17 '14 at 21:36
  • @openingceremony: Your code isn't requiring them to be real, because any functions that would otherwise require a real path have been replaced by your mocks. They don't need real paths, you just configure them to return whatever your code-under-test expects to see. – Martijn Pieters Oct 17 '14 at 21:46
  • I see, when I ran this code the assertions failed because they were not called, shouldn't my code have called isdir/listdir? – openingceremony Oct 17 '14 at 21:53
  • 1
    @openingceremony: make sure you patched the correct objects! See [*Where to patch*](https://docs.python.org/3/library/unittest.mock.html#where-to-patch). If your mocks weren't called, you didn't patch the right object. – Martijn Pieters Oct 17 '14 at 21:54
  • 1
    I double checked and you seem to be right, I put the full path to the object for patching, I think the problem might be because the function I'm trying to run doesn't have listdir as an attribute? It imports it from os, not sure what the protocol would be in this case, I tried mocking os.listdir and that didnt seem to work either. – openingceremony Oct 17 '14 at 22:37
  • @openingceremony: If you are using `from os import listdir` then you need to patch `yourmodulename.listdir` as it'll be a global there. If you are testing *in the same file* then the module is called `__main__` however and you'd patch `__main__.listdir`. – Martijn Pieters Oct 17 '14 at 22:38
  • Ah, I see. I'm using from os.. so that worked. One last question haha, I need the files that I am going over, file1 and file2 to have certain data within it to test properly, can I mock what's in the files themselves? – openingceremony Oct 17 '14 at 22:49
  • @openingceremony: see [mock open for unit testing in python](http://stackoverflow.com/a/25916564) – Martijn Pieters Oct 17 '14 at 22:59
  • I wanted to change the working directory to one level up, so I set the return value to os.listdir('./../') rather than a hard-coded list like in your code. However, calling os.listdir() returned the mock object rather than calling os.listdir('./../'). I don't understand why. Calling patch(target='os.listdir', new=os.listdir(./../')) did the trick. No need to use a with statement. – paperduck Aug 27 '17 at 10:09