python subprocess Popen behaving differently on local and remote system

Question

I am using django and sending a build to travis-ci which is failing the build on my tests. The line that is in question is in this function...

from subprocess import Popen, PIPE

def make_mp3(path, blob):
    process = Popen(
        ['lame', '-', '--comp', '40', path], stdin=PIPE, stdout=PIPE, stderr=PIPE)
    stdout_data = process.communicate(input=blob.read())
    return stdout_data

the line that calls the function is here:

make_mp3(mp3_path, request.FILES['audio_file'])

on my local system the tests run fine and pass. I have been using this function for a while and it always behaves as expected. But when I send it to travis-ci for a build it gives me this traceback...

======================================================================
ERROR: test_post_upload_audio_word (define.tests.ViewTests)
Testing the audio upload function for word audio
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/deltaskelta/langalang/define/tests.py", line 827, in test_post_upload_audio_word
    'audio_file': audio_file})
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/test/client.py", line 541, in post
    secure=secure, **extra)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/test/client.py", line 343, in post
    secure=secure, **extra)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/test/client.py", line 409, in generic
    return self.request(**r)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/test/client.py", line 494, in request
    six.reraise(*exc_info)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
    return bound_func(*args, **kwargs)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
    return func.__get__(self, type(self))(*args2, **kwargs2)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
    return bound_func(*args, **kwargs)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
    return func.__get__(self, type(self))(*args2, **kwargs2)
  File "/home/travis/build/deltaskelta/langalang/define/views.py", line 21, in dispatch
    return super(Define, self).dispatch(*args, **kwargs)
  File "/home/travis/virtualenv/python2.7.12/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/travis/build/deltaskelta/langalang/define/views.py", line 89, in post
    return upload_audio(request, context)
  File "/home/travis/build/deltaskelta/langalang/define/views_func.py", line 251, in upload_audio
    make_mp3(mp3_path, request.FILES['audio_file'])
  File "/home/travis/build/deltaskelta/langalang/define/views_func.py", line 25, in make_mp3
    ['lame', '-', '--comp', '40', path], stdin=PIPE, stdout=PIPE, stderr=PIPE)
  File "/opt/python/2.7.12/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/opt/python/2.7.12/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

I cannot see why this would pass the test on my system and fail on the build. The input file is obviously there because request.FILES found it. The directory exists..I checked. So what is it telling me doesn't exist?

What if you execute it manually on production (if you can)? Thing is that this exception can refer not to path parameter, but to command you execute — valignatev, Sep 07 '16 at 07:49
You posted this right as I figured it out. I forgot to install lame in the build environment. How did you know that could also be the problem? I spend hours going over the same lines again and again checking paths because I thought that is what it was telling me — Joff, Sep 07 '16 at 07:52

score 2 · Accepted Answer · answered Sep 07 '16 at 08:02

Reason it fails: execution command can't execute non-existent file. It's lame in your example. Quote from doc section:

The most common exception raised is OSError. This occurs, for example, when trying to execute a non-existent file. Applications should prepare for OSError exceptions.

You also could check source code for _execute_child method in subprocess.py near the end.

python subprocess Popen behaving differently on local and remote system

1 Answers1