subprocess.Popen simple code does not allow me to perform a cd (change directory)

Question

I'm trying to use a Python script to change directory, but I am getting an error.

The python code:

import subprocess
p = subprocess.Popen(['cd', '~'], stdout=subprocess.PIPE)
output = p.communicate()
print output

I get this error:

File "test_sub.py", line 2, in <module>
p = subprocess.Popen(['cd', '~'], stdout=subprocess.PIPE)
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

What does the error mean, what am I doing wrong, and how do I change directory in a python subprocess?

@Ashalynd This error message is a generic error message in Python2 not specifying that `cd` does not exists. — Torxed, Jun 11 '14 at 11:20

Torxed · Accepted Answer · 2021-01-28T13:01:52.977

>>> Popen('cd ~', shell=True, stdout=PIPE).communicate()
(b'', None)

Without shell=True (which, runs the command in shell, on POSIX that defaults to /bin/sh)

>>> Popen(['cd', '~'], stdout=PIPE).communicate()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/subprocess.py", line 858, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.4/subprocess.py", line 1456, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'cd'
>>>

You can't change directory unless you do it via:

import os
os.chdir(os.path.abspath(os.path.expanduser('~')))

So the problem isn't that the path ~ doesn't exist, but rather cd doesn't exist as an option unless your command is run in a shell that supports it. Passing directly to an actual shell makes cd work. But note that shell=True is a risk, never use it unless you need to..
So use os.chdir instead.

A working scenario:

import os, subprocess
os.chdir(os.path.abspath('/tmp/'))
print(subprocess.Popen(['ls', '-lah'], stdout=subprocess.PIPE).communicate()[0].decode('utf-8'))

Resulting in:

[torxed@archie ~]$ python
Python 3.4.1 (default, May 19 2014, 17:23:49) 

>>> import os, subprocess
>>> os.chdir(os.path.abspath('/tmp/'))
>>> print(subprocess.Popen(['ls', '-lah'], stdout=subprocess.PIPE).communicate()[0].decode('utf-8'))

total 12K
drwxrwxrwt  9 root   root   220 Jun 11 12:08 .
drwxr-xr-x 19 root   root  4.0K May 28 08:03 ..
drwxrwxrwt  2 root   root    40 Jun 11 09:30 .font-unix
drwx------  2 torxed users   60 Jun 11 09:33 gpg-LBLcdd
drwxrwxrwt  2 root   root    40 Jun 11 09:30 .ICE-unix
drwx------  2 torxed users   80 Jun 11 09:34 .org.chromium.Chromium.LEqfXB
-rw-------  1 torxed users  153 Jun 11 09:34 serverauth.EHWB0LqCv6
drwxrwxrwt  2 root   root    40 Jun 11 09:30 .Test-unix
-r--r--r--  1 root   users   11 Jun 11 09:34 .X0-lock
drwxrwxrwt  2 root   root    60 Jun 11 09:34 .X11-unix
drwxrwxrwt  2 root   root    40 Jun 11 09:30 .XIM-unix

>>>

Note that i started the shell in ~ and via os.chdir changed it to tmp and actually got my tmp directory content.

Explanation of shells and commands:

A shell-command is something that's built into the shell while a regular old command is something you'll find under /bin, for instance:

[torxed@archie ~]$ ls /bin
2to3            2to3-2.7
7z              7za
...

Where 7z is a command i can actually execute:

>>> from subprocess import *
>>> Popen(['7z'], stdout=PIPE).communicate()

(b'\n7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18\np7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)\n

While for instance cd is a built in shell command, something that you will not find under /bin but works anyway in most "terminals" (using a shell) because it's (as mentioned), built into the shell you normally see.

But because Python will by default not execute the command in a shell you more or less have to rely on using os.chdir(...) or wrap your command in /bin/sh -c "cd ..." or something similar.

I really don't know the meaning of shell=True but if I add that the error is gone, but I can't see any output, even if I change the 'cd' for a simple 'ls'. I need to use ['command1', 'command2'] because the commands come from variables. Also, of course, I need to see the result of using those commands on the python program. Thanks and sorry, I'm just a beginner. — , Jun 11 '14 at 11:31
@Haddex `cd` doesn't have any output, it just moves you from one location to another. If you're expecting to see `[user@machine ~]` you've missunderstood how shells work. Also `cd` is a shell-command, not a Linux command (a linux "command" is anything located under `/bin` while a shell command is something that's built into the shell). — Torxed, Jun 11 '14 at 11:33
@Haddex hopefully my edit will shed some light on what `cd` is and how you know which commands you can execute by using `ls /bin` to find out. — Torxed, Jun 11 '14 at 11:40
you could pass `cwd` parameter to subprocess instead of `os.chdir()` in the parent. — jfs, Oct 06 '15 at 17:56
This answer is incorrect. `cd` works fine launching a shell from python. The problem is that it changes the directory in the shell itself, not in the parent process, so it doesn't work for changing directory in the calling process. — LtWorf, Jan 25 '21 at 11:22
@LtWorf I mean yea, but the `cd` command is not accessible "as is", as it's part of the subshell. So you need to execute the subprocess call with `shell=True` otherwise no shell will be spawned and subsequently the `cd` command doesn't exist (as it's not a binary in PATH). Two options are to either chdir from python via `os.chdir()` or call your subprocess with a shell. Am I missing something that you desribed? — Torxed, Jan 25 '21 at 12:53
@LtWorf Not sure why the down vote in that case :) And why you deem this answer incorrect - it describes the same thing I just commented? :P — Torxed, Jan 28 '21 at 08:43
"But because Python will emulate a shell "… that's incorrect, had to vote down… — LtWorf, Jan 28 '21 at 10:14
I see you edited your answer to correct it. I don't think insulting me was a necessary part of the process. @Torxed — LtWorf, Jan 29 '21 at 10:42

score 1 · Answer 2 · answered Jun 11 '14 at 11:24

1

cd is a builtin command of the shell which change the environment of the shell to set the current directory of next commands will be run into. It is not a regular program. So it can't be called as a subprocess with Popen.

The right way to change the current directory inside python is:

import os
os.chdir(os.path.abspath(os.path.expanduser('~')))
#now the current directory is home of user

answered Jun 11 '14 at 11:24

Xavier Combelle

10,968
5
28
52

@Torxed sorry but [the version](http://stackoverflow.com/revisions/24161514/1) I saw was quite mysterious about why it did not work – Xavier Combelle Jun 11 '14 at 11:57
Fair enough :) I can be a little "round" in my language at first before i hone in on what i really meant to say :) – Torxed Jun 11 '14 at 12:29
Well it "can" be called by giving `shell=True`, but it will only change the state in the shell itself. – LtWorf Jan 25 '21 at 11:23

score -1 · Answer 3 · edited Jun 20 '18 at 09:04

-1

You need to include all path with

path = os.path.dirname('$0')
currentpath = os.path.abspath(path)
os.chdir(path)

edited Jun 20 '18 at 09:04

greg-449

109,219
232
102
145

answered Jun 20 '18 at 07:20

Isabel Cenamor

9
1

(This post does not seem to provide a [quality answer](https://stackoverflow.com/help/how-to-answer) to the question. Please either edit your answer, or just post it as a comment to the question). – sɐunıɔןɐqɐp Jun 20 '18 at 07:23

subprocess.Popen simple code does not allow me to perform a cd (change directory)

3 Answers3

Explanation of shells and commands:

Linked

Related