0

I'm running a python script where I'm using subprocess to call serious of rclone copy operations. Since rclone isn't a native command, I'm defining it from a shell script run automatically by my .bashrc file. I can confirm that works since subprocess.run("rclone") properly pulls the rclone menu.

The issue is when I run my script, I don't get any errors or exceptions. Instead my terminal window shows the following: enter image description here

I understand the issue is related to the Linux subprocess being backgrounded. However, this solution didn't seem to fix my issue, and I can't find anything about how to prevent this process from pausing. I can confirm it is distro independent as I have run on RedHat and on Amazon EC2.

Last key piece of info: I am calling the subprocess as bash rather than sh to load the alias via my bashrc file. Here is the minimum reproducible code:

start_date = datetime.strptime(datetime.now(timezone.utc).strftime("%Y%m%d"), "%Y%m%d")
# For good measure, double check the day before for more files if the date just changed
time = datetime.utcnow().strftime("%H")
if int(time) <= 3:
    start_date = start_date - timedelta(days = 1)
    end_date = start_date + timedelta(days = 2)

else:
    # End tomorrow
    end_date = start_date + timedelta(days = 1)

# Force python to use the bash shell
def bash_command(cmd):
    subprocess.Popen(['/bin/bash', '-i', '-c', cmd])

for dt in daterange(start_date, end_date):
    cmd = 'rclone copy "/home/test.png" "AWS test:"'
    bash_command(cmd)
Sammy Wammy
  • 55
  • 6
  • If `subprocess.run("rclone")` successfully runs the `rclone` command you're trying to run (albeit not with command-line arguments), then why is it, again, that in the failure case you're running it via `bash -i -c`? What exactly is it that you are relying on your `.bashrc` to put in the environment, and what prevents you from using `os.putenv()` to put it there from the python side? – John Bollinger Nov 05 '20 at 00:27
  • @JohnBollinger the python configuration uses an sh shell by default, and rclone is technically an alias for running an exec. So in order to use the alias, the Python subprocess needs to call via bash, and also needs to do so through the .bashrc file since that is where it is defined. Thus the bash -i -c. Though I admit, os.putenv() could work, but I was still getting command "rclone" not found when I tried it that way – Sammy Wammy Nov 06 '20 at 01:05
  • https://meta.stackoverflow.com/questions/285551/why-not-upload-images-of-code-errors-when-asking-a-question – Nic3500 Nov 06 '20 at 03:24
  • Sorry, @SammyWammy, but that's not adding up. `subprocess.run()` defaults to `shell=False`, so it runs the command directly, or at least attempts to do. And if it did run the command via `/bin/sh` then your `.bashrc` would not be involved, because bash does not read that file when invoked as `/bin/sh`. And certainly none of the *other* shells that might be `/bin/sh` on your system do so. – John Bollinger Nov 06 '20 at 12:48

0 Answers0