Starting a Tmux session from Github Action doesn't persist once the action completes

Question

I have a self-hosted Github Action with the main function of running a bash script to deploy a Django site. I cd into the directory, and then execute the bash script. This works fine, and I can see the print statements (DEPLOY_FOLDER AND CONTAINER_NAME) from the bash script in the Github Action GUI.

name: Deploy Workflow

on:
  workflow_dispatch:

jobs:
  deploy:
    runs-on: [self-hosted, emts]
    steps:
    - name: CD and then deploy
      shell: bash
      run: |
        cd /export/home/scottc/
        ./deployserver.sh test test_deploy

HOWEVER, within my deployserver.sh script, I am starting a (detached) TMUX new session. I receive a notice that my job completed successfully, but when I check tmux sessions (tmux ls) after the job completes, I don't see any running.

#!/bin/bash

echo $1
echo $2
DEPLOY_FOLDER=$1
CONTAINER_NAME=$2

tmux new -d -s $CONTAINER_NAME -- pipenv run python server_web/run_server.py

I have updated the deployserver.sh script to the below and found that a tmux session is actually being created, but must somehow get torn down when the job completes. I know this because after printing the DEPLOY_FOLDER and CONTAINER_NAME, I see tmux ls prints test_deploy: 1 windows (created Thu May 18 11:00:49 2023) [80x24] (while the job is still active but sleeping for 10 seconds). If I run tmux ls on the server during those 10 seconds, I do see an active window. However when the 10 seconds completes and the Github Action ends, the tmux session dissapears with it.

#!/bin/bash

echo $1
echo $2
DEPLOY_FOLDER=$1
CONTAINER_NAME=$2

tmux new -d -s $CONTAINER_NAME -- pipenv run python server_web/run_server.py

sleep 3

tmux ls

sleep 10

What do I need to change for the tmux session to persist after the github action completes?

@pynexj what would this solve? I already know that it persists with `sleep 10` as I mentioned. And I don't want to the github job to be running for 9999 seconds, that defeats the purpose of using tmux. And I need it to work with the pipenv command as that is the most important part of the whole script, so switching to sleep doesn't fix it. — Scott C., May 19 '23 at 12:55

score 1 · Answer 1 · answered May 18 '23 at 16:07

1

By default, a shell session only lasts as long as it has an active window, a window only remains active as long as it has an active pane, and a pane only remains active until the command it is executing exits. When run_server.py exits, the initial pane is destroyed, taking the initial window with it and killing the session.

One way to keep it alive is to set the remain-on-exit option on the pane. (With no -t option, set-option affects the currently active pane. With only one server, one session, one window, and one pane, that happens to be the one running your command.)

tmux new -d ...
tmux set-option -p remain-on-exit

answered May 18 '23 at 16:07

chepner

497,756
71
530
681

I should have mentioned that run_server.py isn't supposed to ever "exit" or "complete". It stays running indefinitely (until server crashes or is cancelled manually). Hence the reason to have it running in a tmux windows so that once the github job completes, the tmux windows stays running to serve the webpage. – Scott C. May 18 '23 at 16:21
If I run the same `deployserver.sh` script manually (not with github action), then the detached tmux shell opens, and stays running in the background. It's only when launching the same script from a github action that the window is destroyed when the job completes (i.e. `tmux new ...` command runs, not the `run_server.py` "finishing"). When I try adding the `set-option...` line, I receive `tmux: unknown option -- p usage: set-option [-aFgosquw] [-t target-window] option [value]`. I tried `tmux set-option remain-on-exit` but still having the same issue. – Scott C. May 18 '23 at 16:21

score 0 · Answer 2 · answered May 23 '23 at 18:43

Solution: Instead of running ./deployserver.sh test test_deploy use RUNNER_TRACKING_ID="" && (./deploysensei.sh test test_deploy)

The problem was evident in the "Complete Job" logs on Github:

Cleaning up orphan processes
Terminate orphan process: pid (1050875) (tmux: server)
Terminate orphan process: pid (1050876) (bash)
Terminate orphan process: pid (1051071) (pipenv)
Terminate orphan process: pid (1051075) (python)

I can see that the action was terminating orphan processes, including tmux. When you reassign the Github env variable RUNNER_TRACKING_ID, then during the teardown process it doesn't terminate the existing processes.

More details here.

Starting a Tmux session from Github Action doesn't persist once the action completes

2 Answers2