running server in tmux through ansible

Question

(forgive me for spamming, I should have adjusted the original question, but it felt like I ran into a new hurdle when solving the problem)

I set a goal of running a web server with ansible: figured I'd sit it inside a tmux session. I quickly ran into ansible play hanging forever due to tmux dropping into it's own shell, but with community's help got halfway to the goal. I can run my server, but there's no tmux session to be found on the box.

the playbook's task is this:

tasks:
    - name: drop into tmux and run gunicorn
      shell:
        cmd: tmux has-session -t api || tmux new-session -d -s api /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/gunicorn backend.wsgi:application -b 127.0.0.1:4000
        chdir: htmshop_parent

, basically, if the session 'api' doesn't exist, launch a gunicorn (from a virtualenv) within a session, and detach.

the play reported certain success:

TASK [drop into tmux and run gunicorn]
*****************************************
    changed: [44.197.228.14] => {"changed": true, "cmd": "tmux has-session -t api || tmux new-session -d -s api /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/gunicorn backend.wsgi:application -b 127.0.0.1:4000", "delta": "0:00:00.012161", "end": "2021-08-25 14:47:14.284544", "rc": 0, "start": "2021-08-25 14:47:14.272383", "stderr": "no server running on /tmp/tmux-0/default", "stderr_lines": ["no server running on /tmp/tmux-0/default"], "stdout": "", "stdout_lines": []}

tmux reporting no server is running error, is what has-session is expected to do; then the second clause of || kicks in... at least it does when I run this command manually on the box. But playbook surprises me; the process (gunicorn) is indeed launched (why two? beats me!):

> pgrep -a guni
    6542 /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/python /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/gunicorn backend.wsgi:application -b 127.0.0.1:4000
    6548 /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/python /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/gunicorn backend.wsgi:application -b 127.0.0.1:4000

but tmux ls tells me no server running on /tmp/tmux-1001/default

Interestingly enough, if I take a look at processes like this:

± |staging ?:13 ✗| → ps auxw | grep guni
root        6541  0.0  0.3   7536  3616 ?        Ss   15:14   0:00 tmux new-session -d -s api /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/gunicorn backend.wsgi:application -b 127.0.0.1:4000
root        6542  0.0  2.2  30352 22340 pts/2    Ss+  15:14   0:00 /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/python /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/gunicorn backend.wsgi:application -b 127.0.0.1:4000
root        6548  0.0  4.1  52532 41984 pts/2    S+   15:14   0:00 /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/python /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/gunicorn backend.wsgi:application -b 127.0.0.1:4000

, I do see the tmux (parent process) also... but where's the session? And in what magical subshell-like thing was this pony started, if not in a tmux session? Or perhaps it is a tmux session of some sort, just not one that my tmux knows about? ¯_(ツ)_/¯

Thank y'all for your time...

Hm. The session is just an internal data structure in the server. `tmux` should have exited immediately; I'm not sure why it didn't. The server is probably running on a different socket than what your manual `tmux` tries to connect to; was it started by a different user? Check for any other `/tmp/tmux-*` directories. — chepner, Aug 25 '21 at 16:24
For a reproducible deployment, you might want to specify the socket to use explicitly, using `-L` or `-S` options to `tmux`, or the `TMUX_TMPDIR` environment variable. — chepner, Aug 25 '21 at 16:27
`gunicorn` is running using the pseudoterminal `pts/2`, which is managed by the `tmux` server (as part of the `api` session), rather than being managed by your terminal emulator. — chepner, Aug 25 '21 at 16:28
thanks for keeping working with me mate! your comments though - do they pertain to me running the command manually on the box (which does create a tmux session) or when it's executed through ansible (which doesn't create a session, but does run _two_ gunicorns lol) — alexakarpov, Aug 25 '21 at 16:35
not sure I understand why you call sessions "internal data structure". All tmux cheatsheets and docs I skimmed through talk about them as first-class citizens; attaching to them, and detaching, seems to be the main thing users are expected to do... — alexakarpov, Aug 25 '21 at 16:38
When you run `tmux foo ...`, you are just sending a command to the server to execute. Part of that might involve some other "local" work, for example, `tmux attach-session ...` creates a client that attaches to a specific session managed by the server. — chepner, Aug 25 '21 at 16:43
A session is a collection of pseudoterminals. Access to those pseudoterminals is mediated via the windows (and its constituent panes) linked to the session. The session has a concept of the *active* window, which is what your client first displays when attaching to the session. — chepner, Aug 25 '21 at 16:45

score 3 · Accepted Answer · answered Aug 25 '21 at 17:03

Not really an answer, but fleshing out some background as to how tmux works.

When you run tmux, it tries to connect to (or create if necessary) a server running on a particular Unix socket. By default, the path to that socket is something like /tmp/tmux-$USERID/default. You can change the directory using the -L option or the TMUX_TMPDIR environment variable. You can ignore both those using -S to specify an exact path your self.

For example,

$ tmux -L ~/foo ...   # Talks to a server via ~/foo/default
$ tmux -S ~/foo/bar   # Talks to a server via ~/foo/bar

Your playbook is probably executed under a different user ID than your interactive shell, so tmux is attempting to communicate with two different servers in that case. I would recommend being explicit in your playbook so that there is no doubt about which server Ansible will create/talk to. Something like

- name: Gunicorn in tmux
  environment:
    TMUX_TMPDIR: /tmp/my/ansible/tmux/session
  tasks:
    - name: drop into tmux and run gunicorn
      shell:
        cmd: tmux has-session -t api || tmux new-session -d -s api /home/akarpov/.local/share/virtualenvs/htmshop_parent-NXHiij2E/bin/gunicorn backend.wsgi:application -b 127.0.0.1:4000
        chdir: htmshop_parent

I'm somewhat surprised to see that tmux new-session -d is still running, as it should have exited immediately after creating the session (or rather, asking the server to create the session). The gunicornprocesses are indeed running in this session. When a session is created, a new window for the session is created as well, and according to your output, this window is associated with the pseudoterminal/dev/pts/2`.

If you were to later try to attach to this session, the command tmux attach-session -t api would send a request to the server, asking it (very roughly speaking) which pseudoterminal is associated with the currently active window of the named session. The tmux client will then communicate with that pseudoterminal, providing a kind of thin terminal emulation.

So aside from tmux new-session ... continuing to run in the background, everything looks like it should.

got to say, you put much more emphasis on what tmux _really_ is than most, if not all, online search results. guess they hide the real things (server and windows) behind sessions, making folks like me to think they're first class... while they're a user interface artifact, eh — alexakarpov, Aug 25 '21 at 17:24
"Your playbook is probably executed under a different user ID than your interactive shell, so tmux is attempting to communicate with two different servers in that case." - oh man, that's exactly it. what a noob mistake; playbook is indeed ran as root - and dropping to root on the box I see my session, no magic, just bloody linux basics... — alexakarpov, Aug 25 '21 at 17:28

running server in tmux through ansible

1 Answers1

Linked