Python 'socketserver' only ever handles the first request

Question

Client Output

To show the problem, here is the code in action when running on the client. As can be seen the first command succeeds (it always does) and it succeeds even in case of a faulty command by which it will return stderr.

Pwny ~>df
Filesystem     1K-blocks     Used Available Use% Mounted on
udev             1895788        0   1895788   0% /dev
tmpfs             386392     1416    384976   1% /run
/dev/sda1      478612200 43470808 410755756  10% /
tmpfs            1931948    78200   1853748   5% /dev/shm
tmpfs               5120        0      5120   0% /run/lock
tmpfs             386388       60    386328   1% /run/user/1000

Pwny ~>pwd

Pwny ~>pwd
[~] Connection aborted
Pwny ~>

Server output

Here is the output from my server.

└─$ /bin/python3 /home/user/Desktop/tcpserver/socketserver.py
[*] Started server 0.0.0.0:9999
[*] 192.168.137.1:1051 connected
[~] 192.168.137.1:1051 df

TCP Client Program

This is my client code:

import socket

class TCPClient:
    @staticmethod
    def run(host, port, buffer_size=1024, encoding='utf-8'):
        HOST = host
        PORT = port
        BUFFER_SIZE = buffer_size
        ENCODING = encoding
        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
            s.connect((HOST, PORT))
            while True:
                tx: bytes = bytes(input("Pwny ~>"), ENCODING)
                tx += b"\n"
                if tx == b"exit":
                    break
                else:
                    try:
                        s.sendall(tx)
                        full_msg: bytes = b''
                        while True:
                            msg: bytes = s.recv(BUFFER_SIZE)
                            if len(msg) == 0:
                                break
                            full_msg += msg
                        print(full_msg.decode(ENCODING))
                    except ConnectionAbortedError:
                        print("[~] Connection aborted")
                    except OSError:
                        print("[~] An OS error occurred")


if __name__ == '__main__':
    tcp_client = TCPClient()
    tcp_client.run(host='192.168.137.2', port=9999, buffer_size=128, encoding='utf-8')

TCP SocketServer Program

And this is the socketserver program (I suspect I'm doing something here, help greatly appreciated)

import socketserver
import subprocess
from subprocess import CompletedProcess

class ThreadingServer(socketserver.ThreadingMixIn,socketserver.TCPServer): pass

class TCPRequestHandler(socketserver.StreamRequestHandler):
    def setup(self) -> None:
        return super().setup()

    def handle(self) -> None:
        ENCODING: str = 'utf-8'
        BUFFER_SIZE: int = 128
        client_address: str = self.request.getpeername()[0]
        client_port: int = self.request.getpeername()[1]

        print(f"[*] {client_address}:{client_port} connected")

        client_cmd: str = self.rfile.readline().decode(ENCODING).strip()
        print(f"[~] {client_address}:{client_port} {client_cmd}")
        output: CompletedProcess = subprocess.run(client_cmd,shell=True,capture_output=True)
        """ Returns 0 for success and >= 1 for failure"""
        if output.returncode == 0: # success
            self.wfile.write(output.stdout)
        else: # failure when > 0
            self.wfile.write(output.stderr)

    def finish(self) -> None:
        return super().finish()

if __name__ == '__main__':
    with ThreadingServer(('0.0.0.0',9999),TCPRequestHandler) as server:
        print(f"[*] Started server {server.server_address[0]}:{server.server_address[1]}")
        server.serve_forever()

Answer 1

You have issues in both the client and server.

The client uses socket .recv() to read data but this will block forever at the end of the first command when the server output finishes. it will return b'' only at EOF, when the socket is closed by the server. Because of the server issue (below), it SEEMS to work.

Your server code only reads one line, spawns the command, sends the output and then returns out of handle() after which the connection is closed. Because of this close, the client actually works (it gets b'' from recv()).

To fix the server, do readline() in a loop until it returns b'' (socket closed by client) and preferably flush() the write stream after sending the command output.

Example server handler:

    def handle(self) -> None:
        ENCODING: str = 'utf-8'
        BUFFER_SIZE: int = 128
        client_address: str = self.request.getpeername()[0]
        client_port: int = self.request.getpeername()[1]

        print(f"[*] {client_address}:{client_port} connected")

        while True:
            line: str = self.rfile.readline()
            if line == b'':
                break
            client_cmd: str = line.decode(ENCODING).strip()
            print(f"[~] {client_address}:{client_port} {client_cmd}")
            output: CompletedProcess = subprocess.run(client_cmd,shell=True,capture_output=True)
            """ Returns 0 for success and >= 1 for failure"""
            if output.returncode == 0: # success
                self.wfile.write(output.stdout)
            else: # failure when > 0
                self.wfile.write(output.stderr)
            self.wfile.flush()

To fix the client and the whole setup, you probably need to implement some kind of framing, for the client commands this is currently a newline (because you use readline) but for the server output you need to figure out a separate framing.