Forwarding messages with Python script and netcat

Question

I am trying to forward a stream of terminal output (in terminal T1) to a localhost port, listened to on terminal T2. Then, using a simple Python socket script to capture that streamed data and print it out. I referenced to this post for the ubuntu command and this post for the Python socket script:

Situation 1: In terminal T2, I run this first: to listen to port 2003

nc -l 2003

In terminal T1, I then run the streaming command (Kafka). The kafka server is on port 9092:

.bin/kafka-console-consumer --topic <topic_name> --from-beginning --bootstrap-server <dns:port> | pv | netcat localhost 2003

In this situation, I can see the streamed data is displayed on T2. Hence, the stream has been sent from Kafka terminal to another port, listened to by another process.

Situation 2: Here is Python code I run on terminal T3

import socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('localhost', 2003))
while True:
    data = client_socket.recv(512)
    if len(data) > 0:
        print("RECEIVED: %s" % data)

The sequence is: I run T2, the Python code in T3, then Kafka command in T1. This time, I don't see any messages in either T3 or T2.

The intention is to capture the data streamed by Kafka with Python script via an intermediary forwarder. Is this the right way or is there any way to do this?

your python code works as client and it connects to `nc -l 2003` which works as server but `nc -l 2003` would have to send something to clients to see something in Python terminla - but normally it sends nothing to clients. You should rather use `socket.listen()` and `socket.bind()` to run Python as a sever instead of `nc -l 2003` — furas, Jun 18 '21 at 03:00

furas · Accepted Answer · 2021-06-18T04:18:48.140

Your Kafka and Python work as a clients and nc -l 2003 as a server.

There are three problems.

This server may receive values only from one client.
Python wait for data from server but this server sends nothing.
This server doesn't have function to send from one client to other client.

You should rather use Python to create server and use it instead of nc

Here is simple server which you can use instead of nc -l 2003

import socket

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 2003))
server.listen(1)

print('WAITING FOR CLIENT')
s, addr = server.accept()
print('CONNECTED:', addr)

try:

    while True:
        data = s.recv(512)
        if not data:
            break
        print("RECEIVED:", data.decode())

except KeyboardInterrupt:
    print('STOPPED by CTRL+C')
finally:    
    s.close()
    server.close()

Forwarding messages with Python script and netcat

1 Answers1