Unicode error for python echo server when "Ctrl+C" pressed

Question

I have created python echo server. The echo server can be started using command

sudo python3.8 echo_server.py

The client can be connected to it using command

telnet localhost 5000

I have implemented some commands like time,exit and network (i.e. if you type 'time' command on telnet client, the server responds with system time.)

For "network" command contain 3 while loops (i.e 1 loop for selecting network type, 2 loop for selecting static\dynamic, 3 loop for configuring ssid/password for wifi settings). Now while configuring ssid/pwd, I wanted to go back to main command prompt for telnet client (i.e. where we can enter time,network command) on press of "Ctrl+c". But I am not able to handle the "Ctrl+c" on the echo server.I am getting below exception when "Ctrl+c" is pressed on telnet client

Exception in thread MyThread:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "echo_server.py", line 47, in run
    tx_data = ""
  File "/usr/lib/python3.8/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
Main Terminating...

Please find my echo_server.py file

import socket
import os
import os.path
import subprocess
import time
from subprocess import Popen, PIPE, STDOUT
from threading import Thread
from time import sleep

class MyThread(Thread):
 def __init__(self, val):
        ''' Constructor. '''
        Thread.__init__(self)
        self.val = val

 class Filter:
        """Substitute \n with \r\n in stream output to terminal."""

        def __init__(self, file):
            self.file = file

        def write(self, data):
            self.file.write(data.replace("\n", "\r\n"))

 def run(self):

  while True:
    HOST = ''                 # Symbolic name meaning all available interfaces
    PORT = 5000              # Arbitrary non-privileged port
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    try:
        s.bind((HOST, PORT))
        s.listen(5)
    except socket.error:
        time.sleep(5)
        continue
    conn, addr = s.accept()
    file = conn.makefile(mode="rw")
    filtered_file =MyThread.Filter(file)
    file.flush()
    filtered_file.write("Enter 'help' to list all valid commands\n")
    while True:
        file.write("Command> ")
        file.flush()
        data = file.readline()
        tx_data = ""
        if not data: break
        data = data.strip()
        if not data:
            continue
        elif data == 'time':
           f= os.popen('date')
           date=f.read().strip()
           tx_data = date + '\n'

        elif data == "network":

            while True:
                file.write("1. WiFi\n")
                file.flush()                
                file.write("2. Ethenet\n")
                file.flush()
                file.write("3. Exi\n")
                file.flush()
                file.write("Enter a choice:")
                file.flush()
                choice = file.readline()
                choice = choice.strip()
                if choice == "1":
                    while True:
                         file.write("1. DHCP\n")
                         file.flush()                
                         file.write("2. Static\n")
                         file.flush()
                         file.write("3. Exit\n")
                         file.flush()
                         file.write("Enter a choice:")
                         file.flush()
                         subchoice = file.readline()
                         subchoice = choice.strip()
                         if subchoice == "1":
                             while True:
                                 file.write("Enter ssid:")
                                 file.flush()
                                 ssid = file.readline()
                                 ssid = ssid.strip()
                                 file.write("Enter pwd:")
                                 file.flush()
                                 pwd = file.readline()
                                 pwd = pwd.strip()
                                 break
                         break
                elif choice == "2":
                    break
                elif choice == "3":
                     break
                else:
                     break

        elif data == 'help':
            tx_data = '''Valid commands are as below:
Enter number against the command for execution

        1. time
        2. network
        3. exit\n'''

        elif data == 'exit':
            break

        else:
            tx_data = "Unknown Command: " + data + \
                    "\nEnter 'help' for list of valid commands\n"

        filtered_file.write(tx_data)
    #print 'Closing connection with client'
    file.close()
    conn.close()


# Run following code when the program starts
if __name__ == '__main__':
   # Declare objects of My Thread class
   my_obj = MyThread(4)
   my_obj.setName('MyThread')


   # Start running the threads!
   my_obj.start()

   # Wait for the threads to finish...
   my_obj.join()

   print('Main Terminating...')

I am not sure how to handle "Ctrl+c" so that control is back at the prompt when we can enter command. Please let me know if any one has any suggestion to resolve this.

Looks like a UTF-16 [Byte order mark](https://en.wikipedia.org/wiki/Byte_order_mark) intruded into _something_? I can't reproduce this: Ctrl+C inserts `U+0003` _End Of Text_ control character in telnet client input… — JosefZ, Jun 09 '20 at 19:24
@JosefZ I am able to reproduce the issue. Are you using python2 for running the echo_server.py. I am using python3.8 for running echo_server.py — cgoma, Jun 10 '20 at 07:47
I'm running `echo_server.py` in Python 3.5. However, trying it, I disarranged `telnet` input mistakenly so that `telnet` session has stopped co-operating with server (or vice versa?). I'm not ready to continue testing this for a while, sorry. — JosefZ, Jun 10 '20 at 09:18

score 0 · Answer 1 · answered Jun 12 '20 at 17:28

Here is my version of your program:

# echo_server.py
import socket
import os
import os.path
import subprocess
import time
from subprocess import Popen, PIPE, STDOUT
from threading import Thread
from time import sleep

def getcommand(thisfile, thisprompt, stripped=True):
    thisfile.write(thisprompt)
    thisfile.flush()
    command = thisfile.readline()[:-1]
    # print(f"{len(command)} chars received")  # just for debug
    # for i in command:
    #    print(ord(i))
    if stripped:
        return command.strip()
    return command


class MyThread(Thread):
  def __init__(self, val):
        ''' Constructor. '''
        Thread.__init__(self)
        self.val = val

  class Filter:
        """Substitute \n with \r\n in stream output to terminal."""

        def __init__(self, file):
            self.file = file

        def write(self, data):
            self.file.write(data.replace("\n", "\r\n"))

  def run(self):

   while True:
      HOST = ''                 # Symbolic name meaning all available interfaces
      PORT = 5000              # Arbitrary non-privileged port
      s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
      s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
      try:
        s.bind((HOST, PORT))
        s.listen(5)
      except socket.error:
        time.sleep(5)
        continue
      conn, addr = s.accept()
      file = conn.makefile(mode="rw", encoding="UTF-8", errors="ignore")
      filtered_file = MyThread.Filter(file)
      file.flush()
      filtered_file.write("Enter 'help' to list all valid commands\n")
      while True:
        data = getcommand(file, "Command> ", stripped=False)
        tx_data = ""
        if not data:
            print("No Command!")
            break  # closing connection
        data = data.strip()
        if not data:
            print("Just Spaces!")
            continue

        elif data == 'time':
           f= os.popen('date')
           date=f.read().strip()
           tx_data = date + '\n'

        elif data == "network":
            while True:
                file.write("1. WiFi\n")
                file.flush()                
                file.write("2. Ethernet\n")
                file.flush()
                file.write("3. Exit\n")
                file.flush()
                choice = getcommand(file, "Enter a choice:")
                if choice == "1":
                    while True:
                         file.write("1. DHCP\n")
                         file.flush()                
                         file.write("2. Static\n")
                         file.flush()
                         file.write("3. Exit\n")
                         file.flush()
                         subchoice = getcommand(file, "Enter a choice:")
                         if subchoice == "1":
                             while True:
                                 ssid = getcommand(file, "Enter ssid:")
                                 pwd = getcommand(file, "Enter password:")
                                 if ssid and pwd:
                                     break
                         break
                elif choice == "2":
                    break
                elif choice == "3":
                     break

        elif data == 'help':
            tx_data = '''Valid commands are as below:
Enter number against the command for execution

        1. time
        2. network
        3. exit\n'''

        elif data == 'exit':
            break

        else:
            tx_data = "Unknown Command: " + data + \
                    "\nEnter 'help' for list of valid commands\n"

        filtered_file.write(tx_data)
      print('Closing connection with client')
      file.close()
      conn.close()


# Run following code when the program starts
if __name__ == '__main__':
   # Declare objects of My Thread class
   my_obj = MyThread(4)
   my_obj.setName('MyThread')


   # Start running the threads!
   my_obj.start()

   # Wait for the threads to finish...
   my_obj.join()

   print('Main Terminating...')

I made a new function to handle the input of text from the terminal. This function strips the new line char appended by readline() and optionally strips leading and trailing blanks from the input string. This enables your choice to close the connection if the user enters no data, while keeping it open and repeating the prompt if s/he just enters whitespace. I also used the 'errors' optional argument in conn.makefile(), to avoid the UnicodeDecodeError, and added a line to repeat the input of ssid and password if either is blank. I also removed the else clause when the main choice is "network", to let the user out only by choosing 3. With this version, there's no need to use Ctrl-C (that also has a meaning as KeyboardInterrupt) and if the user entered it, it wouldn't disrupt the server anymore. Now, if you press Ctrl-C in the client, you stop receiving the server's output, but if you blindly enter the "exit" choices you get out of the connection gracefully.

Daniel · Answer 2 · 2020-06-14T10:52:20.733

Because ctrl-c not a unicode character and _io.TextIOWrapper (your file variable) can't handle it in unicode mode.

You can initialize wrapper in non unicode mode

file = conn.makefile(mode="rw",encoding='latin-1')

This will not disconnect client by ctrl-c and you should handle all special characters by your self (because telnet will still send it)

I changed your code to handle decoder errors as is. This will disconnect client on ctrl-c

while True:
    file.write("Command> ")
    file.flush()
    try:
        data = file.readline()
    except UnicodeDecodeError as e:
        conn.close()
        break;

    tx_data = ""
    #if not data: break <-- This will throw error data undefined 
    data = data.strip()

Actually better solution not to use text wrapper at all

    data = conn.recv(1024)
    if not data: break

score 0 · Answer 3 · answered Jun 14 '20 at 15:34

0

you have to initialize wrapper in other encoding type, use encoding = "latin-1"

answered Jun 14 '20 at 15:34

Piyush Pravin

1
1

Unicode error for python echo server when "Ctrl+C" pressed

3 Answers3