Transfer files with Python 2.7/Pyro4 UnicodeEncodeError

Question

So I'm trying to learn Pyro by creating a simple file server with it. I've implemented a few basic features, and now I'm trying to add file transfer. I've looked at examples of how to do this (https://github.com/irmen/Pyro4/tree/master/examples/filetransfer), and the way it seems to be with pure Pyro done is just returning the bytes read from the file and writing them on the receiving end.

This is what I've done (I know I really should break up the files when sending them, but I can do that once this issue is fixed):

client.py

import Pyro4

server= Pyro4.Proxy("PYRONAME:server")

def download(file_name):
    output_file = open(file_name, "wb")
    output_file.write(server.download(file_name))
    output_file.close()
    print "Downloaded file: {}".format(file_name)

server.py

import Pyro4

@Pyro4.expose
class Server(object):
    def download(self, file_name):
        return open(file_name, "rb").read()

daemon = Pyro4.Daemon()
ns = Pyro4.locateNS()
uri = daemon.register(Server)
ns.register("server", uri)
daemon.requestLoop()

This works just fine for simple files like some .txt documents, but when I try other file types, for example .pdf, I get the error:

UnicodeEncodeError: 'ascii' codec can't encode characters in position 11-14: ordinal no in range(128)

I've spent some time looking up this error, and the closest I can come to a solution is by changing this line in client.py:

output_file.write(server.download(file_name))

to:

output_file.write(server.download(file_name).encode("ascii", "replace"))

This avoids the error, completes the download, and gives a file of the right size. But the file becomes corrupted and unopenable.

Any suggestions how to fix this? If not is there any other way to implement file transfer with Pyro?

Why are you asking how to do correct file transfer with Pyro, while you're already looking at working example code from Pyro itself? The code should explain your problem and how to fix it. Hint: it has to do with the fact that binary data is not transferred as you may expect, if using the defaults. — Irmen de Jong, Mar 07 '18 at 15:00
I've tried using the serpent module as they do in the example, and I still get the same UnicodeEnodeError. Is this what you mean? — gabaxaza, Mar 07 '18 at 16:54

score 0 · Answer 1 · answered Mar 07 '18 at 14:55

0

This is because file.write can write only bytes and it also tries to convert input string to ascii encoding. Your pdf file is of a different encoding (Maybe UTF-8). You can try something like

fptr.write(byte_string).encode("utf-8"))

I observed same error while working with files. And this was due to some encoding other than ascii. You may refer to this SO Post for more info . For more on pdf encoding you may refer to page 86 of this pdf.

answered Mar 07 '18 at 14:55

cutteeth

2,148
3
25
45

I've tried all sorts of parameters in .encode as you suggest here, and none seem to work. They successfully write the files, but they all come out corrupted. Also this problem doesn't only happen when transferring pdfs, but also some other file types like .mp4. – gabaxaza Mar 07 '18 at 16:57

score 0 · Answer 2 · answered Mar 08 '18 at 17:10

see here: https://github.com/irmen/Pyro4/blob/master/examples/filetransfer/client.py#L14

If using serpent, you'll need to pay special attention to properly processing the serialized bytes. The example shows how to do this.

More efficient wire encoding of binary data is done when using the pickle or marshal serializers. Details here https://pyro4.readthedocs.io/en/stable/tipstricks.html#binary-data-transfer-file-transfer

Transfer files with Python 2.7/Pyro4 UnicodeEncodeError

2 Answers2