3

I have a rudimentary grasp of python but am not clear on dealing with binary encoding issues. I am trying to run sample code from a firefox-webextensions example in which a python script sends text that is read by a javascript program. I keep encountering encoding errors.

The python code is:

#! /Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5
import sys, json, struct

text = "pong"

encodedContent = json.dumps(text)
encodedLength = struct.pack('@I', len(encodedContent))
encodedMessage = {'length': encodedLength, 'content': encodedContent}

sys.stdout.write(encodedMessage['length'])
sys.stdout.write(encodedMessage['content'])

The error trace (displayed in firefox console) is:

stderr output from native app chatX: Traceback (most recent call last):
stderr output from native app chatX: File "/Users/inchem/Documents/firefox addons/py/chatX.py", line 10, in <module>
stderr output from native app chatX: sys.stdout.write(encodedMessage['length'])
stderr output from native app chatX: TypeError: write() argument must be str, not bytes

Running python 3.5.1 on OS X El Capitan 10.11.6, x86 64bit cpu; firefox developer ed 52.0

The python script I am using, as shown above, is minimized from the original at https://developer.mozilla.org/en-US/Add-ons/WebExtensions/Native_messaging

I also tried:

sys.stdout.buffer.write(encodedMessage['length'])
sys.stdout.buffer.write(encodedMessage['content'])

which generated:

stderr output from native app chatX: sys.stdout.buffer.write(encodedMessage['content'])
stderr output from native app chatX: TypeError: a bytes-like object is required, not 'str'
martineau
  • 119,623
  • 25
  • 170
  • 301
Cam U
  • 340
  • 3
  • 10
  • have you tried to convert it to string like the following?sys.stdout.buffer.write(str(encodedMessage['length'])) – Shiping Jan 22 '17 at 19:57

2 Answers2

4

The example was probably Python 2-compliant, but things have changed in Python 3.

You are generating a binary representation of the length as bytes with this:

encodedLength = struct.pack('@I', len(encodedContent))

It is not printable. You can write it through a socket stream which is a binary stream but not through stdout which is a text stream.

The trick of using buffer (as explained in How to write binary data in stdout in python 3?) is good but only for the binary part (note that you get the error message for the text part now):

sys.stdout.buffer.write(encodedMessage['length'])

For the text part, just write to stdout:

sys.stdout.write(encodedMessage['content'])

or use sys.stdout.buffer with string to bytes conversion:

sys.stdout.buffer.write(bytes(encodedMessage['content'],"utf-8"))
Community
  • 1
  • 1
Jean-François Fabre
  • 137,073
  • 23
  • 153
  • 219
2

you need to ensure your input is str (unicode) before writing to stdout / stderr.

in your example:

sys.stdout.write(encodedMessage['length'].decode('utf8'))
sys.stdout.write(encodedMessage['content'])

you can see that type(encodedLength)) is bytes while type(encodedContent) is str.

please read the following answer for more info about bytes vs string in python3.X

Community
  • 1
  • 1
ShmulikA
  • 3,468
  • 3
  • 25
  • 40