Python Encoding Issues

Question

I have data that I would like to decode from and its in Windows-1252 basically I send code to a socket and it sends it back and I have to decode the message and use IEEE-754 to get a certain value from it but I can seem to figure out all this encoding stuff. Here is my code.

def printKinds ():
    test = "x40\x39\x19\x99\x99\x99\x99\x9A"

    print (byt1Hex(test))
    test = byt1Hex(test).replace(' ', '')
    struct.unpack('<d', binascii.unhexlify(test))
    print (test)
printKinds()

def byt1Hex( bytStr ):
    return ' '.join( [ "%02X" % ord( x ) for x in bytStr ] )

So I use that and then I have to get the value from that.. But it's not working and I can not figure out why.

The current output I am getting is

struct.unpack('<d', binascii.unhexlify(data))
struct.error: unpack requires a bytes object of length 8

That the error the expected output I am looking for is 25.1 but when I encode it, It actually changes the string into the wrong values so when I do this:

 print (byt1Hex(data))

I expect to get this.

40 39 19 99 99 99 99 9A

But I actually get this instead

78 34 30 39 19 99 99 99 99 9A

Provide some code we can actually run. Show us the byt1Hex function. — René Fleschenberg, Jul 02 '15 at 14:05
What is not working? What is expected output? What is actual output? What does `byt1Hex` do? — Mark Tolonen, Jul 02 '15 at 14:07
There is little sense to interpret cp1252 encoded text as IEEE-754 double value: `struct.unpack('!d', 'abcdefgh'.encode('cp1252'))` — jfs, Jul 02 '15 at 14:12
why do you convert a bytestring into hexstring, only to convert it back immediately? — jfs, Jul 02 '15 at 14:13
Im not sure I am new to this stuff and basically my process to this was to get the data decode it, then use IEEE-754 to convert it — cunniemm, Jul 02 '15 at 14:16
What's encoded in your string? `binary64` or `decimal64` or a combination of smaller values? — dlask, Jul 02 '15 at 14:22

score 3 · Accepted Answer · answered Jul 02 '15 at 14:16

3

>>> import struct
>>> struct.pack('!d', 25.1)
b'@9\x19\x99\x99\x99\x99\x9a'
>>> struct.unpack('!d', _) #NOTE: no need to call byt1hex, unhexlify
(25.1,)

You send, receive bytes over the network. No need hexlify/unhexlify them; unless the protocol requires it (you should mention the protocol in the question then).

answered Jul 02 '15 at 14:16

jfs

399,953
195
994
1,670

But I get the data "\x40\x39\x19\x99\x99\x99\x99\x9A" from the host and it looks like you changed it? – cunniemm Jul 02 '15 at 14:18
@BSD_: it is the same thing: `b"@" == b"\x40"` (if you are on Python 2 then you won't see `b` prefix for bytes literals) – jfs Jul 02 '15 at 14:21
Uhm, im not sure to be honest how could you tell? – cunniemm Jul 02 '15 at 14:30
@BSD_: what are you unsure of? What can I tell? – jfs Jul 02 '15 at 14:38
Im unsure if im decimal or binary haha thats why I asked you – cunniemm Jul 02 '15 at 14:42
`b"\x40\x39\x19\x99\x99\x99\x99\x9A"` is a sequence of bytes that represent `25.1` floating point value in [IEEE 754 format (binary64)](https://en.wikipedia.org/wiki/IEEE_floating_point). – jfs Jul 02 '15 at 14:58
Yes. just like b'@Rs33333' represents 73.8 floating point value in IEEE 754 format (binary64) – cunniemm Jul 02 '15 at 15:12

score 0 · Answer 2 · answered Jul 02 '15 at 14:13

0

You have:

test = "x40\x39\x19\x99\x99\x99\x99\x9A"

You need:

test = "\x40\x39\x19\x99\x99\x99\x99\x9A"

answered Jul 02 '15 at 14:13

dlask

8,776
1
26
30

Python Encoding Issues

2 Answers2