Python socket: cannot receive int array

Question

I try to make a inter process communication between a Python and c program via winsockets. Sending a string does work, but now I try to send an int array from the c socket to the python socket.

I already found out that I have to use htonl() to convert the int array into a byte stream as the send function of winsock2 cannot send int arrays directly.

Now I want to use ntohl() in the python socket but the receive function returns bytes whereas ntohl() needs an integer value as input.

Here is my code

C-Side (just relevant parts):

uint32_t a[1] = {1231};
uint32_t a_converted[1]={0};
a_converted[0] = htonl(a[0]);
iResult = send( ConnectSocket, ( char *)   a_converted, sizeof( a_converted), 0 );

Python Side (just relevant parts):

data = connection.recv(16)
data_i = socket.ntohl(data)

You do __not__ have to `htonl` an int before sending. There is no reason at all, unless you have a protocol defined that needs it. If you are sure that both the server and client use the same endianness, just send it as it is. — Ajay Brahmakshatriya, Nov 22 '17 at 13:38
but i still have to convert the int array to a const char *buf which doenst work as it should — Gora, Nov 22 '17 at 13:41

volcano · Accepted Answer · 2017-11-22T15:14:40.470

1

What you received is string of bytes, did not ntohl cause exception? You may use struct module to unpack - for 16 bytes

struct.unpack('!4I', data)

Meaning - unpack 4 unsigned 32-bit integers in network order

RTM

(I cannot test it - try it on your own)

EDIT: Ooops, did not read your comment through. According to sockets docs, recv should return object of type bytes. If it returns object of type str - you should convert it to bytes - in Python3 it would be data.encode()

PS Which Python are you on?

edited Nov 22 '17 at 15:14

answered Nov 22 '17 at 13:59

volcano

3,578
21
28

1

i had to change it to test =struct.unpack('<4I', data). Now it works. Thank you – Gora Nov 22 '17 at 14:42
But it always throws the error: struct.error: unpack requires a bytes object of length 16. I checked the length of data and it IS 16. The print of the unpacked data is exact the same as it looks in the c socket – Gora Nov 22 '17 at 14:48
1

I found my misstake: The unpack method is the right one as recv returns a bytes array. recv happens in a loop and at the second run the receive buffer is empty, which causes the fault. A simple check if data is empty solved the problem :) – Gora Nov 23 '17 at 12:36

Ajay Brahmakshatriya · Answer 2 · 2017-11-22T14:46:15.440

You said you have managed to send strings over the connection. I assume you sent a char* and received it in python as a string. What you have done is sent a stream of bytes.

Now you want to send an array of integers. In the memory, the integers are again stored as bytes. Each integer could occupy 4/8 bytes. You can check this before hand by printing

printf("Size of integer is %zu", sizeof(int));

Okay, great now we know how many bytes we need to send. Say it is 4 for now.

We also need to know the endianness of the integers but lets assume big endian for now.

This means the lowest significant byte will be first and the highest significant byte at the end.

So now you can send the integer array exactly lile you sent, by casting the array to char* and sending sizeof(array).

On the receiving side though, you just have a stream of bytes. To convert it to array of integers you need to get 4 bytes at a time and combine it into an integer.

We can do that as follows.

Say there are total 10 integers. You have to pass this information on separately somehow.

bytes = connection.recv(10*4)

array = []
for i in range(10):
    x = ord(bytes[i*4+0])
    x += ord(bytes[i*4+1]) << 8
    x += ord(bytes[i*4+2]) << 16
    x += ord(bytes[i*4+3]) << 24
    array += [x]
print x

And you will be able to see you array of integers.

Here the function ord converts a character to its ASCII equivalent integer.

Side notes:

Now, if your system has size of integer as 8 instead of 4, you need to extend the body of the loop in python. It will go till 56. Also each of the index in bytes will be i*8+...

Similarly if the endianess is different, the order of the elements will change. Basically the indices on bytes will go from i*4+3 to i*4+0.

I think we are on the right way now. ntohl() was misleading, actually. With your code i get the error TypeError: ord() expected string of length 1, but int found — Gora, Nov 22 '17 at 14:01
Python has tools for working with communication data - _struct_ at the lowest level. You are re-inventing a wheel - and your shifts are in the wrong direction! And _range_ function has a _step_ argument — volcano, Nov 22 '17 at 14:15

Python socket: cannot receive int array

2 Answers2