Sending a Dictionary using Sockets in Python?

Question

My problem: Ok, I made a little chat program thing where I am basically using sockets in order to send messages over a network.

It works great, but when I decided to take it a step further, I ran into a problem.

I decided to add some encryption to the strings I was sending over the network, and so I went ahead and wrote the script that did that.

The problem is that apparently you can't just send a dictionary through sockets as you might with a string.

I did some research first, and I found this stuff about Pickles. Unfortunately, I couldn't find out exactly how I could use them to convert strings, aside from having it exporting the dictionary to a file, but I can't do that without changing my program.

Can anyone help explain how I am to do this? I've looked around everywhere but I can't seem to find out how.

I've uploaded what I've got so far here, if that comes of any interest to anybody.

print("\n\t\t Fill out the following fields:")
HOST = input("\nNet Send Server Public IP: ")
PORT = int(input("\nNet Send Server Port: "))
#------------------------------------------------
#Assessing Validity of Connection
#------------------------------------------------
try:
    s = socket(AF_INET,SOCK_STREAM)
    s.connect((HOST,PORT))
    print("Connected to server:",HOST,)
except IOError:
    print("\n\n\a\t\tUndefined Connection Error Encountered")
    input("Press Enter to exit, then restart the script")
    sys.exit()
#-------------------------------------------------
#Now Sending and recieving mesages
#-------------------------------------------------


i = True
while i is True:
    try:
        User_input = input("\n Enter your message: ")
    Lower_Case_Conversion = User_input.lower()
    #Tdirectory just stores the translated letters
    Tdirectory = []
    # x is zero so that it translates the first letter first, evidently
    x = 0
    COUNTLIMIT = len(Lower_Case_Conversion)
    while x < COUNTLIMIT:
        for letter in Lower_Case_Conversion[x]:
            if letter in TRvalues:
                Tdirectory += [TRvalues[Lower_Case_Conversion[x]]]
        x = x + 1

        message = input('Send: ')
        s.send(message.encode())
        print("\n\t\tAwaiting reply from: ",HOST,)
        reply = s.recv(1024)
        print(HOST,"\n : ",reply)
    except IOError:
        print("\n\t\aIOError Detected, connection most likely lost.")
        input("\n\nPress Enter to exit, then restart the script")

Oh, and if your wondering what TRvalues is. It's the dictionary that contains the 'translations' for encrypting simple messages.

try:
    TRvalues = {}
    with open(r"C:\Users\Owatch\Documents\Python\FunStuff\nsed.txt", newline="") as f:
        reader = csv.reader(f, delimiter=" ")
        TRvalues = dict(reader)

(The translations are held in a .txt it imports)

Pickle is a decent way of serialization, sure, but you shouldn't use it (security and compatibility reasons). Use JSON instead. — Tom van der Woerdt, Mar 03 '13 at 20:40

score 27 · Accepted Answer · answered Mar 03 '13 at 20:37

27

You have to serialize your data. there would be many ways to do it, but json and pickle will be the likely way to go for they being in standard library.

for json :

import json

data_string = json.dumps(data) #data serialized
data_loaded = json.loads(data) #data loaded

for pickle(or its faster sibling cPickle):

import cPickle as pickle

data_string = pickle.dumps(data, -1) 
#data serialized. -1, which is an optional argument, is there to pick best the pickling protocol
data_loaded = pickle.loads(data) #data loaded.

also, please don't write

i= True
while i is True:
 #do_something

because simple while True: would suffice.

answered Mar 03 '13 at 20:37

thkang

11,215
14
67
83

So once my data is loaded, its ready to be sent, and I can replace the s.send(message.encode()) with s.send(data_loaded) (?) – Micrified Mar 04 '13 at 20:17
1

Here on Python 3 I need to encode and decode the dumps additionally. – NoDataDumpNoContribution Apr 14 '14 at 11:27
1

Using `json.dumps(data)` still have me `TypeError: a bytes-like object is required, not 'str'`, so I did `data= str.encode(json.dumps(data))` then `sock.sendall(data)` and it worked!! – Beginner Apr 29 '20 at 12:00
[pickle](https://davidhamann.de/2020/04/05/exploiting-python-pickle/) [is](https://blog.nelhage.com/2011/03/exploiting-pickle/) [not](https://docs.python.org/3/library/pickle.html) [secure](https://intoli.com/blog/dangerous-pickles/) – laundmo Jan 04 '21 at 12:15

score 5 · Answer 2 · answered Mar 03 '13 at 20:19

5

You need to serialize your data first. There are several ways to do this, the most common probably JSON, XML and (python specific) pickles. Or your own custom serialization.

The basic idea is: Serialize your data, send it, receive it, deserialize it again.

answered Mar 03 '13 at 20:19

Hyperboreus

31,997
9
47
87

Any good tutorials or links you got maybe? Thanks for answering. – Micrified Mar 03 '13 at 20:28
Just read the documentation of the python json module. It is really neat. `ser = json.dumps (your_dictionary)` and `your_dictionary = json.loads (ser)`. – Hyperboreus Mar 03 '13 at 20:30
[pickle](https://davidhamann.de/2020/04/05/exploiting-python-pickle/) [is](https://blog.nelhage.com/2011/03/exploiting-pickle/) [not](https://docs.python.org/3/library/pickle.html) [secure](https://intoli.com/blog/dangerous-pickles/) – laundmo Jan 04 '21 at 12:15

score 4 · Answer 3 · answered Mar 03 '13 at 20:33

If you want to use pickle you can use the loads and dumps functions.

import pickle
a_dict = { x:str(x) for x in range(5) }
serialized_dict = pickle.dumps(a_dict)
# Send it through the socket and on the receiving end:
a_dict = pickle.loads(the_received_string)

You can also use JSON in a similar fashion. I like JSON because it is human readable and isn't python specific.

import json
a_dict = { x:str(x) for x in range(5) }
serialized_dict = json.dumps(a_dict)
# Send it through the socket and on the receiving end:
a_dict = json.loads(the_received_string)

score 4 · Answer 4 · edited Apr 23 '17 at 16:21

4

Using JSON to serialize your data is the way I prefer to do it. I actually made a library that does just that for you: jsonsocket library. It will do the serialization/deserialization automatically for you. It also handles big amounts of data efficiently.

edited Apr 23 '17 at 16:21

Zach Saucier

24,871
12
85
147

answered May 01 '14 at 06:06

Mouad Debbar

3,126
2
20
20

Tied to have a play with this, and the jsonsocket.py has invalid syntax. – Clive Bostock Apr 06 '23 at 18:10

score 2 · Answer 5 · answered Mar 03 '13 at 22:27

2

You can use pickle and python remote object (or pyro only), to send full objects and data over networks (Internet included). For instance, if you want send object (dict, list, class, objects, etc. ) use python remote objects for it.

It very useful for you want to do.

There is more information in this link http://pythonhosted.org/Pyro4/ And this starter manual can be useful to know what you send or execute on network pcs http://pythonhosted.org/Pyro4/intro.html#simple-example

I hope it will help you

answered Mar 03 '13 at 22:27

ManuParra

1,471
6
18
33

[pickle](https://davidhamann.de/2020/04/05/exploiting-python-pickle/) [is](https://blog.nelhage.com/2011/03/exploiting-pickle/) [not](https://docs.python.org/3/library/pickle.html) [secure](https://intoli.com/blog/dangerous-pickles/) – laundmo Jan 04 '21 at 12:15

score 1 · Answer 6 · answered Jun 26 '15 at 12:28

1

You can also use zmqObjectExchanger (https://github.com/ZdenekM/zmq_object_exchanger). It wraps pickle and zmq to transfer python objects over network.

answered Jun 26 '15 at 12:28

ZdenekM

11
1

Sending a Dictionary using Sockets in Python?

6 Answers6

Linked