TypeError: Object of type Response is not JSON serializable (2)

Question

I have been trying to serialize an SQLAlchemy model so that I can pass a number of data from flask server to an html socketio client.

I am trying the following where an object is frirst converted into a dictionary and then jsonified. However, this is not working. I am getting the following error message at the designated line:

TypeError: Object of type Response is not JSON serializable

How can achieve what I need?

app.py

from flask import Flask, render_template, jsonify
from flask_marshmallow import Marshmallow
from flask_sqlalchemy import SQLAlchemy
from datetime import datetime
import secrets
import string
import logging
from flask_socketio import SocketIO, emit
import os
import subprocess

async_mode = None

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///filename.db'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
db = SQLAlchemy(app)

socketio_obj = SocketIO(app)   

class JobQueue(db.Model):
    __tablename__ = 'job_queue'
    job_id = db.Column(db.Integer, primary_key=True)
    unique_job_key = db.Column(db.String(64), index=True)
    user_name = db.Column(db.String, index=True)
    input_string = db.Column(db.String(256))
    is_done = db.Column(db.Boolean)
    created_at = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)

    def to_dictionary(self):
        return {
            "job_id": self.job_id,
            "unique_job_key": self.unique_job_key,
            "user_name": self.user_name,
            "input_string": self.input_string,
            "is_done": self.is_done,
            "created_at": self.created_at
        }
    # end function
# end class


def get_data():
    users = JobQueue.query.order_by(JobQueue.created_at).all()
    return users


def get_serialized_data():
    all_users = get_data()
    returns = [item.to_dictionary() for item in all_users]
    return jsonify(all_users = returns)

... ... ... ... ... ... ... ...

@socketio_obj.on('first_connect_event', namespace='/test')
def handle_first_connect_event(arg1):
    try:
        msg = arg1['first_message']
        print(msg)
        users = get_serialized_data()
        emit('get_data', {'users': users}) #<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
    except Exception as ex:
        logging.exception("")
        print(ex)

The full error trace:

ERROR:root:
Traceback (most recent call last):
  File "C:\git\funkclusterfrontend\app.py", line 106, in handle_first_connect_event
    emit('get_data', {'users': users})
  File "C:\ProgramData\Miniconda3\lib\site-packages\flask_socketio\__init__.py", line 899, in emit
    return socketio.emit(event, *args, namespace=namespace, to=to,
  File "C:\ProgramData\Miniconda3\lib\site-packages\flask_socketio\__init__.py", line 460, in emit
    self.server.emit(event, *args, namespace=namespace, to=to,
  File "C:\ProgramData\Miniconda3\lib\site-packages\socketio\server.py", line 294, in emit
    self.manager.emit(event, data, namespace, room=room,
  File "C:\ProgramData\Miniconda3\lib\site-packages\socketio\base_manager.py", line 167, in emit
    self.server._emit_internal(eio_sid, event, data, namespace, id)
  File "C:\ProgramData\Miniconda3\lib\site-packages\socketio\server.py", line 612, in _emit_internal
    self._send_packet(eio_sid, packet.Packet(
  File "C:\ProgramData\Miniconda3\lib\site-packages\socketio\server.py", line 617, in _send_packet
    encoded_packet = pkt.encode()
  File "C:\ProgramData\Miniconda3\lib\site-packages\socketio\packet.py", line 61, in encode
    encoded_packet += self.json.dumps(data, separators=(',', ':'))
  File "C:\ProgramData\Miniconda3\lib\json\__init__.py", line 234, in dumps
    return cls(
  File "C:\ProgramData\Miniconda3\lib\json\encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "C:\ProgramData\Miniconda3\lib\json\encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "C:\ProgramData\Miniconda3\lib\json\encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Response is not JSON serializable
Object of type Response is not JSON serializable

score 2 · Answer 1 · answered Jan 04 '23 at 16:38

The jsonify() function is used to create responses in Flask endpoints, it should never be used when emitting data using Flask-SocketIO.

In Flask-SocketIO, the data that you send must be passed as a dictionary. The conversion to JSON is automatically handled by the extension.

To fix this, change the get_serialized_data() function to return the list of users as a list of Python dictionaries:

def get_serialized_data():
    all_users = get_data()
    return [item.to_dictionary() for item in all_users]

In addition, you should make sure that your to_dictionary() method only uses JSON-friendly types. The created_at field should be converted to a string to work with JSON.

Once you fix these two problems, I think your emit should serialize without errors.

user366312 · Accepted Answer · 2023-01-04T21:33:12.807

I solved this serialization problem in the following way:

converted the dictionary into a JSON
converted the JSON into string
transmitted the string from server to client
on the cient side, converted the string into JSON again

This is how I was able to skip Marshmallow and other similar libraries.

My code looks as follows:

Server side

def get_serialized_data():
    all_users = get_data()
    dictionary = [item.to_dictionary() for item in all_users]
    json_object = json.dumps(dictionary, indent=4, sort_keys=True, default=str)
    return len(all_users), json_object

@socketio_obj.on('first_connect_event', namespace='/test')
def handle_first_connect_event(arg1):
    try:
        msg = arg1['first_message']
        print(msg)
        rows, users = get_serialized_data()
        print(rows)
        # print(users)
        emit('get_data', {'rows': str(rows), 'users': str(users)})
    except Exception as ex:
        logging.exception("")
        print(ex)

Client side

socket.on('get_data', function (msg){
              let row_count = msg['rows'];
              let json_users = $.parseJSON(msg['users']);
              create_table(row_count, json_users);
            });

            function create_table(row_count, json_users)
            {
                let table_body = $("#table1 tbody");
                table_body.empty();
                let markup = '';
                for(let i=0 ; i<row_count ; i++)
                {
                    markup = "<tr>"
                            + "<td>" + json_users[i]['job_id'] + "</td>"
                            + "<td>" + json_users[i]['unique_job_key'] + "</td>"
                            + "<td>" + json_users[i]['user_name'] + "</td>"
                            + "<td>" + json_users[i]['input_string'] + "</td>"
                            + "<td>" + json_users[i]['is_done'] + "</td>"
                            + "<td>" + json_users[i]['created_at'] + "</td>"
                        +"</tr>";
                    table_body.append(markup);
                }
            }

TypeError: Object of type Response is not JSON serializable (2)

2 Answers2