Flask-RESTful - don't return object property instead of returning null

Question

Let's say I've got a clients table with id, name and email fields. An email field is optional.

The code looks like this:

client_fields = {
   'id' : fields.String,
   'name' : fields.String,
   'email' : fields.String
}

And for displaying:

class ClientList(Resource):
    @marshal_with(client_fields)
    def get(self):
       return model.Client.query.all()

When email is not provided, API returns JSON like this:

{
   "id": "1",
   "name": "John Doe",
   "email": null
}

But instead I want it to return this object:

{
   "id": "1",
   "name": "John Doe"
}

Which basically means that instead of a property with null value I want it to return no such property at all. Is there a way to achieve that?

This is generally not a good design decision. Any consumers of your API that are looking for the email field will now need special handling to deal with whether or not the field will be there. It's much safer for your consumers to have a null field rather than a field that may or may not be there. — kylieCatt, Mar 15 '16 at 16:54
I don't really agree with that.. objects can be from multiple datasources and have loads of incomplete data. On a case by case basis sometimes this is a completely legitimate decision. — Magoo, Nov 23 '20 at 23:21

DaveBensonPhillips · Accepted Answer · 2016-03-21T01:22:10.857

13

I would use the marshal function instead of the marshal_with decorator:

class ClientList(Resource):
    def get(self):
       clients = []
       for client in model.Client.query.all():
           if client.email:
               clients.append(marshal(client_fields))
           else:
               clients.append(marshal(client_fields_no_email))
       return clients

Or even better

class ClientList(Resource):
    def get(self):
       return [client_marshal(client) for client in model.Client.query.all()]

with

def client_marshal(client):
    if client.email:
        return {'id' : fields.String,
                'name' : fields.String,
                'email' : fields.String}
    else:
        return {'id' : fields.String,
                'name' : fields.String}

edited Mar 21 '16 at 01:22

answered Mar 15 '16 at 16:50

DaveBensonPhillips

3,134
1
20
32

4

This works when the number of possible modifications is relatively small, but what if you want to show different sets of fields based on oauth scopes, or based on user privacy settings, etc. – bigblind Mar 22 '16 at 15:17
1

@bigblind aye, indeed I came here for a solution with many solutions. – Tommy Apr 13 '16 at 14:12

Nizam Mohamed · Answer 2 · 2016-03-22T21:25:47.323

There are two ways it can be done, pre-marshalling and post-marshalling modification. Pre-marshalling removes any default values given to field names in the client_fields dict but post-marshalling preserves them.

In pre-marshalling method, you have to pass a modified fields dict to marshal function if client's email is None.
For example;

import json
from flask_restful import fields, marshal, marshal_with

class Client(object):
    def __init__(self, id, name, email=None):
        self.id = id
        self.name = name
        self.email = email

client_fields = {
    'id': fields.String,
    'name': fields.String,
    'email': fields.String
}

def get():
    clients =[Client(1, 'Tom'), Client(2, 'John', 'john@example.com')]
    return [marshal(client, client_fields if client.email else {k: v for k, v in client_fields.items() if k != 'email'}) for client in clients]

print(json.dumps(get()))

Output;

[{"id": "1", "name": "Tom"}, {"email": "john@example.com", "id": "2", "name": "John"}]

In post-marshalling you have to remove the email field of the OrderedDict returned by marshal_with if it is None.
The de_none function by default removes all fields those are None or you have to explicitly pass the field names if that's not desired and you have to pass envelope argument too if marshal_with takes the same.

from functools import wraps
import json
from flask_restful import fields, marshal, marshal_with

client_fields = {
    'id': fields.String,
    'name': fields.String,
    #'email': fields.String(default='user@example.com')
    'email': fields.String
}

class Client(object):
    def __init__(self, id, name, email=None):
        self.id = id
        self.name = name
        self.email = email

def de_none(envelope=None, *fields):
    def decorator(func):
        def dict_remove(d):
            if fields:
                for field in fields:
                    if d[field] is None:
                        d.pop(field)
            else:
                for k, v in d.items():
                   if v is None:
                       d.pop(k)

        @wraps(func)
        def decorated(*args, **kwargs):
            data = result = func(*args, **kwargs)
            if isinstance(result, tuple):
                data = result[0]
            if envelope:
                data = data[envelope]
            if isinstance(data, (list, tuple)):
                for d in data:
                    dict_remove(d)
            else:
                dict_remove(data)
            return result
        return decorated
    return decorator

@de_none()
@marshal_with(client_fields)
def get():
    #return [Client(1, 'Tom'), Client(2, 'john', 'john@example.com')], 200, {'Etag': 'blah'}
    #return [Client(1, 'Tom'), Client(2, 'john', 'john@example.com')]
    #return Client(1, 'Tom'), 200, {'Etag': 'foo'}
    return Client(1, 'Tom')

print(json.dumps(get()))


@de_none()
@marshal_with(client_fields)
def get():
    return Client(2, 'John', 'john@example.com'), 201, {'Etag': 'ok'}

print(json.dumps(get()))

Output;

{"id": "1", "name": "Tom"}
{"email": "john@example.com", "id": "2", "name": "John"}

UPDATE Request hooks
The app.after_request decorator can be used to modify response object. Any default values given to fields are preserved.
The remove_none_fields request hook takes fields parameter which can be None to remove all fields with None value or a list of field names to selectively remove.

import json
from flask import Flask, Response
from flask_restful import fields, marshal_with, Api, Resource

app = Flask(__name__)
api = Api(app)

class Client(object):
    def __init__(self, id, name, email=None):
        self.id = id
        self.name = name
        self.email = email

client_fields = {
    'id': fields.String,
    'name': fields.String,
    'email': fields.String,
    'age': fields.String
}

class ClientList(Resource):
    @marshal_with(client_fields)
    def get(self):
        clients =[Client(1, 'Tom'), Client(2, 'John', 'john@example.com')]
        return clients, 200

@app.after_request
def remove_none_fields(resp, fields=('email',)):
    """
    removes all None fields
    """

    if not 'application/json' in resp.content_type:
        return resp

    def dict_remove(d, fields):
        if fields:
            for field in fields:
                if d[field] is None:
                    d.pop(field)
        else:
            for k, v in tuple(d.items()):
                if v is None:
                    d.pop(k)

    data = json.loads(resp.get_data())
    if isinstance(data, list):
        for obj in data:
            dict_remove(obj, fields)
    else:
        dict_remove(data, fields)

    resp.set_data(json.dumps(data, indent=1))
    resp.content_length = resp.calculate_content_length()
    return resp

api.add_resource(ClientList, '/')

if __name__ == '__main__':
    app.run(debug=True)

Output;

[
 {
  "age": null,
  "name": "Tom",
  "id": "1"
 },
 {
  "age": null,
  "email": "john@example.com",
  "name": "John",
  "id": "2"
 }
]

update patching flask_restful.marshal
I filter out None values in a genexp inside marshal function and replace flask_restful.marshal with marshal defined here.

from collections import OrderedDict
from flask import Flask
import flask_restful
from flask_restful import fields, marshal_with, Api, Resource

app = Flask(__name__)
api = Api(app)

class Client(object):
    def __init__(self, id, name, email=None):
        self.id = id
        self.name = name
        self.email = email

client_fields = {
    'id': fields.String,
    'name': fields.String,
    'email': fields.String,
}

def marshal(data, fields, envelope=None):
    def make(cls):
        if isinstance(cls, type):
            return cls()
        return cls

    if isinstance(data, (list, tuple)):
        return (OrderedDict([(envelope, [marshal(d, fields) for d in data])])
                if envelope else [marshal(d, fields) for d in data])

    items = ((k, marshal(data, v) if isinstance(v, dict)
              else make(v).output(k, data))
             for k, v in fields.items())
    #filtering None
    items = ((k,v) for k, v in items if v is not None)
    return OrderedDict([(envelope, OrderedDict(items))]) if envelope else OrderedDict(items)

flask_restful.marshal = marshal

class ClientList(Resource):
    @marshal_with(client_fields)
    def get(self):
        clients =[Client(1, 'Tom'), Client(2, 'John', 'john@example.com')]
        return clients, 200

api.add_resource(ClientList, '/')

if __name__ == '__main__':
    app.run(debug=True)

Output;

[   
    {   
        "id": "1",
        "name": "Tom"
    },
    {   
        "email": "john@example.com",
        "id": "2",
        "name": "John"
    }
]

hey **down voter**, please leave comment so people may learn from mistakes. — Nizam Mohamed, Mar 23 '16 at 16:05

score 3 · Answer 3 · answered Oct 15 '21 at 13:10

You should use the skip_none property of the @marshal decorator. Its much more convenient than the approaches suggested in the other answers.

@marshal(some_model, skip_none=True)
def get():
   ...

The documentation can be found here: https://flask-restplus.readthedocs.io/en/stable/marshalling.html

This is also possible using Flask Restx :)

Flask-RESTful - don't return object property instead of returning null

3 Answers3