Error in converting from json using nlohmann json package

Question

I'm trying to convert the json of the form

{
   "content": {
     "test_key": "test"
   },
   "sender": "alice",
   "type": "key_type"
}

to my object which is

template<class Content>
struct Event
{
        Content content;
        std::string type;
};

a template is being used as the structure of the Content is not fixed. When I try using the from_json which is like

template<class Content>
void
from_json(const nlohmann::json &obj, Event<Content> &event)
{
        event.content = obj.at("content").get<Content>();
        event.type    = obj.at("type").get<std::string>();
}

I'm getting the error

[json.exception.out_of_range.403] key 'content' not found

although there is content key in the json. Why is it so?

#include <iostream>
#include <nlohmann/json.hpp>

using json = nlohmann::json;
using namespace std;

template<typename Content>
struct Event
{
    Content content;
    string type;
};

template<typename Content>
void from_json(const nlohmann::json &obj, Event<Content> &event)
{
    event.content = obj.at("content").get<Content>();
    event.type    = obj.at("type").get<string>();
}

struct Key{
    string test_key;
    string random_data;
};

int main(){
    json j={{"content",{{"test_key","test"}}},{"sender","alice"},{"type","key_type"}};

    Event<Key> event_instance;

    try{
        from_json(j,event_instance);
    }
    catch(json::exception& e){
        cout<<e.what()<<endl;
    }
}

The above code is a minimum reproducible example

@TedLyngmo I thought I'll elaborate for ease of understanding for the **for_json** function the first argument is the json as mentioned in first snippet and the second argument would be an instance of the class in the second snippet. On passing these arguments to the function I'm getting this error. Hope it looks ok now. — Bad_Panda, Apr 28 '20 at 16:57

Ted Lyngmo · Accepted Answer · 2020-04-29T08:09:54.767

What's missing is serializer support for your type Key. With that added, extraction works:

void from_json(const nlohmann::json& obj, Key& k) {
    k.test_key = obj.at("test_key").get<std::string>();
    // k.random_data missing in json
}

template<typename Content>
void from_json(const nlohmann::json& obj, Event<Content>& event) {
    event.content = obj.at("content").get<Content>();
    event.type = obj.at("type").get<std::string>();
}

Demo

To handle optional fields like random_data in your Key, you could create a helper function, here called get_optional which returns a C++17 std::optional<T>. For earlier C++ versions, you could use boost::optional.

#include <nlohmann/json.hpp>

#include <iostream>
#include <optional>
#include <string>

using json = nlohmann::json;

template<typename Content>
struct Event {
    Content content{};
    std::string type{};
};

struct Key {
    std::string test_key{};
    std::optional<std::string> random_data{}; // optional field made optional
};

template<typename T>
std::optional<T> get_optional(const json& obj, const std::string& key) try {
    return obj.at(key).get<T>();
} catch(const json::exception&) {
    return std::nullopt;
}

void from_json(const json& obj, Key& k) {
    k.test_key = obj.at("test_key").get<std::string>();
    k.random_data = get_optional<std::string>(obj, "random_data");
}

template<typename Content>
void from_json(const json& obj, Event<Content>& event) {
    event.content = obj.at("content").get<Content>();
    event.type = obj.at("type").get<std::string>();
}

int main() {
    json j = {{"content", {{"test_key", "test"}}},
              {"sender", "alice"},
              {"type", "key_type"}};

    try {
        auto event_instance = j.get<Event<Key>>();
        std::cout << event_instance.content.test_key << '\n';

        if(event_instance.content.random_data) {
            std::cout << event_instance.content.random_data.value() << '\n';
        } else {
            std::cout << "no random_data\n";
        }

        std::cout << event_instance.type << '\n';
    } catch(const json::exception& e) {
        std::cerr << e.what() << std::endl;
    }
}

Demo

But my issue is that I can sometimes have **random_data** field in the content, in that case I should have separate serialiser even for that.So is there any way I can have a general serialiser which can have all the possible fields ? — Bad_Panda, Apr 29 '20 at 03:34
@Bad_Panda That issue isn't described at all in your question. As your question stands, my answer answers it. If Anyway, if your json data does not map 1:1 to full C++ objects, you'll have to add checks to see if the json contains the field and provide a default value if it does - alt. make `random_data` into `std::optional random_data;`. — Ted Lyngmo, Apr 29 '20 at 05:59
@Bad_Panda You are welcome. I wanted to add an example of how to use `std::optional` to my answer and how you could create a helper function for that but it triggers leaks in `valgrind`. I'll post it here so you can see the idea anyway: https://godbolt.org/z/DkAjqD — Ted Lyngmo, Apr 29 '20 at 07:07
@Bad_Panda It seems the leak reported by `valgrind` only happens when using LLVM's `libc++` and an exception in thrown in `get_optional()`. Using the default `libstdc++` with `g++` or `clang++` reports no leaks - so I added the example now. — Ted Lyngmo, Apr 29 '20 at 08:04
Thanks, this is really awesome. Using std::optional and that helper function gives more flexibility in exchanging the json. — Bad_Panda, Apr 29 '20 at 08:28

Error in converting from json using nlohmann json package

1 Answers1