How can I do key-agnostic deserialization of JSON objects?

Question

I am working with a less than ideal API that doesn't follow any rigid standard for sending data. Each payload comes with some payload info before the JSON, followed by the actual data inside which can be a single string or several fields.

As it stands right now, if I were to map every different payload to a struct I would end up with roughly 50 structs. I feel like this is not ideal, because a ton of these structs overlap in all but key. For instance, there are I believe 6 different versions of payloads that could be mapped to something like the following, but they all have different keys.

I have these two JSON examples:

{"key": "string"}

{"key2": "string"}

And I want to serialize both into this struct:

#[derive(Debug, Deserialize)]
struct SimpleString {
    key: String,
}

The same can be said for two strings, and even a couple cases for three. The payloads are frustratingly unique in small ways, so my current solution is to just define the structs locally inside the function that deserializes them and then pass that data off wherever it needs to go (in my case a cache and an event handler)

Is there a better way to represent this that doesn't have so much duplication? I've tried looking for things like key-agnostic deserializing but I haven't found anything yet.

Why did you decide to not show *any* examples of the actual JSON or how you'd want them to look the same in Rust? — Shepmaster, Dec 14 '17 at 20:06
Are you already aware you could just deserialize everything into a [`Value`](https://docs.rs/serde_json/1.0.8/serde_json/enum.Value.html)? — Shepmaster, Dec 14 '17 at 20:09
Frankly, unambiguously documenting an ugly protocol with 50 structs doesn't strike me as that bad a proposition. I'd more ask the question of how this is going to evolve and change and what makes that process easier and more reliable... — Zalman Stern, Dec 14 '17 at 20:12
@Shepmaster I did show an example though, unless you want me to change the key to match the value in the payload. The problem is that with the example above, you can get keys that are all strings but are something like "channel", or "message", or "ops", or several other things, but they all just map to a string. Using a Value doesn't help much either since as far as I'm aware that just pushes the deserializing to manual match statements which sounds even more cumbersome. If you still want an example, what would you want it to show? — ozdrgnaDiies, Dec 14 '17 at 20:20
*I did show an example* — I must be missing something, as I'd expect to see two JSON input examples that are basically the same and the final Rust struct that you want to be able to deserialize both JSON inputs to and how you might use it. — Shepmaster, Dec 14 '17 at 20:25
The Rust struct is right there. I suppose I can add json examples that just looks like `{"key": "string"}` and `{"key2": "string"}`, but I'd assume anyone who can help with serde would know how it works. — ozdrgnaDiies, Dec 14 '17 at 20:26
It's not about "knowing how it works" but about being clear what you need. For example, are you making the assertion that the name of the key basically *doesn't matter* in the two JSON blobs you've given? You also mention "same can be said for two strings", but how would `{a: 1, b: 2}` be mapped to `struct Foo { one: i32, two: i32 }`? Should `a`=>`one` or `b`=>`one`? You've got a lot of context of your specific problem that is in your head and people helping on the internet *don't have that context*. When asking for help, you have to be **overly** clear and descriptive. — Shepmaster, Dec 14 '17 at 20:49

Shepmaster · Answer 1 · 2017-12-15T20:12:23.520

You can implement Deserialize for your type to decode a "map" and ignore the key name:

extern crate serde;
extern crate serde_json;

use std::fmt;
use serde::de::{Deserialize, Deserializer, Error, MapAccess, Visitor};

fn main() {
    let a = r#"{"key": "string"}"#;
    let b = r#"{"key2": "string"}"#;

    let a: SimpleString = serde_json::from_str(a).unwrap();
    let b: SimpleString = serde_json::from_str(b).unwrap();

    assert_eq!(a, b);
}

#[derive(Debug, PartialEq)]
struct SimpleString {
    key: String,
}

struct SimpleStringVisitor;

impl<'de> Visitor<'de> for SimpleStringVisitor {
    type Value = SimpleString;

    fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
        formatter.write_str("an object with a single string value of any key name")
    }

    fn visit_map<M>(self, mut access: M) -> Result<Self::Value, M::Error>
    where
        M: MapAccess<'de>,
    {
        if let Some((_, key)) = access.next_entry::<String, _>()? {
            if access.next_entry::<String, String>()?.is_some() {
                Err(M::Error::custom("too many values"))
            } else {
                Ok(SimpleString { key })
            }
        } else {
            Err(M::Error::custom("not enough values"))
        }
    }
}

impl<'de> Deserialize<'de> for SimpleString {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        deserializer.deserialize_map(SimpleStringVisitor)
    }
}

How can I do key-agnostic deserialization of JSON objects?

1 Answers1