Unable to tackle optional fields in JSON with Rustc-serialize

Question

I am trying to deserialize JSON to Rust structure using rustc_serialize. The problem is that certain JSONs have some optional fields, i.e., may or may not be present. The moment the first absent field is encountered, the decoder seems to bail out and not consider subsequent fields, even if they are present. Is there a way to overcome this?

Here is the code:

extern crate rustc_serialize;

#[derive(Debug)]
struct B {
    some_field_0: Option<u64>,
    some_field_1: Option<String>,
}

impl rustc_serialize::Decodable for B {
    fn decode<D: rustc_serialize::Decoder>(d: &mut D) -> Result<Self, D::Error> {
        Ok(B {
            some_field_0: d.read_struct_field("some_field_0", 0, |d| rustc_serialize::Decodable::decode(d)).ok(),
            some_field_1: d.read_struct_field("some_field_1", 0, |d| rustc_serialize::Decodable::decode(d)).ok(),
        })
    }
}

fn main() {
    {
        println!("--------------------------------\n1st run - all field present\n--------------------------------");
        let json_str = "{\"some_field_0\": 1234, \"some_field_1\": \"There\"}".to_string();
        let obj_b: B = rustc_serialize::json::decode(&json_str).unwrap();

        println!("\nJSON: {}\nDecoded: {:?}", json_str, obj_b);
    }

    {
        println!("\n\n--------------------------------\n2nd run - \"some_field_1\" absent\n---------------------------------");
        let json_str = "{\"some_field_0\": 1234}".to_string();
        let obj_b: B = rustc_serialize::json::decode(&json_str).unwrap();

        println!("\nJSON: {}\nDecoded: {:?}", json_str, obj_b);
    }

    {
        println!("\n\n--------------------------------\n3rd run - \"some_field_0\" absent\n---------------------------------");
        let json_str = "{\"some_field_1\": \"There\"}".to_string();
        let obj_b: B = rustc_serialize::json::decode(&json_str).unwrap();

        println!("\nJSON: {}\nDecoded: {:?}", json_str, obj_b);
    }
}

and here's the output:

--------------------------------
1st run - all field present
--------------------------------

JSON: {"some_field_0": 1234, "some_field_1": "There"}
Decoded: B { some_field_0: Some(1234), some_field_1: Some("There") }


--------------------------------
2nd run - "some_field_1" absent
---------------------------------

JSON: {"some_field_0": 1234}
Decoded: B { some_field_0: Some(1234), some_field_1: None }


--------------------------------
3rd run - "some_field_0" absent
---------------------------------

JSON: {"some_field_1": "There"}
Decoded: B { some_field_0: None, some_field_1: None }

Notice that the third run produces an unexpected result. When the decoder fails to find some_field_0 it fails on all subsequent tokens, even though some_field_1 is present.

Seems like a bug in rustc-serialize. Consider filing an issue on [github](https://github.com/rust-lang-nursery/rustc-serialize) — aochagavia, Oct 29 '15 at 13:41

Shepmaster · Accepted Answer · 2015-10-29T14:50:45.667

6

There's something wrong with your Decodable implementation. Using the automatically-generated implementation works:

#[derive(Debug, RustcDecodable)]
struct B {
    some_field_1: Option<String>,
    some_field_0: Option<u64>,
}

JSON: {"some_field_1": "There"}
Decoded: B { some_field_1: Some("There"), some_field_0: None }

Using the generated implementation is the right thing to do if you can. If you cannot, here's the right implementation:

impl rustc_serialize::Decodable for B {
    fn decode<D: rustc_serialize::Decoder>(d: &mut D) -> Result<Self, D::Error> {
        Ok(B {
            some_field_0: try!(d.read_struct_field("some_field_0", 0, |d| rustc_serialize::Decodable::decode(d))),
            some_field_1: try!(d.read_struct_field("some_field_1", 0, |d| rustc_serialize::Decodable::decode(d))),
        })
    }
}

The important change is the use of try!. Decoding can fail. By using ok, you were saying that a failed decoding was actually a success, albeit a successful decoding of a None.

edited Oct 29 '15 at 14:50

answered Oct 29 '15 at 14:38

Shepmaster

388,571
95
1,107
1,366

i must have been drugged to think the generated implementation wouldn't work :D .. but if we do go with manual implementation you posted wouldn't `try!` cause an early return ? That is not what i wanted - i needed to put it as `None` and continue to inspect the next field. I haven't looked at that pretty-expanded version yet though to see what default does to make it work. – ustulation Oct 30 '15 at 09:42
Just as an attempt to answer my comment above: There must be a specialization for `Option` so that it can differentiate between decoding to a `T` for which it will bail out and not do anything further and `Option` for which it will be more lenient and return `None` if the field is not found but not bail out and reach EOF (but otherwise same behavior for any other error). Am i close there ? – ustulation Oct 30 '15 at 10:35

Unable to tackle optional fields in JSON with Rustc-serialize

1 Answers1