Can't parse JSON from StackAPI: expected value", line: 1, column: 1

Question

I'm trying to parse the JSON from this endpoint: https://api.stackexchange.com/2.2/users/13029516?&site=stackoverflow.

It looks like this:

{
"items": [
{
"badge_counts": {
"bronze": 27,
"silver": 14,
"gold": 0
},
"account_id": 17932441,
"is_employee": false,
"last_modified_date": 1677242400,
"last_access_date": 1682981080,
"reputation_change_year": 313,
"reputation_change_quarter": 29,
"reputation_change_month": 0,
"reputation_change_week": 10,
"reputation_change_day": 0,
"reputation": 2234,
"creation_date": 1583703647,
"user_type": "registered",
"user_id": 13029516,
"website_url": "https://matthewtrent.me/links",
"link": "https://stackoverflow.com/users/13029516/matthew-trent",
"profile_image": "https://lh3.googleusercontent.com/a/AATXAJwHVeuBCVo52xa3k_IryQ1llGDNWkHRYiO0R2xh=k-s256",
"display_name": "Matthew Trent"
}
],
"has_more": false,
"quota_max": 300,
"quota_remaining": 271
}

but I can't. I'm using the reqwest package. Here is my endpoint where I'm using it (simplified):

#[get("/test")]
async fn my_function(username: web::Query<QueryParams>) -> impl Responder {
    println!("recieved!");

    match reqwest
        ::get("https://api.stackexchange.com/2.2/users/13029516?&site=stackoverflow")
    .await
    {
        Ok(response) => {
            if response.status() != 200 {
                return HttpResponse::InternalServerError().finish();
            } else {
                let res = response.json::<serde_json::Value>().await.unwrap(); // <-- where it panics
                println!("Success! {:?}", res);
            }
        }
        Err(_) => return HttpResponse::InternalServerError().finish(),
    };
    HttpResponse::Ok().finish()
}

It panics at the line let res = response.json::<serde_json::Value>().await.unwrap();. What's odd is that I've hit different endpoints (ex: https://pokeapi.co/api/v2/pokemon/ditto) and it works perfectly and lists out the JSON results. My only guess it that the Stack API isn't properly formatted JSON?

Here's the output for when I run it against https://api.stackexchange.com/2.2/users/13029516?&site=stackoverflow:

recieved!
thread 'actix-rt|system:0|arbiter:0' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }', src/services/stack_overflow.rs:42:70
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
recieved!
thread 'actix-rt|system:0|arbiter:1' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }', src/services/stack_overflow.rs:42:70
recieved!
thread 'actix-rt|system:0|arbiter:2' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }', src/services/stack_overflow.rs:42:70
recieved!
thread 'actix-rt|system:0|arbiter:3' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }', src/services/stack_overflow.rs:42:70
recieved!
thread 'actix-rt|system:0|arbiter:4' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }', src/services/stack_overflow.rs:42:70
recieved!
thread 'actix-rt|system:0|arbiter:5' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }', src/services/stack_overflow.rs:42:70
recieved!
thread 'actix-rt|system:0|arbiter:6' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }', src/services/stack_overflow.rs:42:70
recieved!
thread 'actix-rt|system:0|arbiter:7' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }', src/services/stack_overflow.rs:42:70
recieved!
thread 'actix-rt|system:0|arbiter:0' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Decode, source: Error("expected value", line: 1, column: 1) }', src/services/stack_overflow.rs:42:70

Any help would be great, thanks.

1

You should save the response as a String (using `.text()`), print it out, and then try parsing it as json so you can see what character is at line 1 col 1. — drewtato, May 02 '23 at 02:30
I tried that and it’s complete gibberish text/characters @drewtato — Matthew Trent, May 02 '23 at 02:31
Try enabling the `gzip` feature for `reqwest`, since stackexchange always returns compressed data: https://api.stackexchange.com/docs/compression — drewtato, May 02 '23 at 02:34
p.s. there's also a `deflate` feature but either one might work — drewtato, May 02 '23 at 02:36
@drewtato That worked! Feel free to add it as an answer and I'll check it. — Matthew Trent, May 02 '23 at 03:54

score 1 · Accepted Answer · answered May 02 '23 at 04:57

This fails because the Stack Exchange API does not follow the HTTP spec exactly.

While our API is HTTP based, we chose to diverge from standard HTTP in one particular area. During normal operation, we guarantee that all responses are compressed, either with GZIP or DEFLATE.

If you inspect the request and response headers, you will see that the API returns a Content-Encoding: gzip despite reqwest not sending an Accept-Encoding containing gzip.

Reqwest decides to ignore the unknown encoding and try to read it anyway, resulting in junk data.

Fortunately, you can enable the gzip feature of reqwest so that it can both send the Accept-Encoding: gzip header and decode GZIP-encoded responses. There is also a deflate feature if you would like to use DEFLATE.

[dependencies]
reqwest = { verson = "0.11", features = ["json", "gzip", "deflate"] }

You can also turn off this behavior at runtime with the ClientBuilder::gzip method.

Can't parse JSON from StackAPI: expected value", line: 1, column: 1

1 Answers1