Serialize Polars `dataframe` to `serde_json::Value`

Question

In Polars, it is simple enough to serialize a DataFrame to a json string: JsonWriter::new(dest).finish(&df)?.

Is it possible to serialize to a json Value — a serde_json::Value — that is the Value::Array of records? Obviously, I could write the json to an in-memory string, then deserialize it into a serde_json::Value. But I'm wondering if there is a direct path from DataFrame to serde_json::Value.

This would be the rough equivalent of pandas df.to_dict(orient="records").

Are you looking for [`serde_json::to_value()`](https://docs.rs/serde_json/1.0.94/serde_json/fn.to_value.html)? — user2722968, Mar 07 '23 at 16:01
@user2722968 That won't help, because `DataFrame` does not implement `Serialize`. AFAICT (from looking through polars' [`json::write`](https://docs.rs/polars/latest/polars/export/arrow/io/json/write/index.html) module), there's no direct way to convert `DataFrame` to `serde_json::Value` beyond serializing it to a byte buffer and then using `serde_json::from_slice` for conversion to `serde_json::Value` — Jonas Fassbender, Mar 07 '23 at 16:28
@JonasFassbender I'm not familiar with Polars but afaics it does not support serde at all. If the object is small (and if you can live with the ugliness), serializing to a json-string and deserializing the string to a `serde_json::Value` seems to be the easiest way. Just don't tell anybody. The correct way would be to implement a `serialize_df() -> serde_json::Value` yourself using a custom [`Serializer`](https://serde.rs/impl-serialize.html) for whatever your `df` contains — user2722968, Mar 07 '23 at 17:53

score 1 · Accepted Answer · answered Mar 29 '23 at 16:24

You need to add the serde feature.

cargo add polars --features serde

use polars::prelude::*;

fn main() {
    let df = df! {
        "a" => [1,2,3,4,5],
    }
    .unwrap();
    let df_json = serde_json::to_value(&df).unwrap();
    println!("df_json {df_json}");
}

Note, Polars implements a custom de/serialize, and it may not align with the pandas equivalent mentioned.

The JSON result from above:

{
  "columns": [
    {
      "datatype": "Int32",
      "name": "a",
      "values": [1,2,3,4,5]
    }
  ]
}

Serialize Polars `dataframe` to `serde_json::Value`

1 Answers1