How to make serde deserialize BigInt as u64?

Question

I'm using toml and num-bigint with the serde feature to deserialize the following data:

[[trades]]
action = "Buy"
date_time = 2019-04-15T15:36:00+01:00
fee = [1, [44000000]]
id = "#1"
price = [-1, [20154500000]]
quantity = [1, [200000000]]

But I'm getting this error:

Error: Error { inner: ErrorInner { kind: Custom, line: Some(7), col: 14, at: Some(156), message: "invalid value: integer `20154500000`, expected u32", key: ["trades", "price"] } }

Of course, if I make the price value smaller than u32::MAX, the program compiles fine. But I want to use this high value, because I'm scaling numbers by 1e8 to avoid dealing with floating-point arithmetic.

Is it possible to make serde deserialize BigInts to u64 instead?

use num_bigint::BigInt;
use serde_derive::Deserialize;
use toml::from_str;
use toml::value::Datetime;

#[derive(Debug, Deserialize)]
pub struct Trade {
    pub action: String,
    pub date_time: Datetime,
    pub fee: BigInt,
    pub id: Option<String>,
    pub price: BigInt,
    pub quantity: BigInt,
}

#[derive(Debug, Deserialize)]
pub struct TradeTable {
    pub trades: Vec<Trade>,
}

fn main() {
    let trades_string: String = String::from("[[trades]]\naction = \"Buy\"\ndate_time = 2019-04-15T15:36:00+01:00\nexchange = \"Degiro\"\nfee = [1, [44000000]]\nid = \"#1\"\nprice = [-1, [20154500000]]\nquantity = [1, [200000000]]");
    let trade_table: TradeTable = from_str(&trades_string).unwrap();
    let trades: Vec<Trade> = trade_table.trades;
}

Also, here's a link to a Rust Playground. Note that you will need to copy the code to your local machine, because you need the serde feature from num-bigint:

Cargo.toml

[dependencies.num-bigint]
version = "0.2.6"
features = ["serde"]

How are you deserializing the data? Could you please provide some example code? There are different ways to deserialize this toml fragment, and we can only tell you what's wrong with your approach if you show us what you are doing. — Sven Marnach, Mar 13 '20 at 13:39
Data in a Toml file is not a `BigInt`, it's just text. You can try to deserialize it as a `BigInt` if you want, but that will only work if you use the expected format. How did you create this file -- did you make it by serializing a `BigInt`, or did you write it by hand? — trent, Mar 13 '20 at 13:51
Sorry guys let me update the question body with a link to a Rust playground — Paul Razvan Berg, Mar 13 '20 at 14:01
@Paul Thanks for the update. Please do include the code in the question itself as well as linking to it. Am I correct in assuming that most of the example is extraneous and you get the same error with just this single line: `let price: BigInt = toml::from_str("[-1, [20154500000]]").unwrap();`? (I can't test it locally at the moment.) — trent, Mar 13 '20 at 14:16
I'm new to Rust, so I'm not sure how the program behaves when the data is deserialized from an array vs just a string containing one BigInt. I ran your snippet and it returned a different compilation error: `kind: Wanted { expected: "a right bracket", found: "a comma" }` — Paul Razvan Berg, Mar 13 '20 at 14:21
hmm, my mistake, I guess you can't do that in Toml. Regardless I'm betting the mistake is that you manually created a Toml file that doesn't contain a valid `BigInt` (which seems to be encoded as a sign plus an array of `u32`-sized integers). — trent, Mar 13 '20 at 14:24
@trentcl yeah that‘s how it has to be defined, as a tuple, otherwise I'm getting this error: `invalid type: integer `44000000`, expected a tuple of size 2` — Paul Razvan Berg, Mar 13 '20 at 14:30
[can't reproduce](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=499cd97ed924d90373a2ad5951b039b2) vote to close — Stargateur, Mar 13 '20 at 17:59
@Stargateur you can't reproduce because you need the "serde" feature and I couldn't see any way to add a feature to an installed crate in the rust playground — Paul Razvan Berg, Mar 13 '20 at 18:18

score 3 · Answer 1 · edited Jun 20 '20 at 09:12

3

How did you create this file -- did you make it by serializing a BigInt, or did you write it by hand?

I wrote the data by hand.

Your data is invalid, the following work:

use num_bigint::BigInt;
use std::str::FromStr;

#[derive(Debug, serde::Serialize, serde::Deserialize, PartialEq)]
pub struct Trade {
    pub price: BigInt,
}

fn main() {
    let price = BigInt::from_str("-201545000000000").unwrap();
    let trade = Trade { price };

    let j = serde_json::to_string(&trade).unwrap();
    println!("{}", j);
    let a: Trade = serde_json::from_str(&j).unwrap();
    assert_eq!(trade, a);

    let j = toml::to_string(&trade).unwrap();
    println!("{}", j);
    let b: Trade = toml::from_str(&j).unwrap();
    assert_eq!(trade, b);
}

You are not supposed to create it by hand.

edited Jun 20 '20 at 09:12

Community

1
1

answered Mar 13 '20 at 18:04

Stargateur

24,473
8
65
91

Well, I'm writing a Rust program for my own consumption. Why are we not supposed to create, or provide, our own toml files? – Paul Razvan Berg Mar 13 '20 at 18:17
@PaulRazvanBerg well stock it as string so. this is not mean to be use outside rust. the format of bigint is not specified so... (or I miss it) – Stargateur Mar 13 '20 at 18:34
@Paul You can provide your own toml file, but if you want to deserialize it as a `BigInt`, it has to contain a valid `BigInt`, right? I mean, you wouldn't expect `"hello, world"` to deserialize successfully as a `BigInt`, so why should `[-1, [20154500000]]`? If you want to know the right way to represent`-20154500000` as a `BigInt` in a Toml file, I suggest serializing it (as Stargateur's program does) and looking at the result. – trent Mar 13 '20 at 18:41
Yeah `20154500000` in serialised form seems to be `[1, [2974630816, 4]]`. I'll try to find an alternative solution to my problem. Thanks both for your help! – Paul Razvan Berg Mar 13 '20 at 19:14

How to make serde deserialize BigInt as u64?

Cargo.toml

1 Answers1