Json in C++: Parse a number as a string to avoid floating-point inaccuracy

Question

I'm dealing with a cryptocurrency RPC and receiving json data as follows:

{
  ...
  "amount": 1.34000000,
  "confirmations": 230016,
  "spendable": true,
  "solvable": true
  ...
}

Using Jsoncpp library or json11 gets the number parsed to a double. When this happens, the result is: 1.3400000000000001, due to double accuracy problems. In general this is catastrophic to financial transations and unacceptable.

I already have a fixed-point library that can take a valid string and treat it as an integer internally. Is there a way I could make Jsoncpp (or any other json library) take selected numeric json values as strings, so that I could treat them the right way with fixed-precision?

I'm not sure but you could probably run a pre-parse and change `"amount": 1.34000000` to `"amount": "1.34000000"` — NathanOliver, Jun 28 '18 at 16:46
@NathanOliver Sure. This would be a hack that I'd do finally. — The Quantum Physicist, Jun 28 '18 at 17:00
If you are storing the value as a double then this will always happen even if you parse the value as a string. Once you put it into a double there is a finite resolution on floating point. This is why financial applications don't use real numbers (they use integers or BCD (Binary coded decimals)). You could multiple by `100000000` and store as an integer. — Martin York, Jul 11 '18 at 06:03
@MartinYork You seem to misunderstand what's going on. Allow me to explain. Storing numbers after multiplying them by 10000000... is basically called "fixed-point precision", which is what I have and mentioned in the question (and which is how bitcoin, for example, operates, where the smallest transactable integer unit is called satoshi). I don't want to store in a double, ever. The problem is that I want to go from string directly to fixed-point precision without having to go through double, which all json libraries seem to be doing without my consent. — The Quantum Physicist, Jul 11 '18 at 06:45

score 2 · Accepted Answer · answered Jun 28 '18 at 20:42

There doesn't seem to be a solution in json libraries, so I had to modify the number myself and wrap it with quotes. I applied this function to the responses to do that.

[](std::string& jsonStr) {
        // matches "amount" field in json
        static std::regex reg(R"((\s*\"amount\"\s*:)\s*(\d*\.{0,1}\d{0,8})\s*)");
        jsonStr = std::regex_replace(jsonStr, reg, "$1\"$2\"");
    };

And now it works properly.

score 0 · Answer 2 · answered Jul 11 '18 at 08:14

I like ThorsSerializer. Disclaimer I wrote it.

It supports what you are looking for.
You can tell the parser to use the standard input/output operators for a class (which you can then define for yourself).

Example:

#include "ThorSerialize/JsonThor.h"
#include "ThorSerialize/SerUtil.h"
#include <sstream>
#include <iostream>
#include <string>
#include <map>

struct FixedPoint
{
    int     integerPart;
    int     floatPart;
    friend std::istream& operator>>(std::istream& stream, FixedPoint& data)
    {
        // This code assumes <number>.<number>
        // Change to suite your needs.
        char c;
        stream >> data.integerPart >> c >> data.floatPart;
        if (c != '.')
        {
            stream.setstate(std::ios::failbit);
        }

        return stream;
    }
};
// This declaration tells the serializer to use operator>> for reading
// and operator<< for writing this value.
// Note: The value must still conform to standard Json type
//       true/false/null/integer/real/quoted string
ThorsAnvil_MakeTraitCustom(FixedPoint);

struct BitCoin
{
    FixedPoint  amount;
    int         confirmations;
    bool        spendable;
    bool        solvable;
};
// This declaration tells the serializer to use the standard
// built in operators for a struct and serialize the listed members.
// There are built in operations for all built in types and std::Types
ThorsAnvil_MakeTrait(BitCoin, amount, confirmations, spendable, solvable);

Example usage:

int main()
{
    using ThorsAnvil::Serialize::jsonImport;
    using ThorsAnvil::Serialize::jsonExport;

    std::stringstream file(R"(
        {
            "amount": 1.34000000,
            "confirmations": 230016,
            "spendable": true,
            "solvable": true
        }
    )");

    BitCoin     coin;
    file >> jsonImport(coin);

    std::cout << coin.amount.integerPart << " . " << coin.amount.floatPart << "\n";
}

Build:

> g++ -std=c++1z 51087868.cpp -lThorSerialize17

score -1 · Answer 3 · answered Mar 11 '19 at 06:53

The native jsoncpp solution is to RTFM!!! (e.g., here: https://open-source-parsers.github.io/jsoncpp-docs/doxygen/class_json_1_1_stream_writer_builder.html)

Json::StreamWriterBuilder builder;
builder["commentStyle"] = "None";
builder["indentation"] = "   ";
builder["precision"] = 15;

That'll set your writer float precision to avoid printing the small truncation errors in the double representations. e.g., instead of a json field,

"amount": 1.3400000000000001,

you will now get

"amount": 1.340000000000000,

as desired.

Json in C++: Parse a number as a string to avoid floating-point inaccuracy

3 Answers3

Linked

Related