performance is very bad to construct boost property tree from json file?

Question

I am using boost property tree to load/dump json file. However, the performance is very bad.

For example, I have a json file whose size is 1.8M. The boost C++ program spends 3 seconds to load the json file and construct the property tree. If I use python to load the json file, it only need 0.1 second. And python will also construct everything as object as well.

The C++ program is like:

int main(int argc, char **argv){
        std::fstream fin;
        fin.open(argv[1], std::fstream::in);
        if (!fin.is_open()){
            ASSERT(false);
        }

        boost::property_tree::ptree pt;
        try{
            read_json(fin, pt);
        }catch(ptree_error & e) {
            ASSERT(false);
        }
        fin.close();

    return 0;
}

The python script which is doing same thing is like:

#!/usr/bin//python

import sys
import json

fp = open(sys.argv[1],"r")
objs = json.load(fp)

I tried the lastest boost (1.54). It's still very slow on doing this.

Appreciate for any advice.

If there is no solution, do you know any other C++ library to load/dump json?

How did you compile your C++ program? Did you turn on optimization? — filmor, Aug 07 '13 at 11:14
have you profiled to see where the C++ program spends most of its time? Even something as simple as `strace` might show some potential bottlenecks. — Sam Miller, Aug 07 '13 at 21:26
Yes, I actually have a profile generated by collect. But I didn't understand it much yet. https://dl.dropboxusercontent.com/u/623030/boost_json.collect.incl.txt — limi, Aug 07 '13 at 22:50
Would you mind to provide the file you are trying to parse (or a similar one)? — filmor, Aug 08 '13 at 08:08

score 3 · Answer 1 · answered Jun 12 '15 at 08:05

We had significant performance problems with boost::property_tree and JSON. Our approach was to stop using std::string and use an in-house string class with a custom allocator, and hash tables for not reallocating the same string twice. This improved performance and memory usage by at least a few orders of magnitude for large JSON files.

Our JSON files were large enough that the std::string allocation consumed all available address space on a 32-bit machine. This approach let us run with headroom.

score 1 · Answer 2 · answered Jun 12 '15 at 07:57

1

I found that there is a huge difference between Release Build vs Debug Build performance numbers from VS for Property Tree. on my specific hardware a parsing through a 1 MB JSON File using read_json was taking 8 sec in Debug build , but only 0.7 sec in release version.

answered Jun 12 '15 at 07:57

pv.

484
1
4
9

1

Great tip! I have the same results here, release performs just fine, saved me from going off to find another solution. – hyperspasm Dec 10 '15 at 01:19

score 0 · Accepted Answer · answered Jan 17 '14 at 04:38

0

It doesn't matter much what's really in the JSON file. I tried multiple JSON files with different conent. Boost is just slow.

Now I already switched to jansson which is much better - fast and nice API to use.

answered Jan 17 '14 at 04:38

limi

832
7
19

It is probably `std::string` that is causing your problem rather than boost. – janm Jun 12 '15 at 08:06

performance is very bad to construct boost property tree from json file?

3 Answers3