Boost.PropertyTree doesn't release memory in destructor

Question

I test the following code

#include <boost/property_tree/xml_parser.hpp>

namespace pt = boost::property_tree;

int main()
{
  { //1
    pt::ptree xml;
    pt::read_xml("very_big_xml.xml", xml);
    // xml.clear(); //destructor should do the job
  }
  { //2
    pt::ptree xml;
    pt::read_xml("very_big_xml.xml", xml);
  }
}

After first part, my application ocuppies huge memory amount (~6GB) and it crashes on second part because of lack of memory. Should I manually release property tree? Documentation says the destructor should clean up.

NOTE: uncommenting xml.clear(); doesn't help.

Have you tried the `std::vector().swap(other)` trick? I would try swapping `xml` with empty temporary just to see what happens. — Big Temp, May 19 '20 at 08:52
@BigTemp I've tried. But this is literally the same as calling destructor. — Mariusz Jaskółka, May 19 '20 at 09:06
@rustyx boost 1.65, g++ 7.5.0 on Linux, no additional flags. — Mariusz Jaskółka, May 19 '20 at 09:20

Marek R · Answer 1 · 2020-05-19T10:06:33.783

0

Problem is that you are trying use DOM for very large xml.

DOM is vary handy and easy to understand, but it fails for large xml-s.

Proper way to solve this problem is to use SAX which is event driven. Using this approach you can process large xml even in 32 bit application without consuming large chunks of memory.

Now boost property tree is clearly DOM style and it fails not because ptree is broken, but this is shortcoming of DOM approach.

Most probably you built your application for 32 bit platform, which makes application to have 3GB memory limit.

edited May 19 '20 at 10:06

answered May 19 '20 at 09:30

Marek R

32,568
6
55
140

You are right - SAX is better for handling very large xmls. But this is not the answer to the question, strictly speaking. – Mariusz Jaskółka May 19 '20 at 10:08

score 0 · Answer 2 · answered May 20 '20 at 14:15

Works fine for me, boost::property_tree destructor is releasing memory as expected (Boost 1.73).

Try running under gdb and getting a backtrace (bt) when the app crashes.

Also try running under valgrind with a smaller file to see if it reports any leaks.

Boost.PropertyTree doesn't release memory in destructor

2 Answers2