2

I am using XML::Simple to parse and edit a very large XML file, and speed is essential (so far of all the method's I have tried XML::Simple has been the fastest)

Now once all my edits are completed I print the XML to a document using XMLout(), though it prints it with proper indentation which is nice if this was read by humans but is completely useless in my situation.

The output file without white space is 1.2 Mb with white space it is 15 Mb.

I have been using:

my $string = XMLout($data);
$string =~ s/>[\s]*</></g;
print $out $string;

But it seems to not only be an extreme CPU hog and takes an enormous amount of memory to do.

Is their a way to simply output my XML object as proper XML without all the useless white space?

Thanks

Sinan Ünür
  • 116,958
  • 15
  • 196
  • 339
Alex Angelini
  • 459
  • 1
  • 4
  • 12

3 Answers3

8

Look at NoIndent option: From XML::Simple manpage:

NoIndent => 1 # out - seldom used

Set this option to 1 to disable "XMLout()"’s default ’pretty printing’ mode. With this option enabled, the XML output will all be on one line (unless there are newlines in the data) - this may be easier for downstream processing.

NormaliseSpace => 0 │ 1 │ 2 # in - handy

This option controls how whitespace in text content is handled. Recognised values for the option are:

  • 0 = (default) whitespace is passed through unaltered (except of course for the normalisation of whitespace in attribute values which is mandated by the XML recommendation)

  • 1 = whitespace is normalised in any value used as a hash key (normalising means removing leading and trailing whites- pace and collapsing sequences of whitespace characters to a single space)

  • 2 = whitespace is normalised in all text content

    Note: you can spell this option with a ’z’ if that is more natural for you.

jira
  • 3,890
  • 3
  • 22
  • 32
2

Just set the NoIndent option in the call to XMLout(). Like this:

my $string = XMLout($data, NoIndent=>1);

Tada!

dyfrgi
  • 21
  • 2
0

An event-driven XML parser is going to be faster than something that needs to load the whole things into memory at once.

You shouldn't do so much extra work in your pattern! Try this instead:

$string =~ s/>\s+</></g;
tchrist
  • 78,834
  • 30
  • 123
  • 180
  • I have read that event-driven would be much better but I am trying to store decision tree counter into a file (basically trying to monitor what path a user takes through an application) and I need to be able to easily and quickly check an element's children... Which I do not think is easy with a stream parser. (Please correct me if I am wrong) – Alex Angelini Nov 08 '10 at 19:50
  • No, I do not think you are wrong about having troubles with determining children with a stream parser. But I’ve only ever used `XML::Simple` myself. To reduce the size of a very large stream, I have however found reasonable record boundaries to break up the input stream at so it is not so large in memory all at once. – tchrist Nov 08 '10 at 19:55
  • If the whole document does fit in memory, then using XML::LibXML will be faster than most other options. This is because the DOM data structures are managed by C code and usually not much data crosses the C <-> Perl API. On the other hand, the SAX API requires the creation and destruction of numerous Perl data structures for each event. SAX is one of those things that's mostly good in theory. If the whole document does not fit in memory then XML::LibXML could still be a good option if you use the 'pull' API provided by XML::LibXML::Reader (not that I've ever done that). – Grant McLean Nov 10 '10 at 03:38