How to parse XML and put all the same named node values into an array

Question

I have XML I'm trying to parse and get the status of each of my heart beat tests using Nokogiri.

Here is my code:

xml = 
<a:HBeat>
  <a:ElapsedTime>3 ms</a:ElapsedTime>
  <a:Name>Service 1</a:Name>
  <a:Status>true</a:Status>
</a:HBeat>
<a:HBeat>
  <a:ElapsedTime>4 ms</a:ElapsedTime>
  <a:Name>Service 2</a:Name>
  <a:Status>true</a:Status>
  </a:HBeat>
<a:HBeat>

I have tried using both css and xpath to retrieve the value for each Status and put it into an array:

doc = Nokogiri::XML.parse(xml)
#service_state = doc.css("a:HBeat, a:Status", 'a' => 'http://schemas.datacontract.org/2004/07/OpenAPI.Entity').map {|node| node.children.text}
service_state = doc.xpath("//*[@a:Status]", 'a' => 'http://schemas.datacontract.org/2004/07/OpenAPI.Entity').map(&:text)

Both will return service_state = []

I have almost identical XML for another test and I used the following snippet of code which does exactly what I wanted but for some reason isn't working with the XML that contains namespaces:

service_state = doc.css("HBeat Status").map(&:text)

score 2 · Answer 1 · edited May 23 '17 at 12:25

Part of the problem is that your XML sample is not correct: You're missing the namespace declaration though you're using namespaces, and you're missing a containing tag. The first can be worked around but the second needs a tweak to the XML.

require 'nokogiri'
require 'pp'

xml = <<EOT
<xml xmlns:a="http://schemas.datacontract.org/2004/07/OpenAPI.Entity"> # <-- changed
  <a:HBeat>
    <a:ElapsedTime>3 ms</a:ElapsedTime>
    <a:Name>Service 1</a:Name>
    <a:Status>true</a:Status>
  </a:HBeat>
  <a:HBeat>
    <a:ElapsedTime>4 ms</a:ElapsedTime>
    <a:Name>Service 2</a:Name>
    <a:Status>true</a:Status>
    </a:HBeat>
  <a:HBeat>
</xml>
EOT

doc = Nokogiri::XML(xml)
service_state = doc.css('a|Status').map(&:text)      # <-- changed to show CSS with namespace
pp service_state

service_state = doc.search('//a:Status').map(&:text) # <-- added
pp service_state                                     # <-- added

>> ruby test.rb
>> ["true", "true"]
>> ["true", "true"]                                  # <-- added

Namespaces are a good thing, but dealing with them can be a pain when all you want to do is get at the data. Nokogiri has some tricks for making them less annoying, such as using CSS accessors like I did above, which means "find the Status tag in all namespaces", so even if the namespaces aren't declared it'll still be good.

If you are in control of the XML then you can do away with the namespaces. They're great when dealing with possible tag collisions but that's not too likely when you own the mechanism generating the file, so, if that's the case, you can probably do away with them. If you need the namespace, then it should be declared something like this:

<xml xmlns:a="http://schemas.datacontract.org/2004/07/OpenAPI.Entity">

Without it the XML parses with a lot of namespace errors:

(rdb:1) pp doc.errors
[#<Nokogiri::XML::SyntaxError: Namespace prefix a on HBeat is not defined>,
#<Nokogiri::XML::SyntaxError: Namespace prefix a on ElapsedTime is not defined>,
#<Nokogiri::XML::SyntaxError: Namespace prefix a on Name is not defined>,
#<Nokogiri::XML::SyntaxError: Namespace prefix a on Status is not defined>,
#<Nokogiri::XML::SyntaxError: Namespace prefix a on HBeat is not defined>,
#<Nokogiri::XML::SyntaxError: Namespace prefix a on ElapsedTime is not defined>,
#<Nokogiri::XML::SyntaxError: Namespace prefix a on Name is not defined>,
#<Nokogiri::XML::SyntaxError: Namespace prefix a on Status is not defined>,
#<Nokogiri::XML::SyntaxError: Namespace prefix a on HBeat is not defined>,
#<Nokogiri::XML::SyntaxError: Opening and ending tag mismatch: HBeat line 12 and xml>,
#<Nokogiri::XML::SyntaxError: Premature end of data in tag xml line 1>]

But after adding it the doc's error list is much smaller:

(rdb:1) pp doc.errors
[#<Nokogiri::XML::SyntaxError: Opening and ending tag mismatch: HBeat line 12 and xml>,
#<Nokogiri::XML::SyntaxError: Premature end of data in tag xml line 1>]

See "How to avoid joining all text from Nodes when scraping" also.

XML's still not right. (extra ``, also present in the original) — benizi, Dec 21 '10 at 23:05
Namespace is present in the actual XML. I wasn't clear in the original post that it was included but not in my XML snippet. At this point I don't have control over the namespace because there are several other consumers of the service. — r3nrut, Dec 22 '10 at 15:35
Nokogiri prefers that the XML be syntactically correct, but is robust enough to keep working if possible. So, while it wants to see the namespace, it's possible to work without it, as the lists of errors show. Though Nokogiri flagged problems it still let me get at the tags and their content. The example code should still work fine with the namespace declaration. That's one of the benefits of using CSS accessors over XPath when parsing using Nokogiri. — the Tin Man, Dec 22 '10 at 16:36
@r3nrut, I adjusted the code to reflect the missing XML namespace declaration. — the Tin Man, Dec 22 '10 at 17:15

score 2 · Accepted Answer · edited Dec 13 '19 at 00:03

2

In addition to Greg's response (that the XML needs a containing element), your XPath expression is selecting the wrong thing:

 //*[@a:Status]

selects all elements that have a:Status attributes. If you want all elements that have a child a:Status element, just remove the @ from the node test:

 //*[a:Status]

edited Dec 13 '19 at 00:03

the Tin Man

158,662
42
215
303

answered Dec 21 '10 at 23:17

benizi

4,046
5
28
25

I modified my XPath expression to omit the "@" but when I do that I'm getting everything back... so It looks like: 2 msService 1true 2 msService 2true Which would be awesome if it was spaced or delimited somehow. – r3nrut Dec 22 '10 at 15:40
doc.xpath() returns a list of elements that match. So you can do whatever you want with them. In your example, though, you've mapped the list to &:text, which just concatenates the text of all descendant elements. E.g. `doc.xpath("//*[a:Status]", # namespace).map { |node| # do something with node }` – benizi Dec 22 '10 at 17:32

How to parse XML and put all the same named node values into an array

2 Answers2