0

I have the following code:

library(tidyverse)
library(xml2)
xmlfile <- paste0("https://www.uniprot.org/uniprot/Q9NSI8.xml")
xml_doc <- read_xml(xmlfile)
xml_find_all(xml_doc, "//d1:reference") 
#> {xml_nodeset (12)}
#>  [1] <reference key="1">\n  <citation type="submission" date="2000-01" d ...
#>  [2] <reference key="2">\n  <citation type="journal article" date="2001" ...
#>  [3] <reference key="3">\n  <citation type="submission" date="2002-06" d ...
#>  [4] <reference key="4">\n  <citation type="journal article" date="2004" ...
#>  [5] <reference key="5">\n  <citation type="journal article" date="2000" ...
#>  [6] <reference key="6">\n  <citation type="journal article" date="2004" ...
#>  [7] <reference key="7">\n  <citation type="journal article" date="2001" ...
#>  [8] <reference key="8">\n  <citation type="journal article" date="2004" ...
#>  [9] <reference key="9">\n  <citation type="journal article" date="2009" ...
#> [10] <reference key="10">\n  <citation type="journal article" date="2011 ...
#> [11] <reference key="11">\n  <citation type="journal article" date="2013 ...
#> [12] <reference key="12">\n  <citation type="submission" date="2010-02"  ...

What I want to do is to convert the output of xml_find_all(xml_doc, "//d1:reference") into tibble. Resulting in this:

key type              date        ...    title        ... etc....
1   submission        2000-01            A novel gene, located on human chromosome 21q11. 
2   journal article   2001       

... etc ...

How can I do that?

littleworth
  • 4,781
  • 6
  • 42
  • 76

2 Answers2

1

1.- Select the nodes that contains the information you need (citation):

lapply(nodeset, function(x) xml_children(x)[1])

2.- Get the title (it is a value):

lapply(nodeset, function(x) c(Title = xml_text(xml_children(x)[1])))

3.- Get another informtion (its are attributes):

ll <- lapply(nodeset, 
       function(x) c(Title = xml_text(xml_children(x)[1]), 
                     xml_attrs(xml_children(x)[1])[[1]]))

4.- Get the data.frame:

You have two different structures in your xml, one of length 4 and another of length 7 (see lapply(ll, length)).

Length 7:

df1 <- as.data.frame(do.call(rbind, ll[unlist(lapply(ll, length)) == 7]))
str(df1)
'data.frame':   9 obs. of  7 variables:
 $ Title : Factor w/ 9 levels "Complete sequencing and characterization of 21,243 full-length human cDNAs.",..: 2 1 6 8 3 7 4 5 9
 $ type  : Factor w/ 1 level "journal article": 1 1 1 1 1 1 1 1 1
 $ date  : Factor w/ 6 levels "2000","2001",..: 2 3 1 3 2 3 4 5 6
 $ name  : Factor w/ 9 levels "Biochem. Biophys. Res. Commun.",..: 9 7 8 4 1 5 2 3 6
 $ volume: Factor w/ 9 levels "10","12","14",..: 4 7 8 3 6 5 1 9 2
 $ first : Factor w/ 9 levels "137","17","2121",..: 8 6 5 3 1 9 7 2 4
 $ last  : Factor w/ 9 levels "141","17","2127",..: 8 6 5 3 1 9 7 2 4

Length 4:

df2 <- as.data.frame(do.call(rbind, ll[unlist(lapply(ll, length)) == 4]))
str(df2)
'data.frame':   3 obs. of  4 variables:
 $ Title: Factor w/ 3 levels "A novel gene, located on human chromosome 21q11.",..: 1 3 2
 $ type : Factor w/ 1 level "submission": 1 1 1
 $ date : Factor w/ 3 levels "2000-01","2002-06",..: 1 2 3
 $ db   : Factor w/ 2 levels "EMBL/GenBank/DDBJ databases",..: 1 1 2

Add the id:

ll <- lapply(nodeset, 
             function(x) c(Title = xml_text(xml_children(x)[1]), 
                           xml_attrs(xml_children(x)[1])[[1]],
                           id = try(xml_attr(xml_child(xml_children(x)[1], 3), "id"))))

df1 <- as.data.frame(do.call(rbind, ll[unlist(lapply(ll, length)) == 8]))
str(df1)
'data.frame':   9 obs. of  8 variables:
 $ Title : Factor w/ 9 levels "Complete sequencing and characterization of 21,243 full-length human cDNAs.",..: 2 1 6 8 3 7 4 5 9
 $ type  : Factor w/ 1 level "journal article": 1 1 1 1 1 1 1 1 1
 $ date  : Factor w/ 6 levels "2000","2001",..: 2 3 1 3 2 3 4 5 6
 $ name  : Factor w/ 9 levels "Biochem. Biophys. Res. Commun.",..: 9 7 8 4 1 5 2 3 6
 $ volume: Factor w/ 9 levels "10","12","14",..: 4 7 8 3 6 5 1 9 2
 $ first : Factor w/ 9 levels "137","17","2121",..: 8 6 5 3 1 9 7 2 4
 $ last  : Factor w/ 9 levels "141","17","2127",..: 8 6 5 3 1 9 7 2 4
 $ id    : Factor w/ 9 levels "10830953","11536050",..: 2 4 1 6 3 5 7 8 9
Juan Antonio Roldán Díaz
  • 2,017
  • 8
  • 22
1

Try this workaround purrr::map_df(~as.list(.)). In your case, 

tmp <- xml_doc %>%
  xml2::xml_find_all("//d1:reference")

key <- tmp %>%
  xml2::xml_attrs() %>%
  purrr::map_df(~as.list(.))

ref <- tmp %>%
  xml2::xml_children() %>%
  xml2::xml_attrs() %>%
  purrr::map_df(~as.list(.))

and then merge them with dplyr::bind_cols(key, ref). Hope this helps.

Kentaroh Toyoda
  • 11
  • 1