Need to read several .h5 files using loop with indices

Question

I have been given a crap ton of files I need to read over and graph. Some of the datasets have more files that other. So I need to write a loop that will do this. The code that I have written so far works but I cannot for the life of me figure out how to index the loop to call the files. I have tried a simple

i <-0 
for (i in files){
i <- i + 1
file_[i] <-H5fopen(files[i])
 }

I have tried other versions but I have changed it so much I can't remember. But when I load the files individually at the bottom. They load So I know that everything before the area where the loop is needed works.


library(data.table)
library(zoo)
library(Rtools)
library(tidyr)
library( dplyr)
library(scales)
library(readr)
library(tools)
library(plotly)
library(rootSolve)
library(stringr)
library(rhdf5)

library( ggplot2)
library(tokenizers)
library(readr)
library(tools)

library(stringr)
library(h5)
options(max.print=10000)
options(digits = 5, scipen = -2)



askUser_file_x <-0
while(askUser_file_x != "none"){

  askUser_file_x <- readline(prompt="Please, type on DS you wish to analyze, else type none (format DS##):   ")


  x <- str_extract(askUser_file_x, "(?<=DS)[0-9]+")
  x <-as.numeric(x) 
if (x==23){
  setwd("/H5files/08/14/")

  }else if  (x>=24 & x<=26){
    setwd("C:/H5files/08/15/")

  }else if  (x>=27 & x<=30){  
    setwd("C:s/H5files/08/21/")
  }else if  (x>=31 & x<=36){  
    setwd("C:/H5files/08/20/")

  }else if  (x>=01 & x<=06){
    setwd("C:/H5files/08/21/")
  }else if  (x>=37 & x<=44){
    setwd("C:/H5files/08/21/")
  }else if  (x>=07 & x<=18){
    setwd("C:/H5files/08/26/")
  }else if  (x>=19 & x<=22){
    setwd("C:/H5files/08/27/")
  }else if  (x>=45 & x<=46){
    setwd("C:/H5files/08/28/")
  }else if  (x>=47 & x<=50){
    setwd("C:/H5files/08/29/")

  }else
    print( "Data Set does not exist")



switch(askUser_file_x,
       DS01= (file <-paste0("TIVA_000",c(282:286))),
       DS03= (file <-paste0("TIVA_000",c(266:271))),
       DS04= (file <-paste0("TIVA_000",c(272:276))),
       DS05= (file <-paste0("TIVA_000",c(277:281))),
       DS06= (file <-paste0("TIVA_000",c(290:294))),
       DS07= (file <-paste0("TIVA_000",c(316:320))),
       DS08 =(file <-paste0("TIVA_000",c(321:325))),
       DS09 =(file <-paste0("TIVA_000",c(326:330))),
       DS10 =(file <-paste0("TIVA_000",c(331:335))),
       DS11 =(file <-paste0("TIVA_000",c(295:300))),
       DS12 =(file <-paste0("TIVA_000",c(301:305))),
       DS13 =(file <-paste0("TIVA_000",c(306:310))),
       DS14 =(file <-paste0("TIVA_000",c(311:315))),
       DS15 =(file <-paste0("TIVA_000",c(336:340))),
       DS16 =(file <-paste0("TIVA_000",c(341:345))),
       DS17 =(file <-paste0("TIVA_000",c(346:350))),
       DS18 =(file <-paste0("TIVA_000",c(351:355))),
       DS19 =(file <-paste0("TIVA_000",c(356:360))),
       DS20 =(file <-paste0("TIVA_000",c(361:365))),
       DS21 =(file <-paste0("TIVA_000",c(366:370))),
       DS22 =(file <-paste0("TIVA_000",c(371:375))),
       DS23 =(file <-paste0("TIVA_000",c(100:115))),
       DS24 =(file <-paste0("TIVA_000",c(116:121))),
       DS25a =(file<-paste0("TIVA_000",c(121:125))),
       DS25b =(file<-paste0("TIVA_000",c(126:130))),
       DS26 =(file<-paste0("TIVA_000",c(131:133))),
       DS27 =(file<-paste0("TIVA_000",c(161:163))),
       DS28a =(file<-paste0("TIVA_000",c(146:150))),
       DS28b =(file<-paste0("TIVA_000",c(156:160))),
       DS29 =(file<-paste0("TIVA_000",c(151:155))),
       DS30 =(file<-paste0("TIVA_000",c(156:160))),
       DS31 =(file<-paste0("TIVA_000",c(164:170))),
       DS32 =(file<-paste0("TIVA_000",c(171:175))),
       DS33 =(file<-paste0("TIVA_000",c(175:180))),
       DS34 =(file<-paste0("TIVA_000",c(181:188))),
       DS35 =(file<-paste0("TIVA_000",c(189:193))),
       DS36 =(file<-paste0("TIVA_000",c(194:198))),
       DS37 =(file<-paste0("TIVA_000",c(199:198))),
       DS38 =(file<-paste0("TIVA_000",c(211:220))),
       DS39a =(file<-paste0("TIVA_000",c(221:230))),
       DS39b =(file<-paste0("TIVA_000",c(231:240))),
       DS40 =(file<-paste0("TIVA_000",c(241:243))),
       DS41 =(file<-paste0("TIVA_000",c(244:245))),
       DS42a =(file<-paste0("TIVA_000",c(246:249))),
       DS43a =(file<-paste0("TIVA_000",c(250:252))),
       DS44 =(file<-paste0("TIVA_000",c(253:255)))

       )

    files <-paste0(file, ".h5")



 ## this is the part I need to loop some DS's have 3-5 files one has eleven

       file_1<- H5Fopen(files[1])  

       file_2<- H5Fopen(files[2])  

       file_3<- H5Fopen(files[3])

       file_4<- H5Fopen(files[4])

      file_5<- H5Fopen(files[5])

      h5ls(file_1)

``````
Edited for clarity

If I choose say DS23 then files becomes [1:6] TIVA_000116.h5 , TIVA_000117.h5, ... 
which is what I want, and if I run ``` file_1<- H5Fopen(files[1])``` it opens the files the way I expect it too. Is there a way  loop over files 
and create something like this 
````
file_1 <- H5Fopen(files[1])  # where file_1 take its name from  i = 1 and  
                             #H5Fopen(files[1]) is selected from the 1 element in the files vector
````
I want it to be able to select them because I have 45 Data sets of 5 files each that have to be created and converted. So I am trying to automate as much as possible. 
THanks in advance

What is `files`? You seem to be using it as a numeric vector `(i + i)` and as a vector (perhaps of file names in `i in files`). But why are you incrementing the index of a for loop? The whole point is that the for loop does that automatically? You don't need a loop. If files is a list of file names, `file <- lapply(files, H5fopen)` will load them all and store them in a list. — dcarlson, Nov 21 '19 at 05:26
Yes, lapply works to open each file individually. How would I be able to call each one individually if need be? Also some of the datasets have a different number of files that can be select some have 3, 4, 5 and 11. I need the code to be recognize that variation. — alittleloopy, Nov 21 '19 at 16:59
`file[[1]]` gets the first one, etc. I don't know the structure of `H5f` objects is but you would access the parts the using file[[1]] as the base name. If you want to name the parts something like `names(file) <- files` or perhaps `basename(files)` should do it. Most good tutorials on R basics discuss how to work with lists. — dcarlson, Nov 21 '19 at 18:26
for(i in seq_along(files)) { nam <- paste0("file_", i) assign(nam,files) } # this function assigns the variables like I want BUT the assign populates all the elements of files in each variable. So now I need to be able to separate each element from files and reassign it to the corresponding file_# — alittleloopy, Nov 22 '19 at 17:06

score 0 · Answer 1 · answered Dec 02 '19 at 22:17

This was the specific code that answered my needs for something that would open all files from the switch case regardless of how many of them there were.

data_subsets<- purrr::map2(H5_files, askUser_test_1,  h5read)
df <-purrr::map_dfc(data_subsets, "Data", H5Fopen)

`````

Need to read several .h5 files using loop with indices

1 Answers1