0

I have 160 text files all with the same data columns, but no headers. There is no delimiter so I have to use read.fwf, I can read one in at a time using this code:

myfile= "//PATH/AllFiles/10914_1Mile_TextFile.txt"

read_fwf(myfile, fwf_cols(NUM=5,YNUM=20,STREETNUM=8,STREETPRED=2,STREETNAME=20,STREETTYPE=8,STREETPOSTD=2,STREETADD2=40,CITY=29,STATE=2,ZIP=5,ZIP4=4,EFFDATE=8,TYPE=1,
                                          IUM=6,FILL=1,VAMOUNT=9,FILLS=1,OVAMOUNT=9))

But doing this 160 times is not ideal, so I am trying to get them all in at once to bind them. This is my code so far:

CompleteDataCollection <- do.call( "rbind", lapply(myfile, function(fn) 
  data.frame(Filename=fn, read.fwf(fn,
                                   widths = c(5,20,8,2,20,8,2,40,29,2,5,4,8,1,6,1, 9,1,9),

                                   header = FALSE,
                                   col.names = c("NUM","YNUM","STREETNUM","STREETPRED","STREETNAME","STREETTYPE","STREETPOSTD","STREETADD2","CITY",
                                                  "STATE","ZIP","ZIP4","EFFDATE","TYPE", "PREMIUM","FILL", "VAMOUNT","FILLS","OVAMOUNT"))
  )))

I keep getting errors and fn is not set to anything, should it be? Any suggestions are appreciated and welcomed thank you in advance.

Brian Gal
  • 13
  • 2

1 Answers1

0

fn should be a list with all the file names in your directory. You could save a list of all file names when your working directory is set to the directory in which the text files are in with:

fn <- dir()

Then execute lapply on fn.

CompleteDataCollection <- lapply(fn, function(x) read.fwf(fn,
                               widths = c(5,20,8,2,20,8,2,40,29,2,5,4,8,1,6,1, 9,1,9),

                               header = FALSE,
                               col.names = c("NUM","YNUM","STREETNUM","STREETPRED","STREETNAME","STREETTYPE","STREETPOSTD","STREETADD2","CITY",
                                              "STATE","ZIP","ZIP4","EFFDATE","TYPE", "PREMIUM","FILL", "VAMOUNT","FILLS","OVAMOUNT"))
ReelSaemon
  • 33
  • 6
  • Thank you, i get a new error which is progress Error in file(file, "rt") : invalid 'description' argument – Brian Gal Apr 29 '20 at 17:22
  • I cannot reproduce this issue if I run the code above. If the directory is filled with text files there are no errors for me. Though your error seems to be a prominent issue when reading in data. Maybe you could take a look at this if it helps: https://github.com/alyssafrazee/ballgown/issues/111 It may not be a perfect fit for your question but maybe you find some info there. It could just be a working directory problem so make sure your working directory is set to the directory in which the files are located. – ReelSaemon Apr 30 '20 at 21:11