1

I have the following issue. I have imported multiple csv files into my working directory. Would appreciate any help

files
[1] "sept2010.csv__001.csv" "sept2010.csv__002.csv" "sept2010.csv__003.csv" "sept2010.csv__004.csv""sept2010.csv__005.csv" "sept2010.csv__006.csv"

Here I have more than 200 csv files. If I wand to open the files I can do it with 

data<-rbind(sept2010.csv__001.csv,sept2010.csv__002.csv) # It is time consuming to rbing 200 files.

When I try to open the files with :

myfiles = do.call(rbind, lapply(files, function(x) read.csv(x, stringsAsFactors = FALSE)))

I got an error message:

Error in file(file, "rt") : cannot open the connection

When I try the following:

 data<-do.call("rbind", lapply(files, read.csv, header = TRUE))

I get the same error message

If I try to open the files manually with:

folder <- "C:/Users/NewPap/Desktop/DATA/test"     
file_list <- list.files(path=folder, pattern="*.csv")
for (i in 1:length(file_list)){
  assign(file_list[i], 
     read.csv((paste(folder, file_list[i], sep='')))
   )}

I get the same error

I am not sure what did I do wrong. Would appreciate any help

kelamahim
  • 577
  • 1
  • 4
  • 21
  • 2
    There's not enough for us to help you. However, if you used `list.files("path/")` to find files in a different directory, I occasionally forget to add `list.files(..., full.names=TRUE)` to get the path included with the filename. – r2evans Sep 01 '17 at 08:02
  • I think the problem is in the corporate computer and the administration rights. This is the reason why it is impossible to open the connection – kelamahim Sep 01 '17 at 08:07
  • Ok ... so can you open the files manually? There's nothing R can do if the OS I'd denying you access. – r2evans Sep 01 '17 at 08:09
  • I can open them but just with rbind. Is there any other faster way of doing it? – kelamahim Sep 01 '17 at 08:10
  • `rbind` doesn't open files, it just puts together its arguments, forming a larger matrix or data frame. – Rui Barradas Sep 01 '17 at 08:31
  • So what is my only option for putting together all the files in one data frame? – kelamahim Sep 01 '17 at 08:34
  • I really don't want to manually rbind 200 files – kelamahim Sep 01 '17 at 08:35
  • It is not clear whether you can open even a single of your 200 files. Does the command `read.csv("sept2010.csv__001.csv")` return an error? – Vincent Guillemot Sep 01 '17 at 08:57
  • No it is working fine and it opens the file – kelamahim Sep 01 '17 at 09:12
  • OK, great. Now when you run the last solution you gave us (with the `for` loop), which file returns an error? In other words, what is the value of `file_list[i]` after you get the error? – Vincent Guillemot Sep 01 '17 at 09:15
  • Why I ask this question: I got some kind of similar error when I had a file opened in Excel (e.g.) and I wanted to open it with R. – Vincent Guillemot Sep 01 '17 at 09:18
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/153442/discussion-between-vincent-guillemot-and-kelamahim). – Vincent Guillemot Sep 01 '17 at 09:18

2 Answers2

1

If all your files are in your working directory, then 

lapply(grep(".csv",list.files(full.names=T),value="TRUE"),read.csv)

should open all the CSV files in a list (each file content will be in an element of the list).

If all CSV files have the same number of column, then 

do.call("rbind",lapply(grep("csv",list.files(full.names=T),value="TRUE"),read.csv))

will produce a single dataframe with all CVS files.

xraynaud
  • 2,028
  • 19
  • 29
  • do you see your files when you type file.list() ? Are you sure that you have the permissions to open the files (which OS are you on ?) ? – xraynaud Sep 01 '17 at 12:32
  • list.files() yes I can see the data, and I'm on windows – kelamahim Sep 01 '17 at 12:37
  • If you open the directory with a file explorer and ask for file info. Do you have read permissions on the files you want to open ? – xraynaud Sep 01 '17 at 12:40
  • Yes I have. The basic problem is that I can easily open and read the files with rbind but when I try to open it all together is not working – kelamahim Sep 01 '17 at 12:43
  • https://codereview.stackexchange.com/questions/171309/the-fastest-way-to-import-and-edit-data-in-r This is almost the identical approach I'm using for opening the files – kelamahim Sep 01 '17 at 12:43
  • Finally is working! How do I remove the first 5 rows of each data? I can import the data but before making the whole dataframe I need to remove the headers on each data – kelamahim Sep 01 '17 at 13:06
1
folder <- "C:/Users/NewPap/Desktop/DATA/test"     
files <- list.files(path=folder, pattern="*.csv")

Try this:

data = Map(f = read.csv,files,header=T)
Reduce(function(x,y){rbind(x,y)},data)
tushaR
  • 3,083
  • 1
  • 20
  • 33
  • @kelamahim Then it's permission issue. Just try to read one file first using `read.csv(files[1],header=T)`. Does that throw an error? – tushaR Sep 01 '17 at 09:15