for loop and downloads with multiple urls

Question

Hi I was referring to the question here - Downloading multiple files using "download.file" function in R

But I couldn't find the answer I was looking for. I want to download data from multiple urls and I am using the following code:

I am trying to do something similar and am relatively new at R. Below is my code:

temp <- tempfile(pattern = "my", fileext = ".txt")    #my is a vector in YYMM form

masterfile = as.data.frame(NULL)

for(i in 1:length(my)) {

  download.file(url = paste0("http://www2.census.gov/econ/bps/Metro/ma", "my[i]", "c.txt"), destfile = paste0("/Users/shashankrai/GitHub/data-science/homeworks/homework1/","my[i]","c.txt"), mode = wb)

temp <- read.table(paste0("/Users/shashankrai/GitHub/data-science/homeworks/homework1/","my[i]","c.txt"), sep = ",", skip = 3)[, c(1,3,5)]

masterfile <- rbind(masterfile, temp)

}

It throws up the following error:

curl: (3) [globbing] bad range in column 44

Error in file(file, "rt") : cannot open the connection

In addition: Warning messages:

1: In download.file(url = paste0("http://www2.census.gov/econ/bps/Metro/ma", : download had nonzero exit status

2: In file(file, "rt") : cannot open file '/Users/shashankrai/GitHub/data-science/homeworks/homework1/my[i]c.txt': No such file or directory

Could you tell me what I am doing wrong?

I have also tried this:

temp <- tempfile(pattern = "my", fileext = ".txt")

masterfile = as.data.frame(NULL)

for(i in 1:length(my)) {

  download.file(url = paste0("url", "my[i]", "c.txt"), destfile = my[i], mode = wb)

  temp <- read.table(my[i], sep = ",", skip = 3)[, c(1,3,5)]

  masterfile <- rbind(masterfile, temp)

}

`"my[i]"` should be `my[i]` – MichaelChirico Feb 07 '17 at 16:00 — MichaelChirico, Feb 07 '17 at 16:00
Thanks Michael. Yeah, I noticed that. :) – shashank rai Feb 07 '17 at 16:06 — shashank rai, Feb 07 '17 at 16:06

score 0 · Answer 1 · edited Feb 07 '17 at 17:11

Nevermind. Figured it out. This is a code that finally worked:

masterfile = as.data.frame(NULL)

for(i in 1:length(my)) {

  temp <- tempfile(pattern = "my", fileext = ".txt")

  download.file(url = paste0("http://www2.census.gov/econ/bps/Metro/ma",     my[i], "c.txt"), destfile = temp[i], mode = wb)

  masterfile <- rbind(masterfile, read.table(file = temp[i], sep = ",", skip = 3)[, c(1,3,5)])

}

for loop and downloads with multiple urls

1 Answers1