In PHP, how to download multiple files of same type with cURL from web directory?

Question

I'm new to PHP, however, I've got a task which requires the widely used cURL function. Via PHP, I want to download all XML files from here to a specific directory on my own server (the PHP file will be executed on the same server as well).

I've found lots of explanations of how to download a particular file with a fixed URL, but not all files of the same type from a web directory.

You would normally use FTP to download a directory of files. Can you get FTP access? — KIKO Software, Jun 14 '19 at 09:41
Hi Kiko! Unfortunately the link above is the only source I have, no FTP access — Abr_7th, Jun 14 '19 at 09:44
Then the only thing left to do is to parse the HTML for the file names, and download those. — KIKO Software, Jun 14 '19 at 09:51
This *might* be relevant, if you're not strictly required to use curl: https://superuser.com/questions/200426/curl-to-download-a-directory — Piskvor left the building, Jun 14 '19 at 10:44
You just need a web scraper for this. Since there is no JS, I would suggest Goutte - you can use a CSS expression to list the elements, and then follow each one in turn. — halfer, Jun 14 '19 at 15:06

score 0 · Answer 1 · answered Jun 14 '19 at 16:22

i'm voting to close this issue as too broad, there's 2 separate things you need to figure out here,

1: how to download stuff with curl (of which you can find countless examples of on stackoverflow, and the curl_exec manual has a decent example too)

2: how to parse out links from HTML (of which you can find countless examples of on SO, for example this post has a good list)

try to figure out how to do these 2 things in order, and if you have a specific issue with 1 of these steps that you need help with, try making a new post about that.

once you know how to fetch a page with curl, you can fetch the page, and once you know how to parse HTML with PHP, you can parse out the links, then you can simply fetch all the links you found in the HTML. i suggest you start with the curl_exec() manual, and as for parsing out the links, try using DOMDocument, for example

$domd=@DOMDocument::loadHTML($html); 
foreach($html->getElementsByTagName("a") as $link){
    echo "found a link: ".$link->getAttribute("href").PHP_EOL;
}

In PHP, how to download multiple files of same type with cURL from web directory?

1 Answers1