Get list of files from FTP server

Question

I'm trying to get a list of all the files we have on a server ( specifically every pdf file we have there ). I've tried using total commander and search for the files. It worked to some extent, as in, i got a list of every pdf we had there, but no way of exporting the results ( we have 100.000+ files there )

I've tried using a bash script to get the information, but i'm not very experienced with linux, and i don't really know what i'm doing.

My script looks like this :

#!/bin/bash
hostname="<host>"
ftp -i -nv $hostname << EOF
user <username> <password>
ls -R 
EOF

Running the above script i get

?Invalid command
501 Illegal PORT command
ftp: bind: Address already in use
221 Goodbye

Any help or pointing me on what to search would be greatly appreciated.

score 48 · Answer 1 · answered Apr 06 '13 at 21:28

48

With curl, this is handy
curl ftp://yourftpserver/dir/ --user username:password

answered Apr 06 '13 at 21:28

user1587276

906
6
4

21

FYI: `curl -l` returns just the names. – Asclepius Sep 10 '14 at 22:48
4

how do you script this to recurse? – B Johnson Apr 27 '16 at 05:41
6

I do not understand why most people are using username:password in their command, that's bad practice and can be history hjacked! At least use a (protected) file and use a substitution for it, e.g.: `curl ftp://yourftpserver/dir/ --user "$(cat .userpw)"` – ioCron Mar 06 '19 at 15:41
Is there a human readable size output version? – zabop Oct 10 '22 at 09:17

Asclepius · Answer 2 · 2015-11-09T23:16:28.240

7

ncftpls ftp://yourftpserver/dir/*.pdf

Note that patterns such as *.pdf, etc. in the above command do work as expected.

For recursive, use -R. For more options, see man ncftpls.

ncftpls is provided by the ncftp package. For RHEL, this package is available in the epel repo.

edited Nov 09 '15 at 23:16

answered Sep 10 '14 at 23:48

Asclepius

57,944
17
167
143

Thank you @A-B-B, I didn't know about this utility. Now I can do this to get the name of the latest `coreutils` source code tarball: `ncftpls -x "-t" ftp://ftp.gnu.org/gnu/coreutils/coreutils*.xz|head -1` – Setaa Jun 19 '19 at 22:09
A shame this does not use .netrc – mckenzm Jun 19 '21 at 07:43

score 7 · Answer 3 · answered Feb 08 '17 at 09:54

7

curl ftp://user:password@<ip>/path/

The last / is a must if it is a directory. This worked in curl version 7.29.0

answered Feb 08 '17 at 09:54

Talespin_Kit

20,830
29
89
135

score 6 · Accepted Answer · edited May 23 '17 at 12:25

6

Try to configure ftp to use the PASV (passive) mode for data transfers. This done with the -p switch.

I'm not sure if you will be able to do a recursive file listing with this ftp-client. ls -R in my case just gave the list of files and directories in the current working dir. Maybe Recursive FTP directory listing in shell/bash with a single session (using cURL or ftp) will help you.

edited May 23 '17 at 12:25

Community

1
1

answered Mar 29 '13 at 15:30

Patrick B.

11,773
8
58
101

score 4 · Answer 5 · answered Mar 06 '19 at 15:49

As mentionend in one of my comments, do not use user:password as plain text in your commands ever, otherwise you are running at risk of beeing history hjacked! Instead use at least a protected/restricted file with the username+password and substitute it in your command, e.g.:

ftp://yourftpserver/dir/ --user "$(cat .userpw)"

Whereas .userpw is your protected/restricted file with the example content: myusername:mypassword

score 3 · Answer 6 · answered Jun 30 '20 at 13:44

I have always run this FTP script in a linux system, after login ls command is submitted to a ftp connection(passive). List of files is written to a stdout stream. Script is inline in a sh script file.

#!/bin/sh

HOST=11.22.33.44
USER=myuser
PWD="mypwd"

ftp -p -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PWD
ls
quit
END_SCRIPT
exit 0

score 0 · Answer 7 · answered Dec 07 '22 at 13:53

0

curl --list-only ftp://user:password@ftp.adress/directory/JPG/

with curl request lists files

answered Dec 07 '22 at 13:53

Alperen KILIÇ

17
3

Get list of files from FTP server

7 Answers7