Recursing remote directory tree via ftp in Perl (or other lang) to populate file hierarchy info only into MySQL - looking for existing libraries

Question

I want to populate a MySQL database with the information (i.e. not the files/folders themselves but their names+ other info) from a remote directory via FTP, retaining the hierarchical positions of the files.

I'm looking for existing libraries that do some or all of this, particularly the parsing of the text/strings returned from ftp output.

I've done some research here already but not found anything quite close enough:

How can I read a file's contents directly with Perl's Net::FTP?
FTP Recursive…how to skip it if I already have the file? (might be useful)
How do I traverse a remote (ftp) directory tree with Perl? (very close but there are bad reviews of this module, e.g. apparently, regex is broken and it doesn't recurse into directories, is this true? are there examples that work?
Download all images from a single directory of a website (this uses HTTP not FTP, but the concept is similar but not directly usable for my situation)

I would use the following approach to store the file hierachy in MySQL:

Storing folder hierarchy in relational database

Just thought I would ask before I potentially re-invent the wheel. If I don't get answers I'll assume what I'm asking for doesn't exist in entirety yet so I would post what my approach would be (as I have done with other, different questions I have posted here). Thanks in advance if you can offer advice though.

Using Perl to do this is not mandatory, any other cross platform language would be OK, e.g. C, C++, Java, PHP, Python. It's just that I'm "in-the-zone" with Perl doing a lot of work using it at the moment but am open to other languages having programmed extensively with most of those listed.

score 1 · Accepted Answer · answered Jun 28 '11 at 09:58

1

With ncftpls you can do a remote directory listing, and then you can parse it with Perl or any other language. You will need the -R option to recurse through subdirectories.

ncftpls -R ftp://ftp.example.com/

answered Jun 28 '11 at 09:58

Francisco R

4,032
1
22
37

+1 looks good. Thank you very much. I'll read on up on it and feedback. I'll leave the question open for a little longer to give others the change to contribute. But, yes, this looks good. Thanks again. – therobyouknow Jun 28 '11 at 10:18
Accepted answer. I think I have everything I need, including daxim's comment. Thankyou both. – therobyouknow Jun 29 '11 at 08:37

Recursing remote directory tree via ftp in Perl (or other lang) to populate file hierarchy info only into MySQL - looking for existing libraries

1 Answers1