12

Pardon the whacky question, but is there anyway to have my Rails app receive a FTP transmission?

I have daily FTP upload that I have no control over, which uploads several hundred HTML pages to our existing server each day. I want to move this site to a Rails-only deployment on Heroku, but I can't touch this FTP upload (which still needs to happen).

Since Heroku doesn't offer public storage space or FTP, I can't upload the files directly to Heroku (and I don't really want to). What I would love is to point the FTP upload to my Rails app, and have my Rails app receive and parse the HTML files to pull out the information I need, store it in the database, and do whatever else I need to do with it. (Kinda like a RESTful action, but via FTP instead of any of the standard REST verbs).

Is this at all possible or am I barking mad for thinking it? If it is possible, how would I go about doing this?

neezer
  • 19,720
  • 33
  • 121
  • 220

5 Answers5

17

You can schedule a rake task with cron which will retrieve files on the ftp server and upload them in your db.

Here is how your cron.rake file should look like:

    require 'net/ftp'

    task :cron => :environment do
      Dir.chdir("tmp") do
        Net::FTP.open("ftp.example.com") do |ftp|
          ftp.passive = true
          ftp.login('login', 'password')
          ftp.chdir("your_dir")
          ftp.get("your_file")
          # ... Load the files in the database here
        end
      end
    end

Two things to keep in mind:

  • Do not forget ftp.passive = true since Heroku does not support active mode.
  • Everything you want to do should be contained in the rake task since Heroku will erase the tmp directory once the task is over.
jbescoyez
  • 1,393
  • 9
  • 17
  • You could have two different folders on the FTP, pending and processed. The rails app could grab all files from pending and move them to processed once it's done. – baash05 Jul 10 '12 at 03:01
  • As a side note I think this is needed in Ubuntu in general – Nick Nov 09 '14 at 22:53
4

Heroku does not support your application receiving FTP.

You could possibly write a Heroku addon around receiving FTP, if you really want it.

yfeldblum
  • 65,165
  • 12
  • 129
  • 169
  • This is what I figured. Ahh, well, I'll give it a shot. – neezer May 07 '11 at 14:27
  • This is not 100% accurate. I have confirmed that, on the celadon cedar stack, you can store files in #{Rails.root}/tmp/. However, these files are only available to the process that created them. When the process completes, the files are purged. The answer from jbescoyez is the best answer. – spyderman4g63 Sep 12 '12 at 19:19
  • @spyderman4g63 My answer is exactly to the question. The question was: "Is there anyway to have my Rails app [on Heroku] receive a FTP transmission?" This is the question I answered, and the answer is: no. jbescoyez's answer does not address the original question as asked; rather, it offers a workaround that works in this particular case, but does not work in general. – yfeldblum Sep 13 '12 at 02:43
  • Yeah, you are right. This is the correct answer. A possible work around might be to try the suggestion by jbescoyez. – spyderman4g63 Sep 14 '12 at 00:25
2

I also need a Heroku instance to receive files via FTP.

I am considering using Brick FTP.

When a webhook signals that a file has been uploaded I plan to download and import the file contents into the database.

masonforest
  • 91
  • 1
  • 2
1

FTP does not support host headers so you'll struggle getting your request into the Heroku grid and to your application I would imagine.

I would be more inclined to get your files to an Amazon S3 bucket and have your application get the files from there and process them that way, or similarly have your application reach out to the FTP server and retrieve the files for processing that way.

John Beynon
  • 37,398
  • 8
  • 88
  • 97
  • Host headers isn't the issue. Indeed, Heroku applications work fine over SSL, where routing needs to be done by IP address in layer 4 rather than by the host header in layer 7. – yfeldblum Apr 05 '11 at 12:03
0

Yep this should be possible. You'll need to look into the net/ftp module and check out how that works but it should be possible.

I've found this project that could probably serve as an example.

Jakub Hampl
  • 39,863
  • 10
  • 77
  • 106
  • Thanks for the link. Are there any complications with Heroku and port 21, or the FTP protocol in general? Don't know if Heroku blocks those or not... thoughts? – neezer Apr 04 '11 at 19:03
  • I guess the easiest would be to try - I don't think they have any documentation on that. – Jakub Hampl Apr 04 '11 at 19:05