8

I'm working on a tool to get some data that is buried within a very large file on a remote system. It would be impractical to copy the entire file over, and all of the data I need exists within the first 1000 or so bytes of the file. I know that I can start a get and just cancel it with ^C to get a partial file, however this would be difficult (if not impossible) to automate with any consistency.

I would like to tell my ftp client to only grab x bytes of the remote file and quit as soon as it has them. I've found a few windows clients that do partial downloads, but I haven't found anything in the ftp man page and documentation online is sparse.

I found this HowTo: http://cdsarc.u-strasbg.fr/doc/ftp.htx that suggests the following syntax:

ftp> get bigfile.dat:0-5000 bigfile.nxt

It is unclear to me if this is supposed to be implemented in the client or the server, but in either case, it doesn't seem to work in my environment. (Standard linux ftp client connecting to an FTP server running on z/OS)

Even when trying between the linux standard ftp client and a filezilla server on windows, my attempts fail in the following way

ftp> get green.gif:0-10c
local: green.gif:0-10c remote: green.gif:0-10c
227 Entering Passive Mode (9,42,91,226,4,105)
550 File not found

So the :0-10c is interpreted as part of the filename it seems. Fail. Any thoughts?

Jax
  • 428
  • 3
  • 10
  • 17

3 Answers3

10

Use curl. From the man page:

   -r/--range <range>
          (HTTP/FTP/FILE)  Retrieve  a byte range (i.e a partial document)
          from a HTTP/1.1, FTP server or a local FILE. Ranges can be spec-
          ified in a number of ways.

          0-499     specifies the first 500 bytes

          500-999   specifies the second 500 bytes

          -500      specifies the last 500 bytes

          9500-     specifies the bytes from offset 9500 and forward

          0-0,-1    specifies the first and last byte only(*)(H)

          500-700,600-799
                    specifies 300 bytes from offset 500(H)

However, note that the SIZE extension must be supported by the server for this to work.

MikeyB
  • 39,291
  • 10
  • 105
  • 189
  • whoa! That is...coincidental... – Matt Simmons Jun 03 '09 at 15:30
  • Haha, wow, thanks guys, that's exactly what I needed, I wish I could give you both credit for the answer, but sadly, I only get one checkmark so I guess Supermathie gets it for having faster ^V ^C skills :) – Jax Jun 03 '09 at 15:39
7

I think you want to use curl for this

From the man page: 

   -r/--range <range>
          (HTTP/FTP/FILE)  Retrieve  a byte range (i.e a partial document)
          from a HTTP/1.1, FTP server or a local FILE. Ranges can be spec‐
          ified in a number of ways.

          0-499     specifies the first 500 bytes

          500-999   specifies the second 500 bytes

          -500      specifies the last 500 bytes

          9500-     specifies the bytes from offset 9500 and forward
Matt Simmons
  • 20,396
  • 10
  • 68
  • 116
  • I think I beat you to it by like 5 seconds or so :p – MikeyB Jun 03 '09 at 15:33
  • You missed the caveat: 
    TP range downloads only support the simple syntax ’start-stop’ (optionally with one of the numbers omitted). It depends on the non-RFC command SIZE.
     – David Pashley Jun 03 '09 at 15:35
0

Schemes like this depends heavily on the implementation of the server. Some servers support a lot more features than others.

I would investigate some way of splitting the file up or consider a custom application that will send the range you ask for.

David Pashley
  • 23,497
  • 2
  • 46
  • 73