Download a percentage of a video with curl

Question

I have some videos that are hosted on S3 (.mp4 and .mov) some of which are rather large (1.2GB+).
I want to get the first frame from each video using the PHP wrapper for FFmpeg but I don't want to have to download the full file first.

What I really want to do is download a certain percentage of the file, something like 2%, so that I can guarantee that I will get the first frame.

I found a way to download 1mb of the file here: https://code.i-harness.com/en/q/c09357

However, it is the following chunk of this code that I don't really understand how it is only downloading 1mb.

function myfunction($ch, $data)     {
    $length = fwrite($this->fh, $data);
    $size=&$this->size;

    if($length === FALSE) {
        return 0;
    } else {
        $size += $length;
    }

    // Downloads 1MB.

    return $size < 1024 * 1024 * 1 ? $length : 0;
}

To me that says set the size to be the size of the file and then if the size is less than 1mb return the length, else return 0.
Now, I know it does work because I have run it, but I don't know how it works so that I can convert this into getting the percentage of the file.

Downloading 1 or 2 MB of the file is fine for the smaller files and the mp4 files, however the .mov files fail to get the first frame if it is less than about 20mb and some frames throw a division by zero error when getting the frame, I guess from the above function returning 0.

Could anyone shed some light on how all of this is working please, or even better if you could suggest an improvement?

Are you sure that this is the correct approach? To me generating thumbnails on S3 itself seems a better option, with Lambda it should do the trick. You would save a lot of bandwidth and gain a lot of speed in return. https://concrete5.co.jp/blog/creating-video-thumbnails-aws-lambda-your-s3-bucket — Razor_alpha, Sep 03 '18 at 19:36
Well, that was an extremely interesting read and I think this is exactly what I need for a larger scale production environment, however all of that is WAAAAYYY beyond me and my skill level. I got as far as setting up EC2 before I got completely lost. Thank you very much for the information, though I think I will stick to the php method for now. — Jimmyb_1991, Sep 03 '18 at 21:55
Still, maybe generating the thumbnails on upload would help. — Razor_alpha, Sep 04 '18 at 01:15

hanshenrik · Answer 1 · 2018-09-11T05:24:40.513

myfunction is almost certainly set as the CURLOPT_WRITEFUNCTION callback function for curl_exec, and if that function returns 0 (or any number other than the size of $data), then curl will terminate the transfer, and curl_exec will return the CURLE_ABORTED_BY_CALLBACK error code. thus after you've downloaded >=1 mebibyte, curl_exec will stop with the CURLE_ABORTED_BY_CALLBACK error.

What I really want to do is download a certain percentage of the file, something like 2%, so that I can guarantee that I will get the first frame. - depending on the movie encoding, the first mebibyte may not be enough. there are some encoding schemes (as a specific example, .mpeg movies can be encoded this way) where you need a few bytes from the end of the file to render the first frame (iirc for .mpeg it's called the MOOV Atom - on mpeg movies where the MOOV atom is at the end of the file, you need a few bytes from the end of the file to render the first frame. for all streaming-optimized .mpeg movies, the MOOV atom is at the beginning of the file, not the end, and your 1st mebibyte scheme would work, but if it's at the end your scheme won't work unless the whole movie is <1 mebibyte)

a much better approach is to just let ffmpeg deal with it. ffmpeg will know how much data to download, and will attempt to only download the required parts, instead of the whole movie, and you'd need a program like ffmpeg to extract the first frame later anyway.

try

function getFirstFrameAsJpg(string $url):string{
        if(file_exists("/dev/null")){
            $ret=shell_exec("ffmpeg -i ".escapeshellarg($url)." -f image2pipe -frames 1 -r 1 -c:v:1 jpeg - 2>/dev/null");
        }else{
            // windows, probably, where /dev/null isn't supported but NUL works the same way.
            $ret=shell_exec("ffmpeg -i ".escapeshellarg($url)." -f image2pipe -frames 1 -r 1 -c:v:1 jpeg - 2>NUL");
        }
        return $ret;
}

it will return the first frame of the video in the url as the binary for a .jpg image. (meaning you can do file_put_contents('image.jpg',getFirstFrameAsJpg($url)); - interestingly, if ffmpeg is not installed, $ret will be NULL, which means if you use strict_types=1, you will get an exception, otherwise you will get an empty string. )

ps before you allow potential hackers to specify the url for this function, make sure to validate that it is indeed a http url, as i did not consider the security implications of letting hackers run getFirstFrameAsJpg("/etc/passwd") or similar.

if you need to download with a bunch of headers, consider setting up a proxy scheme for ffmpeg where ffmpeg is told to download from a unique proxy-url instead, and still let ffmpeg deal with which parts of the movie to download, and make sure to implement the http range header for such a proxy, as ffmpeg will need it if extracting the 1st frame from a movie where the last part of the movie is required to extract the first frame.

(thanks to c_14 @ freenode #ffmpeg for the image2pipe command)

Download a percentage of a video with curl

1 Answers1