Generating number of thumbnails depending on video size using AWS MediaConvert

Question

After reading this article I get the sense that AWS media convert job template cannot be re-used to generate thumbnails of arbitrary video size. The article assumes that we know the size/duration of video uploaded upfront hence the number of thumbnails we desire.

What I am looking for is to generate a random number of thumbnails based of video size (e.g large number of thumbnails for large video and small number of thumbnails for small video). I approached this solution using lambda trigger and ffmpeg lambda layer but lambda function timeouts (15 minutes max) for videos larger than 150MB (since it takes time to read the video from the s3 bucket).

What are my options to process large number of video , generate a variable number of thumbnails, merge those thumbs to generate a sprite?

I tried lambda trigger with ffmpeg/ffprob to generate sprite, but that has timeout issue. Now I have setup a cloud watch event rule to trigger lambda function on mediaconvert job status change( completed) and merge the thumbs to generate sprite, which seems much lighter but I need an arbitrary number of thumbs.

score 3 · Answer 1 · answered Dec 01 '20 at 01:11

You could switch from using ffprobe to MediaInfo [1]. This can get you the duration and file size of an input file. From there you can use the calculations from the article and use MediaConvert to create your sprite images. After the job completes you can use the COMPLETE CloudWatch Event to trigger a postprocessing workflow that can create the sprite manifest. By default the COMPLETE details section will include the output path to the last JPEG frame capture from your MediaConvert output [2].

As a pre-processing step you can gather the duration of the file as well as the size. If you only require the size of the file, then you can use the S3 SDK to get the content-length of the file (calls HEAD to the object) and calculate from there.

s3 = boto3.client('s3')
response = s3.head_object(Bucket='bucket', Key='keyname')
size = response['ContentLength']
print(size)

You can also use parameter overrides in job templates. For example you can create a job that reference a template but you can specify settings that override those settings with you call the CreateJob API.

The following is for presets however the same concept applies to Job Templates.

Using the following HLS System Presets

System-Avc_16x9_720p_29_97fps_3500kbps    
System-Avc_16x9_360p_29_97fps_1200kbps
System-Avc_16x9_270p_14_99fps_400kbps

JSON Payload changes the Audio Selector Source Name for Output 2 (System-Avc_16x9_360p_29_97fps_1200kbps) and Output 3 (System-Avc_16x9_270p_14_99fps_400kbps)

{
  "Queue": "arn:aws:mediaconvert:us-west-2:111122223333:queues/Default",
  "UserMetadata": {},
  "Role": "arn:aws:iam::111122223333:role/EMFRoleSPNames",
  "Settings": {
    "OutputGroups": [
      {
        "Name": "Apple HLS",
        "Outputs": [
          {
            "NameModifier": "_1",
            "Preset": "System-Avc_16x9_720p_29_97fps_3500kbps"    
          },
          {
            "NameModifier": "_2",
            "Preset": "System-Avc_16x9_360p_29_97fps_1200kbps",
             "AudioDescriptions": [
              {
                "AudioSourceName": "Audio Selector 2"
              }
            ]
          },
          {
            "NameModifier": "_3",
            "Preset": "System-Avc_16x9_270p_14_99fps_400kbps",
            "AudioDescriptions": [
              {
                "AudioSourceName": "Audio Selector 2"
              }
            ]
          }
        ],
        "OutputGroupSettings": {
          "Type": "HLS_GROUP_SETTINGS",
          "HlsGroupSettings": {
            "ManifestDurationFormat": "INTEGER",
            "SegmentLength": 10,
            "TimedMetadataId3Period": 10,
            "CaptionLanguageSetting": "OMIT",
            "Destination": "s3://myawsbucket/out/master",
            "TimedMetadataId3Frame": "PRIV",
            "CodecSpecification": "RFC_4281",
            "OutputSelection": "MANIFESTS_AND_SEGMENTS",
            "ProgramDateTimePeriod": 600,
            "MinSegmentLength": 0,
            "DirectoryStructure": "SINGLE_DIRECTORY",
            "ProgramDateTime": "EXCLUDE",
            "SegmentControl": "SEGMENTED_FILES",
            "ManifestCompression": "NONE",
            "ClientCache": "ENABLED",
            "StreamInfResolution": "INCLUDE"
          }
        }
      }
    ],
    "Inputs": [
      {
        "AudioSelectors": {
          "Audio Selector 1": {
            "Offset": 0,
            "DefaultSelection": "DEFAULT",
            "ProgramSelection": 1,
            "SelectorType": "TRACK",
            "Tracks": [
              1
            ]
          },
          "Audio Selector 2": {
            "Offset": 0,
            "DefaultSelection": "NOT_DEFAULT",
            "ProgramSelection": 1,
            "SelectorType": "TRACK",
            "Tracks": [
              2
            ]
          }
        },
        "VideoSelector": {
          "ColorSpace": "FOLLOW"
        },
        "FilterEnable": "AUTO",
        "PsiControl": "USE_PSI",
        "FilterStrength": 0,
        "DeblockFilter": "DISABLED",
        "DenoiseFilter": "DISABLED",
        "TimecodeSource": "EMBEDDED",
        "FileInput": "s3://myawsbucket/input/test.mp4"
      }
    ],
    "TimecodeConfig": {
      "Source": "EMBEDDED"
    }
  }
}

== Resources ==

[1] https://aws.amazon.com/blogs/media/running-mediainfo-as-an-aws-lambda-function/
[2] https://docs.aws.amazon.com/mediaconvert/latest/ug/file-group-with-frame-capture-output.html

thanks, I am looking at this in more detail, ffprobe does not take that much time, however , it ffmpeg. It timeouts 15 sec for ~ 150 MB mp4 video. I assumed it would seek to specific frame and extract that but that is not the case , it just reads whole video (downloads it, if its remote resource like s3 bucket). — sakhunzai, Dec 01 '20 at 04:32
`By default the COMPLETE details section will include the output path to the last JPEG frame capture`, thanks for the tip. This will help me avoid listing s3 bucket — sakhunzai, Dec 01 '20 at 08:14
With regards to ffmpeg, I would expect this. Because the binary doesn't support file streaming, you are correct the file has to be downloaded then processed. This limits you to 2 factors in lambda, time and local space in lambda. If you go the ffmpeg process route I would suggest building a container workflow with ECS and use that instead. That way you are not limited to size or processing time. It's adds complexity to the solution as you have to build a queuing system but you won't run into the limitations of lambda. — jjohn, Dec 01 '20 at 16:25

Generating number of thumbnails depending on video size using AWS MediaConvert

1 Answers1