14

I would like to retrieve some meta data I added (using the console x-amz-meta-my_variable) every time I upload an object to S3.

I have set up lambda through the console to trigger every time an object is uploaded to my bucket

I am wondering if I can use something like variable = event['Records'][0]['s3']['object']['my_variable'] to retrieve this data or if I have to connect back to S3 with the bucket and key and then call some function to retrieve it?

Below is the code: 

from __future__ import print_function

import json
import urllib
import boto3

print('Loading function')

s3 = boto3.client('s3')


def lambda_handler(event, context):

    # Get the object from the event and show its content type
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = urllib.unquote_plus(event['Records'][0]['s3']['object']['key']).decode('utf8')

    # variable = event['Records'][0]['s3']['object']['my_variable']

    try:
        response = s3.get_object(Bucket=bucket, Key=key)

        # Call some function here?

        print("CONTENT TYPE: " + response['ContentType'])
        return response['ContentType']

    except Exception as e:
        print(e)
        print('Error getting object {} from bucket {}. Make sure they exist and your bucket is in the same region as this function.'.format(key, bucket))
        raise e
Y Anderson
  • 427
  • 3
  • 5
  • 15
  • 2
    I have not found any authoritative documentation on exactly what is included in S3 events sent to a Lambda function. My suggestion would be to log the event and then inspect the log to see if the information you want is included in the event. – garnaat Mar 02 '16 at 13:13
  • 1
    Documentation on what is included in S3 events can be found here: https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html . It's a shame that the metadata is not included :( – jjanczyszyn Feb 05 '18 at 03:36

2 Answers2

13

The metadata is not in the event but in the head object.

The HEAD operation retrieves metadata from an object without returning the object itself. This operation is useful if you are interested only in an object's metadata. To use HEAD, you must have READ access to the object.

A HEAD request has the same options as a GET operation on an object. The response is identical to the GET response except that there is no response body.

s3.head_object(Bucket=bucket, Key=key)

Below code is a snippet to get the metadata.

from __future__ import print_function
import boto3, logging

s3 = boto3.client('s3')
logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
  for record in event['Records']
    bucket = record['s3']['bucket']['name']
    key = record['s3']['object']['key']
    response = s3.head_object(Bucket=bucket, Key=key)
    
    logger.info('Response: {}'.format(response))

    print("Author : " + response['Metadata']['author'])
    print("Description : " + response['Metadata']['description'])

Output:

[INFO]  2016-05-18T01:30:47.900Z    241f0cfc-1c98-12e6-b9a7-cf406f32a0dc    Response: {u'AcceptRanges': 'bytes', u'ContentType': 'binary/octet-stream', 'ResponseMetadata': {'HTTPStatusCode': 200, 'HostId': 'K8JMVbEt5xA+qXuXOedb1y5nxuv6scMXnNH/rHVtxcg=', 'RequestId': 'D05BE92E55E0'}, u'LastModified': datetime.datetime(2016, 5, 17, 22, 54, 37, tzinfo=tzutc()), u'ContentLength': 94320, u'ETag': '"0e4d457d912bce9ff81952"', u'Metadata': {'author': 'Satyajit Ray', 'description':'He was an Indian filmmaker, widely regarded as one of the greatest filmmakers of the 20th century.'}}
Author : Satyajit Ray
Description : He was an Indian filmmaker, widely regarded as one of the greatest filmmakers of the 20th century.
Community
  • 1
  • 1
Anna23
  • 680
  • 2
  • 8
  • 16
  • Hmm, I seem to get empty metadata. I've set the ```Cache-Control``` metadata on my object, yet ```response['Metadata']``` is empty. Is there a link to those docs you reference? – Adam Parkin Apr 15 '18 at 18:49
3

You can get the meta-data from the head object where you have to pass an object which contains bucket and key:- Eg : Below is a code(in NodeJs) that you have to use in order to get the meta-data which was attached with the pre-signedUrl while generating it from the aws-sdk.

//for generating pre-signed url with meta data
exports.getSignedUrl = async (myKey, metadata) => {
  const signedUrlExpireSeconds = 20000;
  const params = {
    Bucket: BUCKET,
    Key: myKey,
    Expires: signedUrlExpireSeconds,
    /* ACL: 'bucket-owner-full-control', ContentType:'image/jpeg', */
    ContentType: 'image/jpeg',
    ACL: 'public-read',
    Metadata: metadata,
  };
  const url = await s3.getSignedUrl('putObject', params);
  return url;
};
//for obtainig the meta data for the bucket and key
    const s3Object = reqBody.Records[0].s3;
    const bucketName = s3Object.bucket.name;
    const objectKey = s3Object.object.key;

    const params = {
      Bucket: bucketName,
      Key: objectKey,
    };
    const data = await s3.headObject(params).promise();
    const metadata = (!data) ? null : data.Metadata;```
Nishant Dwivedi
  • 380
  • 3
  • 8