4

In my current project I need to check my S3 bucket contents every 4 seconds for new files.

This script will run for around 3 hours every time that the service is used, and will have something around 2700 files by the end at a single prefix.

This is my function to list those files:

public function listFiles($s3Prefix, $limit, $get_after = ''){
    $command = $this->s3Client->getCommand('ListObjects');
    $command['Bucket']  = $this->s3_bucket;
    $command['Prefix']  = $s3Prefix;
    $command['MaxKeys'] = $limit;
    $command['Marker']  = $s3Prefix.'/'.$get_after;

    //command['Query'] = 'sort_by(Contents,&LastModified)';

    $ret_s3 = $this->s3Client->execute($command);

    $ret['truncated'] = $ret_s3['IsTruncated'];
    $ret['files'] = $ret_s3['Contents'];

    return $ret;
}// listFiles

What I do need is get the files, order by the LastModified field, so I do not need to fetch over 2k files. Is there an extra parameter like

command['Query'] = 'sort_by(Contents,&LastModified)';

to add in the php API?

---------- EDITED ------------

As pointed for Abhishek Meena answer, in the shell it is possible to use 

aws s3api list-objects --bucket "bucket-name" --prefix "some-prefix" --query "Contents[?LastModified>=\`2017-03-08\`]"

What I'm looking is how to implement this in PHP.

PHP API: https://github.com/aws/aws-sdk-php

Moisés
  • 1,324
  • 15
  • 43

1 Answers1

1

I don't know if they have some thing to sort the objects on the bases of LastModified but you can query and filter objects on the LastModified column. This is what you can use to filter all the files modified after certain time aws s3api list-objects --bucket "bucket-name" --prefix "some-prefix" --query "Contents[?LastModified>=\`2017-03-08\`]"

This is for the shell they might have something similar for the php.

Abhishek Meena
  • 19
  • 2
  • Yeah, from the shell documentation that I tried something like command['Query'] = 'sort_by(Contents,&LastModified)'. The something similar is what I'm looking for. Thanks though. – Moisés Mar 08 '17 at 17:09
  • Since all this is getting converted to the same API definition. You can try this and check `command['Query'] = "Contents[?LastModified>=$date";` where $date will contain your datetime as mentioned in the [aws-sdk-php](http://docs.aws.amazon.com/aws-sdk-php/v3/api/api-s3-2006-03-01.html#listobjects) result section. **'LastModified' => ** – Abhishek Meena Mar 08 '17 at 17:39