0

I have multiple folders inside a bucket each folder is named as a unique guid and it is always going to contain a single file.

I need to fetch only those files which have never been read before. If I'll fetch all the objects at once and then do client side filtering it might introduce latency in the near future as every day the number of new folders getting added could be hundreds.

Initially I tried to list object by specifying StartAfter, but soon I realized it only works with alphabetically sorted list. https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html

I am using AWS C# SDK. Can someone please give me some idea about the best approach.

Thanks

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Ajendra Prasad
  • 299
  • 1
  • 8
  • 22

1 Answers1

0

Amazon S3 does not maintain a concept of "objects that have not been accessed".

However, there is a different approach to process each object only once:

  • Create an Amazon S3 Event that will trigger when an object is created
  • The Event can then trigger:
    • An AWS Lambda function, or
    • Send a message to an Amazon SQS queue, or
    • Send a message to an Amazon SNS topic

You could therefore trigger your custom code via one of these methods, and you will never actually need to "search" for new objects.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470