0

Whenever I run my code, I upload the result to my s3 bucket, which doesn't have a specific name pattern.

Suppose the following files are present in the bucket right now.

2017-12-06 11:40:47 93185 RAW_D_3600_S_1_1294573351559346926-0.metadata

2017-12-06 11:40:47 167170 RAW_D_3600_S_1_1294573351559346926-0.txt

2017-12-06 10:55:54 21 USERENROLL_1_1-0.metadata

2017-12-06 10:55:54 190 USERENROLL_1_1-0.txt

2017-12-05 17:56:36 174 USERENROLL_1_1-duke1.csv

2017-12-06 11:13:45 105 USERENROLL_1_7-0.metadata

2017-12-06 11:13:45 599 USERENROLL_1_7-0.txt

2017-12-06 11:15:51 126 USERENROLL_1_8-0.metadata

2017-12-06 11:15:51 600 USERENROLL_1_8-0.txt

2017-12-06 10:59:26 21 USERENROLL_1_9-0.metadata

2017-12-06 10:59:26 181 USERENROLL_1_9-0.txt

I want to download the file which was created on 2017-12-06 11:40:47.

I fetch the file by storing the current time and search for files in the bucket that are generated after the stored time. But this doesn't give accurate results all the time, as file might take longer to be generated.

Is there a way to tackle this problem? PS - Can't change the naming convention being used.

Vaibhav Rai
  • 183
  • 1
  • 8
  • Have a look at https://stackoverflow.com/questions/31062365/get-last-modified-object-from-s3-cli – Arafat Nalkhande Dec 06 '17 at 06:39
  • Why do you want to do this? Please elaborate your use case. Storing files with similar prefix in S3 is a bad idea and affects S3 performance. And what are you actually doing with these files and what is your criteria to get such files? – Aniket Thakur Dec 06 '17 at 14:49

0 Answers0