Does Fog AWS S3 take time to refresh its index of files?

Question

I have this code in an rspec test:

specify 'Saves files to S3' do
  subject.upload
  expect(connection.directories.get(credentials[:bucket]).files.map(&:key))
    .to include("#{directory}/groucho.jpg")
end

The subject.upload adds the file to S3 and the expect pulls down the list of files in the bucket, looking for the name of the file just uploaded. The expect used to pass (like two days ago), but now it does not.

Even if I sleep for a while, the expect never passes. I have no idea what changed, but am looking for some guidance.

Answer 1

The answer is that the index is updated right away. However, the contents of the index are paginated. In my case, a lot more files were added to the bucket, so my file no longer appeared within the first page of results. You can paginate the index from S3. See the docs. You can also do a more expensive...

file = connection.directories.get(credentials[:bucket]).files.detect { |f| f.key == "#{directory}/sample.txt" }

... which iterates one at a time.