0

According to the docs, it is possible to create a CodePipeline that uses AmazonS3 as a deployment provider

On the other hand S3 only provides eventual consistency after put.

Can it happen that a code pipeline triggers with old code because it fetches an old version from S3?

Gabriel Furstenheim
  • 2,969
  • 30
  • 27

2 Answers2

1

When the CodePipeline triggers it is done via a CloudWatch event.

To avoid the scenario you're also talking about (with consistency) it uses the eTag and the Version Id (versioning is required for an S3 source bucket) to ensure it gets the correct version of the object.

So to answer your question you do not need to care for consistency when it comes to CodePipeline, it will always return the correct code when the event triggers.

Chris Williams
  • 32,215
  • 4
  • 30
  • 68
1

The eventual consistency of S3 objects is only for overwrite PUTS and DELETES:

Amazon S3 offers eventual consistency for overwrite PUTS and DELETES in all Regions.

For new objects, there is read-after-write consistency :

Amazon S3 provides read-after-write consistency for PUTS of new objects in your S3 bucket in all Regions with one caveat. The caveat is that if you make a HEAD or GET request to a key name before the object is created, then create the object shortly after that, a subsequent GET might not return the object due to eventual consistency.

Therefore when overwriting the same object that CP is using, eventual consistency applies.

To supplement @ChrisWilliams answer, as he correctly explains versioning, you can also observe and verify that version numbers are used in the CodePipeline console:

enter image description here

Marcin
  • 215,873
  • 14
  • 235
  • 294
  • If the key and buckets are the same, then the object is not considered new, right? Because in order to trigger the pipeline you need to configure those, otherwise you would need to configure the pipeline on each deploy – Gabriel Furstenheim Jul 24 '20 at 08:31
  • @GabrielFurstenheim That's correct. When you overwrite the same key, eventual consistency applies. – Marcin Jul 24 '20 at 08:32
  • But then, my question remains. I'm not asking if the first execution will pick the correct code, but if later executions will – Gabriel Furstenheim Jul 24 '20 at 08:57
  • @GabrielFurstenheim Check ChrisWilliams answer, he explains this better ;-) – Marcin Jul 24 '20 at 08:59
  • Hey @marcin thanks for the comment on the version id. How can I match the version id and the s3 bucket? In the s3 bucket I can definitely see the etag, but I don't know how to match one and the other to be sure – Gabriel Furstenheim Jul 24 '20 at 10:32
  • @GabrielFurstenheim No problem. When you setup your CP you only specify object key, not its specific version. But you can go to S3 console where your object is, and toggle Hide/Show versions in the bucket. CP will automatically fetch the latest version when triggered, and it will display it as shown in the screenshot. – Marcin Jul 24 '20 at 10:37