0

I need to be able to retrieve an S3 object residing in Australia S3 Storage from London EC2 and minimise latency.

On the London EC2 instance, I set the RegionEndpoint on the S3Client to ApSouthEast2 and acquire it.

The issue I am facing is that the it takes far too long to retrieve even small files (10kb file - 1.2s+). I have Peering setup on the server also so I can ping EC2 instances in ApSouthEast2 from London and my round trips are all 0ms.

However, when retrieving data from S3 it is a different story. How can I allow the s3 bucket to be available "internally" (vpc endpoints and via existing peering??) as I am assuming traversal over WAN is creating this huge lag.

It must be stressed that any changes I make will still require the S3 bucket to be available in its current state as this is a production system.

The_Chud
  • 991
  • 1
  • 11
  • 24

1 Answers1

2

First, it is impossible to have 0ms latency between Europe and Australia. Something to do with the speed of light. See: Ping time between London and Sydney - WonderNetwork

I would recommend you try to connect to S3 directly (rather than via VPC Peering) and see whether the speed is any better.

Then, try connecting via Amazon S3 Transfer Acceleration and see if it is any better. Connections are routed via the AWS network.

Worst case, you could always replicate the bucket to the London region so that objects are copied soon after they are created. You can then access the objects locally.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
  • Thanks John - Yes I obviously stuffed up the pings. I have enabled Transfer Acceleration on both ends up/down. The performance increase is substantial and that is probably an understatement. It seems though, If I don't make a request within a few minutes of the first - it's like it loses it's "path" and the performance blows out (to the point where it is slower than non-accelerated requests). Any ideas on how to resolve this or what causes this? – The_Chud Oct 04 '21 at 05:07
  • Transfer acceleration significantly reduced transit time in 90%+ cases. The other 10% seem to "lose" their path, or cache, or something I'm not aware of and produce a slower retrieval than the standard method seemingly. – The_Chud Oct 27 '21 at 02:50