0

We are hosting S3 public assets (images) under a local path using a reverse proxy from NGINX to S3.

We have noticed periodic errors in our logs (400 errors) which are very infrequent, but are causing issues for visitors. We can tell these are AWS errors since the content type returned is application/xml. Loading these same assets right after the logged error returns the correct response.

I've enabled logging on my relevant S3 buckets, but upon inspecting the logs I do not see any 400 errors listed during the timeframes the errors occured.

  • Would AWS throttle our requests since they are coming from one IP (through the NGINX reverse proxy)?
  • What types of 400 statuses would S3 return for public objects that are valid?
  • Is there another place in the AWS console that would display these 400 errors so we could investigate?

Updated specific example case:

Example of our asset local path: https://www.example.com/assets/images/Oasis_PalmImage_20210809_Web_v01.png

Public S3 URL: https://sb-oasis.s3.amazonaws.com/images/Oasis_PalmImage_20210809_Web_v01.png

Example of the NGINX log during logged error:

response_content_type: application/xml

status: 400

content_length: 355 bytes
dlrust
  • 715
  • 2
  • 8
  • 12
  • I'm curious how application/xml tell you it's an AWS error, can you explain that a bit more? Can you correlate the request in your Nginx logs with an AWS request, regardless of status code? I wonder if the error is between Nginx and S3, with the requests not actually arriving at S3. Suggest you edit your post to give more detail - where is your server, maybe a picture, etc. Also check your Nginx error logs. – Tim Aug 30 '21 at 19:30
  • @Tim We have not been able to visually see these error responses other than the logs. We are pretty confident that the application/xml means an S3 error due to 1) the assets being served are Image files 2) other S3 errors (invalid bucket, etc) are returned as XML 3) Dont have anything in nginx configuration that would be of that type – dlrust Aug 30 '21 at 21:47
  • I wonder if there's a way to log the body of error responses, as that might tell you more about the problem. I also wonder if you could have the user directly access S3, but that might not work with browser security settings. https://serverfault.com/questions/361556/is-it-possible-to-log-the-response-data-in-nginx-access-log/598045 – Tim Aug 30 '21 at 22:39

0 Answers0