S3AbortableInputStream : Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. Warning when reading only ObjectMetadata

Question

I am using <aws.java.sdk>1.11.637</aws.java.sdk> with Spring boot 2.1.4.RELEASE.

Code : which causes S3 Warning (Here i have to access Only getUserMetadata from S3Object and not whole Object Content)

private Map<String, String> getUserHeaders(String key) throws IOException {
    Map<String, String> userMetadata = new HashMap<>();
    S3Object s3Object = null;
    try {
      s3Object = s3Client.getObject(new GetObjectRequest(bucketName, key));
      userMetadata.putAll(s3Object.getObjectMetadata().getUserMetadata());
    } finally {
      if (s3Object != null) {
        s3Object.close();
      }
    }
    return userMetadata;
  }

Output : Whenever s3Object.close(); is invoked , then i see warning on console with below message

{"msg":"Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use.","logger":"com.amazonaws.services.s3.internal.S3AbortableInputStream","level":"WARN","component":"demo-app"}

My Investigation for error cause:

I further checked https://github.com/aws/aws-sdk-java/blob/c788ca832287484c327c8b32c0e2b0090a74c23d/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/internal/S3AbortableInputStream.java#L173-L187 and it says if _readAllBytes() is not true (in my case where i am using S3Object just to getUserMetadata and not whole stream content) then there would be warning always.

Questions:

a) How S3Object.close is leading to invoke S3AbortableInputStream.close as I assume code inside S3Object.close https://github.com/aws/aws-sdk-java/blob/c788ca832287484c327c8b32c0e2b0090a74c23d/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/model/S3Object.java#L222 only invoke SdkFilterInputStream.close via is.close();

b) How should I get rid of these warnings when i want use S3Object only to read meta data and not whole object content.

score 2 · Accepted Answer · answered Oct 10 '19 at 12:45

2

Maybe try using this API function designed to retrieve only the metadata for an S3 object:

ObjectMetadata getObjectMetadata(GetObjectMetadataRequest getObjectMetadataRequest)
                          throws SdkClientException,
                                 AmazonServiceException

So change your code to:

ObjectMetadata s3ObjectMeta = null;
s3ObjectMeta = s3Client.getObjectMetadata(new GetObjectMetadataRequest(bucketName, key));
userMetadata.putAll(s3ObjectMeta.getUserMetadata());

answered Oct 10 '19 at 12:45

Ashaman Kingpin

1,467
1
11
11

I think this is the exact solution I am looking. I am new to AWS but with no excuses, I feel ashamed that I missed the API documentation. – Arun Kumar Oct 10 '19 at 14:42
No problem @ArunKumar, happens to the best of us. Please don't forget to mark this as the accepted answer. – Ashaman Kingpin Oct 10 '19 at 14:46

S3AbortableInputStream : Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. Warning when reading only ObjectMetadata

1 Answers1