Minio Weird Error for Listing Next Bucket Page

Question

I'm getting a strange error when using minio for local development / testing:

com.amazonaws.services.s3.model.AmazonS3Exception: A header you provided
implies functionality that is not implemented (Service: Amazon S3; Status Code: 
501; Error Code: NotImplemented; Request ID: 3L137; S3 Extended Request ID: 3L137),
S3 Extended Request ID: 3L137

The stack trace is showing that this is flying out of

at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:840)
at com.amazonaws.services.s3.AmazonS3Client.listNextBatchOfObjects(AmazonS3Client.java:911)
at com.amazonaws.services.s3.AmazonS3Client.listNextBatchOfObjects(AmazonS3Client.java:888)

On Minio itself I see this log in relation to the error:

minio_1  | Error: Invalid combination of marker 'd18d5d46-296d-4327-9304-fb72401fc5b6/some-data/ea4026b6-86a6-4950-b8e4-be0e1da80dfd.txt' and prefix 'd18d5d46-296d-4327-9304-fb72401fc5b6//some-data/'
minio_1  |        1: cmd/object-api-input-checks.go:87:cmd.checkListObjsArgs()
minio_1  |        2: cmd/fs-v1.go:1047:cmd.(*FSObjects).ListObjects()
minio_1  |        3: cmd/bucket-handlers-listobjects.go:156:cmd.(ObjectLayer).ListObjects-fm()
minio_1  |        4: cmd/bucket-handlers-listobjects.go:163:cmd.objectAPIHandlers.ListObjectsV1Handler()
minio_1  |        5: cmd/api-router.go:101:cmd.(objectAPIHandlers).ListObjectsV1Handler-fm()
minio_1  |        6: net/http/server.go:1947:http.HandlerFunc.ServeHTTP()

The important thing here is that the initial request to list objects works fine. Its when I go to call listNextBatchOfObjects that there is a problem, relevant code:

List<String> nextPage() {

    List<String> blobNames = objectListing.objectSummaries.collect { S3ObjectSummary objectSummary ->
        String blobName = objectSummary.key
        if (removePrefix) {
            blobName = blobName.replaceAll("^${removePrefix}/", '')
        }
        return blobName
    }
    objectListing = s3Client.listNextBatchOfObjects(objectListing)
    return blobNames
}

Relevant versions:

Minio on Docker minio/minio:RELEASE.2018-06-29T02-11-29Z

Amazon AWS S3 SDK com.amazonaws:aws-java-sdk-s3:1.11.356

score 0 · Answer 1 · answered Jul 01 '18 at 13:00

So it had to do with how I built the keyName. Weirdly the initial objectListing request will work with a prefix like "thing//stuff/" however that prefix is invalid to minio on subsequent requests. Making sure my keys do not contain the duplicate slashes fixed the issue.

Minio Weird Error for Listing Next Bucket Page

1 Answers1