0

I'm using MultipartObjectAssembler to upload data from a Database to OCI object storage. Is there a way to know the reason for failure when using multipart upload?

When I try to commit assembler I'm getting IllegalStateException with the message "One or more parts were have not completed upload successfully". I would like to know why any part got failed? I couldn't find a way to get this information from SDK.

try {
    assembler.addPart(new ByteArrayInputStream(part, 0, length),
                    length,
                    null);

    assembler.commit();
} catch (Exception e) {
    assembler.abort();
    throw new RuntimeException(e.getMessage());
} 

Edit: I need to get an Exception thrown by a failed part and propagate the error message.

akshay
  • 207
  • 1
  • 2
  • 7

1 Answers1

2

Is there a reason you are not using the UploadManager? It should do everything for you, including adding parts and committing. Here's an end-to-end example: https://github.com/oracle/oci-java-sdk/blob/master/bmc-examples/src/main/java/UploadObjectExample.java

If, for some reason, you cannot use UploadManager, please take a look at its source code nonetheless, since it demonstrates the intended usage of MultipartObjectAssembler: https://github.com/oracle/oci-java-sdk/blob/master/bmc-objectstorage/bmc-objectstorage-extensions/src/main/java/com/oracle/bmc/objectstorage/transfer/UploadManager.java#L175-L249

  1. You create the MultipartObjectAssembler:
MultipartObjectAssembler assembler =
                createAssembler(request, uploadRequest, executorServiceToUse);
  1. You create a new request. This gives you back a MultipartManifest, which will later let you check if parts failed.
manifest =
                    assembler.newRequest(
                            request.getContentType(),
                            request.getContentLanguage(),
                            request.getContentEncoding(),
                            request.getOpcMeta());
  1. Then you add all the parts:
                    assembler.addPart(
                            ProgressTrackingInputStreamFactory.create(
                                    chunk, progressTrackerFactory.getProgressTracker()),
                            chunk.length(),
                            null);
  1. Then you commit. This is where your code currently throws. I suspect not all parts have been added.
CommitMultipartUploadResponse response = assembler.commit();
  1. If something goes wrong, check MultipartManifest.listCompletedParts(), MultipartManifest.listFailedParts(), and MultipartManifest.listInProgressParts(). The manifest should tell you what parts failed. Unfortunately not why; for that, you can enable ERROR level logging for com.oracle.bmc.objectstorage.transfer (for the class com.oracle.bmc.objectstorage.transfer.internal.MultipartTransferManager in particular).

If I have misunderstood something, please let me know. In that case, a larger, more complete code snippet would help me debug. Thanks!

  • 1
    I can't use UploadManger as I'll not be having all the data in hand before. MultiPartObjectAssembler is working fine for me except in the cases when upload aborts because of some error(say network issue). In these scenarios, I want the exact reason why the upload got failed and need to propagate the error. I tried MultipartManifest but it doesn't provide the cause of the failure as mentioned by you also. – akshay Oct 29 '20 at 08:20
  • Okay, that makes sense. I'm sorry, that's not possible right now. I have taken reporting the failure reason as a feature request, and we'll add that to a future version. If you like, you can create a GitHub issue as well. That allows others to request the feature as well. – Mathias Ricken Oct 29 '20 at 16:48
  • Thanks. I'll create a Github issue – akshay Oct 30 '20 at 05:41
  • Hi @akshay, my teammate Yash would like me to inform you that this change has now been implemented and released. Java SDK (v1.27.1) has been added with the feature to get a list of failed parts along with the Exception that caused the parts to fail. `MultipartManifest` now exposes the function `MultipartManifest.listFailedPartsDetails()` which returns a list of `MultipartUploadFailedPartDetails`, which comprises of the part number and the failureCause. Please check your GitHub issue for details: https://github.com/oracle/oci-java-sdk/issues/264 – Mathias Ricken Dec 09 '20 at 20:27