0

I try to upload a file as a byte[] to an Azure Storage Account this way:

    public void uploadFilesInBatch(Map<Document, byte[]> documents) throws InterruptedException, IOException {
//        int numThreads = Runtime.getRuntime().availableProcessors();
    int numThreads = 6;

    ExecutorService executorService = Executors.newFixedThreadPool(numThreads);

    List<Runnable> tasks = new ArrayList<>();

    TikaConfig config = TikaConfig.getDefaultConfig();

    for (Map.Entry<Document, byte[]> entry : documents.entrySet()) {
        MediaType mediaType = config.getMimeRepository().detect(new ByteArrayInputStream(entry.getValue()), new Metadata());
        tasks.add(() -> {
            String fileName = entry.getKey().getFileName();
            BlobClient blobClient = blobContainerClient.getBlobClient(fileName);
            // synchronized (blobClient) {
                blobClient.setHttpHeaders(new BlobHttpHeaders().setContentType(mediaType.getType()));
//                blobClient.setHttpHeaders(new 
BlobHttpHeaders().setContentType(mediaType.toString()));
                blobClient.upload(new ByteArrayInputStream(entry.getValue()));
            // }
        });
    }

    for (Runnable task : tasks) {
        executorService.submit(task);
    }

    executorService.shutdown();
    executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
}

It works well inclusively setting the HttpHeaders but the following upload statement is never reached and I don't understand the reason. If I set a breakpoint in the blobClient.setHttpHeaders() line and in the blobClient.upload() line as well, IntelliJ stops in the blobClient.setHttpHeaders() line. Stepping further the next task is added to the tasks ArrayList.

I am using Java 11 and these Azure dependencies:

<azure-core.version>1.36.0</azure-core.version>
<spring-cloud-azure-starter-keyvault-secrets.version>4.3.0</spring-cloud-azure-starter-keyvault-secrets.version> <!-- Version 5.0.0 requires Java major version 61 instead of 55 -->
<azure-storage-blob.version>12.21.0</azure-storage-blob.version>
<azure-storage-blob-batch.version>12.18.0</azure-storage-blob-batch.version>

Azurite runs in a Docker container: "mcr.microsoft.com/azure-storage/azurite:3.23.0"

Maybe it's of of interest that the mentioned uploadFilesBatch() method is called from a method which implements code within

public myMethod() {
    return CompletableFuture.supplyAsync(() -> { ... }
}
du-it
  • 2,561
  • 8
  • 42
  • 80

1 Answers1

0

I solved it using BlobParallelUploadOptions along with BlobClient.uploadWithResponse():

for (Map.Entry<Document, byte[]> entry : documents.entrySet()) {
    MediaType mediaType = config.getMimeRepository().detect(new ByteArrayInputStream(entry.getValue()), new Metadata());
    tasks.add(() -> {
        String fileName = entry.getKey().getFileName();
        BlobClient blobClient = blobContainerClient.getBlobClient(fileName);
        BlobParallelUploadOptions bpuo = new BlobParallelUploadOptions(new ByteArrayInputStream(entry.getValue()))
            .setHeaders(new BlobHttpHeaders().setContentType(mediaType.getType()));
        blobClient.uploadWithResponse(bpuo, Context.NONE);
    });
}
du-it
  • 2,561
  • 8
  • 42
  • 80