0

Is there any way to upload files from local to ADLS using Java SDK? I have tried using the following code to upload but the code is not getting compiled as it says the constructor is not visible.

com.microsoft.azure.datalake.store.Utils ut=new com.microsoft.azure.datalake.store.Utils(adlStoreclient);

ut.upload(filePathToUpload, fileToUpload, IfExists.OVERWRITE);
Jay Gong
  • 23,163
  • 2
  • 27
  • 32
Abhra Ray
  • 99
  • 1
  • 10
  • Did you check the official [sample](https://github.com/Azure-Samples/data-lake-store-java-upload-download-get-started)? Because your code snippet differs a lot from the provided sample.... – astaykov Feb 14 '18 at 19:50

2 Answers2

0

I checked the official doc, there are create,list,delete methods etc. except upload and download method in Java SDK.

Then I searched the Azure Data Lake java sdk source code , it also doesn't seem to contain upload and download method in ADLStoreClient Class.

Based on this situation, I suggest you using REST API to upload file to ADL.

You need to follow this doc to upload data into ADL via REST API. How to get the application id and secretkey you could refer to official document. More detail steps to get the permission to access the datalake you could refer to another SO thread.

Get Access Token sample code:

private static String clientId = "***";
private static String authTokenEndpoint = "https://login.microsoftonline.com/***/oauth2/token";
private static String clientKey = "***";

public static void main(String[] args) throws IOException {
        AccessTokenProvider provider = new ClientCredsTokenProvider(authTokenEndpoint, clientId, clientKey);
        String accessToken=provider.getToken().accessToken;
}

Then use this Upload Data steps to upload your local file into ADL.

Hope it helps you.

Jay Gong
  • 23,163
  • 2
  • 27
  • 32
0

I managed to solve the issue. The SDK provides a method to upload files like I mentioned in my question. Also, I had mentioned that the constructor was not visible. So, I found the upload methods from the sdk here (https://github.com/Azure/azure-data-lake-store-java/blob/master/src/main/java/com/microsoft/azure/datalake/store/Utils.java) and implemented the same concept to stream the data to ADLS from my local file.

Abhra Ray
  • 99
  • 1
  • 10