3

I am making a custom activity in ADF, which involves reading multiple files from Azure Storage Blobs, doing some work on them, and then finally writing a resulting file to the Azure Data Lake Store. Last step is where I stop, because as far as I can see, the .NET SDK only allows for uploading from a local file.

Is there any way to to (programmatically) upload a file to ADL Store, where it is not from a local file? Could be a blob or a stream. If not, any workarounds?

Anders
  • 894
  • 2
  • 10
  • 25

1 Answers1

1

Yes, it's possible to upload from Stream, the trick is to create file first and then append your stream to it:

string dataLakeAccount = "DLSAccountName";
var adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(credentials);
adlsFileSystemClient.FileSystem.Create(dataLakeAccount, filepath, overwrite: true);
adlsFileSystemClient.FileSystem.Append(dataLakeAccount, filepath, stream);

See also this article.

arghtype
  • 4,376
  • 11
  • 45
  • 60
  • 1
    Excellent, thank you! On a side note, I had already seen the article referred, and strangely it doesn't mention the .Create method. – Anders Oct 10 '17 at 05:17