5

I have PowerShell script which splits a complex CSV file to a smaller CSV file for every 1000 records. Here is the code:

$i=0;Get-Content C:\Users\dell\Desktop\Powershell\Input\bigsizeFile.csv -ReadCount 1000 | %{$i++; $_ | Out-File C:\Users\dell\Desktop\Powershell\Output\file$i.csv
}

Now I want to use this script in Azure PowerShell and I want to run this from Azure Data Factory. Can Someone please help with this.

kenlukas
  • 3,616
  • 9
  • 25
  • 36
Rahul Reddy Vudem
  • 51
  • 1
  • 1
  • 5

2 Answers2

2

I was able to run a PowerShell script by loading the script as an application in the Batch account associated to the Batch Custom Activity. From there it was figuring out the correct syntactical sugar to use the environment variables to get the path to my script and run it. I did this like so:

powershell powershell -command (\"(Get-ChildItem Env:AZ_BATCH_APP_PACKAGE_powershellscripts#1.0).Value\" + '\\Powershell\\processFromAzure.ps1

If you just want to call from the task dir, this should work:

powershell powershell -command ("$env:AZ_BATCH_TASK_DIR" + '\wd\processFromAzure.ps1')

Hope this helps someone else out there!

user2197446
  • 1,065
  • 3
  • 15
  • 31
  • Sorry to revive this - but I'm at my wits end trying to run a PS Script through Custom Activity - and I was interested in " ... by loading the script as an application in the Batch account associated to the Batch Custom Activity". Is there any chance you could explain this to me? If for example, I had a PS Script here > https://PSTestBlob.blob.core.windows.net/TestingBlo/PowerShell/ps_test.ps1 what do I do next? – Celador Sep 24 '20 at 09:23
1

you could execute your powershell command by using Custom activity in ADFv2. Here is an example.

Wang Zhang
  • 317
  • 1
  • 3
  • 1
    The example is running a .Net activity. Getting a powershell script to run via ADFv2 is a very different process. Yes a custom activity is the answer, but more details on how to setup the custom activity to have access to the storage to find and reference the input file and output directory correctly would help immensely. – user2197446 Oct 25 '19 at 14:41