trying to connect to Azure from my Java code for test automation and run data factory pipeline

Question

We are in process to try and create Java based test automation

1. The build will be in Azure DevOps
2. Will run a Data Factory Pipeline
3. Check Status

I am looking for help with being able to Connect to Azure (options so as to achieve CI)

Please help suggest, preferably with some Snipets.

Answer 1

To establish connection to Azure data factory, add below package to your Product

<dependency>
    <groupId>com.azure.resourcemanager</groupId>
    <artifactId>azure-resourcemanager-datafactory</artifactId>
    <version>1.0.0-beta.16</version>
</dependency>

Azure Active Directory token authentication depends on correct configuration of following environment variables.

• AZURE_CLIENT_ID for Azure client ID.

• AZURE_TENANT_ID for Azure tenant ID.

• AZURE_CLIENT_SECRET or AZURE_CLIENT_CERTIFICATE_PATH for client secret or client certificate.

You can follow the Java code given below to create Data Factory resource.

// container
final String containerName = "adf";
storageManager.blobContainers().defineContainer(containerName)
    .withExistingStorageAccount(resourceGroup, STORAGE_ACCOUNT)
    .withPublicAccess(PublicAccess.NONE)
    .create();

// blob as input
BlobClient blobClient = new BlobClientBuilder()
    .connectionString(connectionString)
    .containerName(containerName)
    .blobName("input/data.txt")
    .buildClient();
blobClient.upload(BinaryData.fromString("data"));

// data factory
Factory dataFactory = manager.factories().define(DATA_FACTORY)
    .withRegion(REGION)
    .withExistingResourceGroup(resourceGroup)
    .create();


// pipeline
PipelineResource pipeline = manager.pipelines().define("CopyBlobPipeline")
    .withExistingFactory(resourceGroup, DATA_FACTORY)
    .withActivities(Collections.singletonList(new CopyActivity()
        .withName("CopyBlob")
        .withSource(new BlobSource())
        .withSink(new BlobSink())
        .withInputs(Collections.singletonList(new DatasetReference().withReferenceName(inputDatasetName)))
        .withOutputs(Collections.singletonList(new DatasetReference().withReferenceName(outputDatasetName)))))
    .create();

// run pipeline
CreateRunResponse createRun = pipeline.createRun();

// wait for completion
PipelineRun pipelineRun = manager.pipelineRuns().get(resourceGroup, DATA_FACTORY, createRun.runId());
String runStatus = pipelineRun.status();
while ("InProgress".equals(runStatus)) {
    sleepIfRunningAgainstService(10 * 1000);    // wait 10 seconds
    pipelineRun = manager.pipelineRuns().get(resourceGroup, DATA_FACTORY, createRun.runId());
    runStatus = pipelineRun.status();
}

For more information follow this documentation