Azure Data Factory Plugin

The plugin provides functionality to interact with Azure Data Factory

Installation

Example 1. build.gradle
implementation(group: 'org.vividus', name: 'vividus-plugin-azure-data-factory', version: '0.3.4')

Configuration

Authentication

The authentication process relies on the configuration of the environment variables.

See the official "Azure identity" guide to get more details on what types of authentication could be used.

Properties

The properties marked with bold are mandatory.
Property name Acceptable values Default Description

azure.data-factory.environment

AZURE

AZURE_CHINA

AZURE_GERMANY

AZURE_US_GOVERNMENT

AZURE

The Azure environment

azure.data-factory.tenant-id

UUID

<empty>

The Azure tenant ID

azure.data-factory.subscription-id

UUID

<empty>

The Azure subscription ID

Steps

Run Pipeline

Creates a run of a pipeline in Data Factory, waits for its completion or until the timeout is reached and validates the run status is Succeeded.

When I run pipeline `$pipelineName` in DataFactory `$factoryName` from resource group `$resourceGroupName` with wait timeout `$waitTimeout`

  • $pipelineName - The name of the pipeline to run.

  • $factoryName - The name of the factory.

  • $resourceGroupName - The name of the resource group of the factory.

  • $waitTimeout - The maximum duration of time to wait for the pipeline completion in ISO-8601 format.

Example 2. Run pipeline
When I run pipeline `vividus-pipeline` in Data Factory `vividus-data-factory` from resource group `vividus-resource-group-ingestion` with wait timeout `PT30S`