Azure Data Factory Plugin
The plugin provides functionality to interact with Azure Data Factory
Installation
implementation(group: 'org.vividus', name: 'vividus-plugin-azure-data-factory', version: '0.3.11')
Configuration
Authentication
The authentication process relies on the configuration of the environment variables.
See the official "Azure identity" guide to get more details on what types of authentication could be used.
Azure environment selection
Azure environment
could be optionally specified using either global property azure.environment
(sets the environment for all Azure plugins) or plugin-specific property
azure.data-factory.environment
. The plugin-specific property takes
precedence over the global one. The default value is AZURE
.
The supported environments are only:
-
AZURE
-
AZURE_CHINA
-
AZURE_GERMANY
-
AZURE_US_GOVERNMENT
Steps
Run Pipeline with expected status
Creates a run of a pipeline in Data Factory, waits for its completion or until the timeout is reached and validates the run status is equal to the expected one.
When I run pipeline `$pipelineName` in Data Factory `$factoryName` from resource group `$resourceGroupName` with wait timeout `$waitTimeout` and expect run status to be equal to `$expectedPipelineRunStatus`
-
$pipelineName
- The name of the pipeline to run. -
$factoryName
- The name of the factory. -
$resourceGroupName
- The name of the resource group of the factory. -
$waitTimeout
- The maximum duration of time to wait for the pipeline completion in ISO-8601 format. -
$expectedPipelineRunStatus
- The expected pipeline run status, e.g.Succeeded
.
When I run pipeline `vividus-pipeline` in Data Factory `vividus-data-factory` from resource group `vividus-resource-group-ingestion` with wait timeout `PT30S` and expect run status to be equal to `Succeeded`
Run Pipeline
This step is deprecated and will be removed in VIVIDUS 0.4.0. The replacement is the step accepting an expected status as input parameter. |
Creates a run of a pipeline in Data Factory, waits for its completion or until
the timeout is reached and validates the run status is Succeeded
.
When I run pipeline `$pipelineName` in Data Factory `$factoryName` from resource group `$resourceGroupName` with wait timeout `$waitTimeout`
-
$pipelineName
- The name of the pipeline to run. -
$factoryName
- The name of the factory. -
$resourceGroupName
- The name of the resource group of the factory. -
$waitTimeout
- The maximum duration of time to wait for the pipeline completion in ISO-8601 format.
When I run pipeline `vividus-pipeline` in Data Factory `vividus-data-factory` from resource group `vividus-resource-group-ingestion` with wait timeout `PT30S`