Azure Storage Account Plugin

The plugin provides functionality to interact with Azure Storage Account.

Installation

Copy the below line to dependencies section of the project build.gradle file

Example 1. build.gradle

implementation(group: 'org.vividus', name: 'vividus-plugin-azure-storage-account', version: '0.5.2')

If the project was imported to the IDE before adding new plugin, re-generate the configuration files for the used IDE and then refresh the project in the used IDE.

Blob storage

Configuration

Authentication

The authentication process relies on the configuration of the environment variables.

See the official "Azure identity" guide to get more details on what types of authentication could be used.

Endpoints

The blob storages endpoints are configured with the following property format:

azure.storage-account.<key>.blob-service=<endpoint>

where:

key - The logical key used to refer the storage account in the steps.
endpoint - The blob service URL, e.g. https://mystorageaccount.blob.core.windows.net/. It’s allowed to use blob service SAS URL here, in this case the authentication configuration is not required.

Steps

Retrieve the blob service properties

Retrieves the properties of a storage account’s Blob service and saves them as JSON to a variable. For more information, see the Azure Docs.

When I retrieve blob service properties of storage account `$storageAccountKey` and save them to $scopes variable `$variableName`

$storageAccountKey - The key of storage account from the configuration.
$scopes - The comma-separated set of the variables scopes.
$variableName - The variable name to store the blob service properties.

Example 2. Retrieve the blob service properties

When I retrieve blob service properties of storage account `testaccount` and save them to scenario variable `blob-service-properties`

Download the blob

Downloads the entire blob from the container and saves its content as a text to a variable.

When I download blob with name `$blobName` from container `$containerName` of storage account `$storageAccountKey` and save its content to $scopes variable `$variableName`

$blobName - The full path to the blob in the container.
$containerName - The name of the container to point to.
$storageAccountKey - The key of storage account from the configuration.
$scopes - The comma-separated set of the variables scopes.
$variableName - The variable name to store the blob content.

Example 3. Download the blob

When I download blob with name `raw/test.json` from container `vividus-container` of storage account `testaccount` and save its content to scenario variable `blob`

Download the blob to a temporary file

Downloads the entire blob from the container into a temporary file with the specified name and saves the full path to the specified variable.

When I download blob with name `$blobName` from container `$containerName` of storage account `$storageAccountKey` to temporary file with name `$baseFileName` and save path to $scopes variable `$variableName`

$blobName - The full path to the blob in the container.
$containerName - The name of the container to point to.
$storageAccountKey - The key of storage account from the configuration.
$baseFileName - The base file name used to generate the prefix and the suffix for the creating temporary file.
$scopes - The comma-separated set of the variables scopes.
$variableName - The variable name to store the path to the temporary file with the blob content.

Example 4. Download the blob to a temporary file

When I download blob with name `raw/test.json` from container `vividus-container` of storage account `testaccount` to temporary file with name `test.json` and save path to scenario variable `pathToBlob`

Retrieve the blob properties

Retrieves the blob properties (all user-defined metadata, standard HTTP properties, and system properties for the blob) and saves them as JSON to a variable.

When I retrieve properties of blob with name `$blobName` from container `$containerName` of storage account `$storageAccountKey` and save them to $scopes variable `$variableName`

$blobName - The full path to the blob in the container.
$containerName - The name of the container to point to.
$storageAccountKey - The key of storage account from the configuration.
$scopes - The comma-separated set of the variables scopes.
$variableName - The variable name to store the blob properties.

Example 5. Retrieve the blob properties

When I retrieve properties of blob with name `raw/test.json` from container `vividus-container` of storage account `testaccount` and save them to scenario variable `blob-properties`

Upload the blob

Uploads the blob to the container.

When I upload blob with name `$blobName` and data `$data` to container `$containerName` of storage account `$storageAccountKey`

$blobName - The full path to the blob in the container.
$data - The data to store as blob.
$containerName - The name of the container to point to.
$storageAccountKey - The key of storage account from the configuration.

Example 6. Upload the text blob

When I upload blob with name `raw/test.json` and data `{"test":"data"}` to container `vividus` of storage account `testaccount`

Example 7. Upload the binary blob

When I upload blob with name `folder/image.jpg` and data `#{loadBinaryResource(/data/image.jpg)}` to container `vividus` of storage account `testaccount`

Upsert the blob

Uploads the blob to the container. If blob already exists it will be replaced.

When I upsert blob with name `$blobName` and data `$data` to container `$containerName` of storage account `$storageAccountKey`

$blobName - The full path to the blob in the container.
$data - The data to store as blob.
$containerName - The name of the container to point to.
$storageAccountKey - The key of storage account from the configuration.

Example 8. Upsert the blob

When I upsert blob with name `raw/test.json` and data `{"test":"data"}` to container `vividus` of storage account `testaccount`

Delete the blob

Deletes the specified blob from the container.

When I delete blob with name `$blobName` from container `$containerName` of storage account `$storageAccountKey`

$blobName - The full path to the blob in the container.
$containerName - The name of the container to point to.
$storageAccountKey - The key of storage account from the configuration.

Example 9. Delete the blob

When I delete blob with name `raw/test.json` from container `vividus-container` of storage account `testaccount`

Find blobs

Finds the blobs with names matching the specified comparison rule.

When I find all blobs with name which $comparisonRule `$blobNameToMatch` in container `$containerName` of storage account `$storageAccountKey` and save result to $scopes variable `$variableName`

$comparisonRule - The blob name comparison rule.
$blobNameToMatch - The full or partial blob name to be matched.
$containerName - The name of the container to point to.
$storageAccountKey - The key of storage account from the configuration.
$scopes - The comma-separated set of the variables scopes.
$variableName - The variable name to store the list of found blob names.

Example 10. Find blobs

When I find all blobs with name which contains `json` in container `vividus` of storage account `testaccount` and save result to scenario variable `blobsCollection`

Find blobs by filter

Finds blobs with names filtered by the specified rules in the container.

At least one rule is required.

When I filter blobs by:$filter in container `$containerName` of storage account `$storageAccountKey` and save result to $scopes variable `$variableName`

$filter - The ExamplesTable with filter rules to apply on blob names.

Table 1. The supported filter rules
Filter	Description	Example value
`blobNamePrefix`	The prefix which blob names should start with.	`path/to/data/` (equal to `/path/to/data`)
`resultsLimit`	The maximum number of blob names to return.	`10`
`blobNameFilterRule`	The blob name comparison rule. It should be specified along with `blobNameFilterValue`.	`MATCHES`
`blobNameFilterValue`	The full or partial blob name to be matched. It should be specified along with `blobNameFilterRule`.	`.*`

$containerName - The name of the container to point to.
$storageAccountKey - The key of storage account from the configuration.
$scopes - The comma-separated set of the variables scopes.
$variableName - The variable name to store the list of found blob names.

Example 11. Find blobs by filter

When I filter blobs by:
|blobNameFilterRule|blobNameFilterValue|blobNamePrefix                       |resultsLimit |
|matches           |.*_test            |images_for_testing_05.10.2021/images/|10           |
in container `vividus` of storage account `testaccount` and save result to story variable `imagePathList`

File share

Configuration

Endpoints

The file share endpoints are configured with the following property format:

azure.storage-account.<key>.file-service=<endpoint>

where:

key - The logical key used to refer the storage account in the steps.
endpoint - The file service URL, e.g. https://mystorageaccount.file.core.windows.net/. It’s recommended to use file service SAS URL here, because the authentication via Service Principals is not available for file shares.

Steps

Download the file

Downloads the file from the file share and saves its content as a text to a variable.

When I download file with path `$filePath` from file share `$shareName` of storage account `$storageAccountKey` and save its content to $scopes variable `$variableName`

$filePath - The full path to the file in the file share.
$shareName - The name of the file share to point to.
$storageAccountKey - The key of the storage account file service endpoint from the configuration.
$scopes - The comma-separated set of the variables scopes.
$variableName - The variable name to store the file content.

Example 12. Download the file

When I download file with path `data/file.json` from file share `vividus-file-share` of storage account `testaccount` and save its content to scenario variable `json-file`

Download the file to a temporary file

Downloads the file from the file share into a temporary file with the specified name and saves the full path to the specified variable.

When I download file with path `$filePath` from file share `$shareName` of storage account `$storageAccountKey` to temporary file with name `$baseFileName` and save path to $scopes variable `$variableName`

$filePath - The full path to the file in the file share.
$shareName - The name of the file share to point to.
$storageAccountKey - The key of the storage account file service endpoint from the configuration.
$baseFileName - The base file name used to generate the prefix and the suffix for the creating temporary file.
$scopes - The comma-separated set of the variables scopes.
$variableName - The variable name to store the path to the temporary file with the file content.

Example 13. Download the file to a temporary file

When I download file with path `data/file.json` from file share `vividus-file-share` of storage account `testaccount` to temporary file with name `test.json` and save path to scenario variable `path-to-file`

Upload the file

Uploads the file to the file share.

When I upload file with path `$filePath` and data `$data` to file share `$shareName` of storage account `$storageAccountKey`

$filePath - The full path to the creating file in the file share.
$data - The data to store as a file.
$shareName - The name of the file share to point to.
$storageAccountKey - The key of the storage account file service endpoint from the configuration.

Example 14. Upload the text file

When I upload file with path `data/file.json` and data `{"test":"data"}` to file share `vividus-file-share` of storage account `testaccount`

Example 15. Upload the binary file

When I upload file with path `images/file.jpg` and data `#{loadBinaryResource(/data/image.jpg)}` to file share `vividus-file-share` of storage account `testaccount`

Delete the file

Deletes the specified file from the file share.

When I delete file with path `$filePath` from file share `$shareName` of storage account `$storageAccountKey`

$filePath - The full path to the file in the file share.
$shareName - The name of the file share to point to.
$storageAccountKey - The key of the storage account file service endpoint from the configuration.

Example 16. Delete the file

When I delete file with path `data/file.json` from file share `vividus-file-share` of storage account `testaccount`

Storage account management

Configuration

Azure environment selection

Azure environment could be optionally specified using the property azure.environment (sets the environment for all Azure plugins). The default value is AZURE.

The supported environments are only:

AZURE
AZURE_CHINA
AZURE_GERMANY
AZURE_US_GOVERNMENT

Azure subscription selection

Azure subscription must be configured via AZURE_SUBSCRIPTION_ID environment variable.

Steps

Collect storage accounts info

The step is deprecated and will be removed in VIVIDUS 0.6.0. The replacement is the generic step retrieving information about Azure resources. The replacement pattern is:

When I get Azure resource with identifier `resourceGroups/<resourceGroupName>/providers/Microsoft.Storage/storageAccounts` using API version `2021-08-01` and save it to <scopes> variable `<variableName>`

Collects storage accounts info in the specified resource group and saves it as JSON to a variable. For more information, see the Azure Docs.

When I collect storage accounts in resource group `$resourceGroupName` and save them as JSON to $scopes variable `$variableName`

$resourceGroupName - The name of the resource group to list the storage accounts from.
$scopes - The comma-separated set of the variables scopes.
$variableName - The variable name to store the info about storage accounts as JSON.

The client should have permission to run action Microsoft.Storage/storageAccounts/read over scope /subscriptions/{subscription ID}/resourceGroups/{resource group name}/providers/Microsoft.Storage.

Example 17. Collect storage accounts info

When I collect storage accounts in resource group `TEST-SA` and save them as JSON to scenario variable `storage-accounts`

Retrieve the blob service properties

The step is deprecated and will be removed in VIVIDUS 0.6.0. The replacement is the generic step retrieving information about Azure resources. The replacement pattern is:

When I get Azure resource with identifier `resourceGroups/<resourceGroupName>/providers/Microsoft.Storage/storageAccounts/<storageAccountName>/blobServices/default` using API version `2021-08-01` and save it to <scopes> variable `<variableName>`

Retrieves the properties of a storage account’s Blob service, including properties for Storage Analytics and CORS (Cross-Origin Resource Sharing) rules, and saves them as JSON to a variable. For more information, see the Azure Docs.

When I retrieve blob service properties of storage account with name `$storageAccountName` from resource group `$resourceGroupName` and save them as JSON to $scopes variable `$variableName`

$storageAccountName - The name of the storage account within the specified resource group.
$resourceGroupName - The name of the resource group within the user’s subscription to retrieve the storage account from. The name is case-insensitive.
$scopes - The comma-separated set of the variables scopes.
$variableName - The variable name to store the blob service properties as JSON.

Example 18. Retrieve the blob service properties

When I retrieve blob service properties of storage account with name `testaccountname` from resource group `TEST-SA` and save them as JSON to scenario variable `blob-service-properties`