Azure Storage Account Plugin

The plugin provides functionality to interact with Azure Storage Account.

Installation

Example 1. build.gradle
implementation(group: 'org.vividus', name: 'vividus-plugin-azure-storage-account', version: '0.4.8')

Common configuration

Authentication

The authentication process relies on the configuration of the environment variables.

See the official "Azure identity" guide to get more details on what types of authentication could be used.

Blob storage

Properties

The blob storages endpoints are configured with the following property format:

azure.storage-account.<key>=<endpoint>

where:

  1. key - The logical key used to refer the blob storage in the steps.

  2. endpoint - The blob service URL, e.g. https://mystorage.blob.core.windows.net/. It’s allowed to use blob service SAS URL here, in this case the authentication configuration is not required.

Steps

Retrieve the blob service properties

Retrieves the properties of a storage account’s Blob service and saves them as JSON to a variable. For more information, see the Azure Docs.

When I retrieve blob service properties of storage account `$storageAccountKey` and save them to $scopes variable `$variableName`
Example 2. Retrieve the blob service properties
When I retrieve blob service properties of storage account `testaccount` and save them to scenario variable `blob-service-properties`

Download the blob

Downloads the entire blob from the container and saves its content as a text to a variable.

When I download blob with name `$blobName` from container `$containerName` of storage account `$storageAccountKey` and save its content to $scopes variable `$variableName`
  • $blobName - The full path to the blob in the container.

  • $containerName - The name of the container to point to.

  • $storageAccountKey - The key of storage account from the configuration.

  • $scopes - The comma-separated set of the variables scopes.

  • $variableName - The variable name to store the blob content.

Example 3. Download the blob
When I download blob with name `raw/test.json` from container `vividus` of storage account `testaccount` and save its content to scenario variable `blob`

Download the blob to a temporary file

Downloads the entire blob from the container into a temporary file with the specified name and saves the full path to the specified variable.

When I download blob with name `$blobName` from container `$containerName` of storage account `$storageAccountKey` to temporary file with name `$baseFileName` and save path to $scopes variable `$variableName`
  • $blobName - The full path to the blob in the container.

  • $containerName - The name of the container to point to.

  • $storageAccountKey - The key of storage account from the configuration.

  • $baseFileName - The base file name used to generate the prefix and the suffix for the creating temporary file.

  • $scopes - The comma-separated set of the variables scopes.

  • $variableName - The variable name to store the path to the temporary file with the blob content.

Example 4. Download the blob to a temporary file
When I download blob with name `raw/test.json` from container `vividus` of storage account `testaccount` to temporary file with name `test.json` and save path to scenario variable `$pathToBlob`

Retrieve the blob properties

Retrieves the blob properties (all user-defined metadata, standard HTTP properties, and system properties for the blob) and saves them as JSON to a variable.

When I retrieve properties of blob with name `$blobName` from container `$containerName` of storage account `$storageAccountKey` and save them to $scopes variable `$variableName`
  • $blobName - The full path to the blob in the container.

  • $containerName - The name of the container to point to.

  • $storageAccountKey - The key of storage account from the configuration.

  • $scopes - The comma-separated set of the variables scopes.

  • $variableName - The variable name to store the blob properties.

Example 5. Retrieve the blob properties
When I retrieve properties of blob with name `raw/test.json` from container `vividus` of storage account `testaccount` and save them to scenario variable `blob-properties`

Upload the blob

Uploads the blob to the container.

When I upload blob with name `$blobName` and data `$data` to container `$containerName` of storage account `$storageAccountKey`
  • $blobName - The full path to the blob in the container.

  • $data - The data to store as blob.

  • $containerName - The name of the container to point to.

  • $storageAccountKey - The key of storage account from the configuration.

Example 6. Upload the text blob
When I upload blob with name `raw/test.json` and data `{"test":"data"}` to container `vividus` of storage account `testaccount`
Example 7. Upload the binary blob
When I upload blob with name `folder/image.jpg` and data `#{loadBinaryResource(/data/image.jpg)}` to container `vividus` of storage account `testaccount`

Upsert the blob

Uploads the blob to the container. If blob already exists it will be replaced.

When I upsert blob with name `$blobName` and data `$data` to container `$containerName` of storage account `$storageAccountKey`
  • $blobName - The full path to the blob in the container.

  • $data - The data to store as blob.

  • $containerName - The name of the container to point to.

  • $storageAccountKey - The key of storage account from the configuration.

Example 8. Upsert the blob
When I upsert blob with name `raw/test.json` and data `{"test":"data"}` to container `vividus` of storage account `testaccount`

Delete the blob

Deletes the specified blob from the container.

When I delete blob with name `$blobName` from container `$containerName` of storage account `$storageAccountKey`
  • $blobName - The full path to the blob in the container.

  • $containerName - The name of the container to point to.

  • $storageAccountKey - The key of storage account from the configuration.

Example 9. Delete the blob
When I delete blob with name `raw/test.json` from container `vividus` of storage account `testaccount`

Find blobs

Finds the blobs with names matching the specified comparison rule.

When I find all blobs with name which $comparisonRule `$blobNameToMatch` in container `$containerName` of storage account `$storageAccountKey` and save result to $scopes variable `$variableName`
Example 10. Find blobs
When I find all blobs with name which contains `json` in container `vividus` of storage account `testaccount` and save result to scenario variable `blobsCollection`

Find blobs by filter

Finds blobs with names filtered by the specified rules in the container.

At least one rule is required.
When I filter blobs by:$filter in container `$containerName` of storage account `$storageAccountKey` and save result to $scopes variable `$variableName`
  • $filter - ExamplesTable with filter rules to apply on blob names.

    Table 1. The supported filter rules
    Filter Description Example value

    blobNamePrefix

    The prefix which blob names should start with.

    path/to/data/ (equal to /path/to/data)

    resultsLimit

    The maximum number of blob names to return.

    10

    blobNameFilterRule

    The blob name comparison rule. It should be specified along with blobNameFilterValue.

    MATCHES

    blobNameFilterValue

    The full or partial blob name to be matched. It should be specified along with blobNameFilterRule.

    .*

  • $containerName - The name of the container to point to.

  • $storageAccountKey - The key of storage account from the configuration.

  • $scopes - The comma-separated set of the variables scopes.

  • $variableName - The variable name to store the list of found blob names.

Example 11. Find blobs by filter
When I filter blobs by:
|blobNameFilterRule|blobNameFilterValue|blobNamePrefix                       |resultsLimit |
|matches           |.*_test            |images_for_testing_05.10.2021/images/|10           |
in container `vividus` of storage account `testaccount` and save result to story variable `imagePathList`

Storage account management

Configuration

Azure environment selection

Azure environment could be optionally specified using either global property azure.environment (sets the environment for all Azure plugins) or plugin-specific property azure.storage-account.environment. The plugin-specific property takes precedence over the global one. The default value is AZURE.

The plugin-specific property azure.storage-account.environment is deprecated and will be removed in VIVIDUS 0.5.0. The global property must be used.

The supported environments are only:

  • AZURE

  • AZURE_CHINA

  • AZURE_GERMANY

  • AZURE_US_GOVERNMENT

Azure subscription selection

Azure subscription must be configured via AZURE_SUBSCRIPTION_ID environment variable.

Steps

Collect storage accounts info

Collects storage accounts info in the specified resource group and saves it as JSON to a variable. For more information, see the Azure Docs.

When I collect storage accounts in resource group `$resourceGroupName` and save them as JSON to $scopes variable `$variableName`

The client should have permission to run action Microsoft.Storage/storageAccounts/read over scope /subscriptions/{subscription ID}/resourceGroups/{resource group name}/providers/Microsoft.Storage.

Example 12. Collect storage accounts info
When I collect storage accounts in resource group `TEST-SA` and save them as JSON to scenario variable `storage-accounts`

Retrieve the blob service properties

Retrieves the properties of a storage account’s Blob service, including properties for Storage Analytics and CORS (Cross-Origin Resource Sharing) rules, and saves them as JSON to a variable. For more information, see the Azure Docs.

When I retrieve blob service properties of storage account with name `$storageAccountName` from resource group `$resourceGroupName` and save them as JSON to $scopes variable `$variableName`
  • $storageAccountName - The name of the storage account within the specified resource group.

  • $resourceGroupName - The name of the resource group within the user’s subscription to retrieve the storage account from. The name is case-insensitive.

  • $scopes - The comma-separated set of the variables scopes.

  • $variableName - The variable name to store the blob service properties as JSON.

Example 13. Retrieve the blob service properties
When I retrieve blob service properties of storage account with name `testaccountname` from resource group `TEST-SA` and save them as JSON to scenario variable `blob-service-properties`