Azure DevOps Exporter

Azure DevOps Exporter is a tool used to export test cases into Azure Test Plans.

Features:

  • Create and update Test Cases. VIVIDUS scenarios are mapped to Azure DevOps Test Cases by using testCaseId meta tags.

  • Create automated Test Runs.

Azure DevOps Export Properties

The properties marked with bold are mandatory.
Property Default value Acceptable values Description

azure-devops-exporter.json-results-directory

string

Path to directory with test execution JSON results.

The path is managed by the bdd.report-directory property which default value is output/reports/jbehave relatively to the tests directory.

Please make sure that the bdd.configuration.formats property includes JSON value in order to generate test execution results.

azure-devops-exporter.organization

string

The name of the Azure DevOps organization.

azure-devops-exporter.project

string

Project ID or project name.

azure-devops-exporter.area

string

The area of the product which a test case is associated with.

If the Area name is inherited from the Project name in AzureDevOps this property should be empty. You can check this by making a request for any Work Item within current Area and checking if the System.AreaPath contains any data besides the Project name.

azure-devops-exporter.section-mapping.steps

AUTOMATED

AUTOMATED or MANUAL

The scenario part to export into Steps section of Test Case work item, the remaining scenario part is exported into Summary section.

azure-devops-exporter.create-test-run

false

true or false

Whether to create test run.

In Azure DevOps only Test Cases with associated automation can be included into automated Test Run. The associated automation is automatically added by exported only for Automated Test Cases. Attempts to include Manual Test Case into automated Test Run will fail the execution process.

azure-devops-exporter.test-run.name

string

The test run name.

This property is mandatory when the test run creation is enabled, see azure-devops-exporter.create-test-run property for details.

azure-devops-exporter.test-run.test-plan-id

number

The ID of Azure DevOps Test Plan that the Azure DevOps Test Run belongs to.

In Azure DevOps Test Plan is used to group individual Test Cases and Test Suites, in their turn Test Suites are used to group Test Cases into separate testing scenarios within a single Test Plan and also may include child Test Suites. Azure DevOps allows to have an individual Test Case to be included into several Test Suites and into the Test Plan, BUT this is not supported by the exporter which requires the Test Case to be linked to either Test Plan or only one Test Suite, the export process fails if this condition is not met.

This property is mandatory when the test run creation is enabled, see azure-devops-exporter.create-test-run property for details.

The ID can be found by opening appropriate Azure DevOps Test Plan in "Test Plans" and copying number value of the planId query parameter in the page URL, e.g. having URL https://dev.azure.com/organization/project/_testPlans/define?planId=911 the plan ID is 911.

Authentication properties

Property Acceptable values Description

azure-devops.username

string

Username from Azure DevOps

azure-devops.password

string

Personal access token

Scenario Meta Attributes

Name Example Description

testCaseId

@testCaseId 911

Map scenario to Azure DevOps test case in 1 to 1 relation

azure-devops.skip-export

@azure-devops.skip-export

Skip test case while exporting

Test Case Types

This separation of automated and manual test cases by using special keywords is specific to VIVIDUS.

Automated Test Cases

Any scenario that doesn’t correspond to the manual test case rules is considered as automated test case.

Example 1. Automated.story
Scenario: Verify link
Meta: @testCaseId TEST-231

Given I am on a page with the URL '${vividus-test-site-url}/links.html'
When I wait until the page has the title 'Links'
Then number of elements found by `<locator>` is equal to `1`
Examples:
|locator                        |
|By.linkUrl(#ElementId)         |
|By.linkUrlPart(Element)        |
|By.linkText(Link to an element)|
Example 2. AutomatedWithManualPart.story
Scenario: Verify link
Meta: @testCaseId TEST-566

!-- Step: Open main app page
!-- Step: Wait for page with title is loaded
!-- Step: Verify number of links
!-- Data: * link url is '#ElementId'
!-- * link url part is 'Element'
!-- * link text is 'Link to an element'
!-- Result: The number of links for all locators is equal to 1

Given I am on a page with the URL '${vividus-test-site-url}/links.html'
When I wait until the page has the title 'Links'
Then number of elements found by `<locator>` is equal to `1`
Examples:
|locator                        |
|By.linkUrl(#ElementId)         |
|By.linkUrlPart(Element)        |
|By.linkText(Link to an element)|

Manual Test Cases

  • Scenario is considered as Manual Test Case if all of its lines start with !-- prefix.

  • The Manual Test Case step must start with Step: used to specify action to perform and can optionally contain Data: and Result: for specifying action data and action expected result respectively.

  • The Manual Test Case step parts are allowed to have multiple lines.

  • The JBehave Keywords values (e.g. Given, When, Then …​) on new lines not prefixed with Step:, Data: or Result must be escaped with - sign.

Example 3. Manual.story
Scenario: Buy an item
Meta: @testCaseId TEST-435

!-- Step: Go to the test item with the following id
!-- Data: 39914061
!-- Result: The current stock is 1

!-- Step: Add the item to the shopping cart
!-- Result: Shopping cart now displays one 39914061 item and no items in stock

!-- Step: In the backoffice app update the stock to 0 on item

!-- Step: Back in the browser, proceed to checkout.
!-- Go through all the steps and confirm the payment method.
!-- Result: Then you should get an error message mentioning that there are no more items in the stock.
!-- - Then payment method was not made

Export

  • Go to the corresponding GitHub Packages repository

  • Find the link with name vividus-to-azure-devops-exporter-0.4.16.jar

  • Click the link and download the exporter JAR-file

  • Create a new file with name application.properties and put all necessary properties there

  • Run the following command replacing <jar-file name> with the name of the JAR-file downloaded previously

java -jar <jar-file name> -Dspring.config.location=classpath:/application.properties,./application.properties