Tests Configuration
configuration.properties
The Vividus configuration includes the following parts: profiles, environments and suites.
Suites
The property configuration.suites
defines the suites set. It is a comma separated set of suite file addresses:
configuration.suites=testSuite,anotherDir/anotherTestSuite
The Vividus suite consists of the unlimited number of batches. In its turn the batch consists of any number of stories represented by the files with *.story
extension. The batches are run sequentially, while the stories in the batch are run in parallel.
The suites are loaded one by one starting from the last one in the sequence, in other words, the values of the properties defined in the first suite will take precedence over the values of the same properties defined in all subsequent suites.
The following properties are used to configure batches (Vividus doesn’t provide any default batch configuration out of the box). batch-number
is one-based batch index.
The properties marked with bold are mandatory. |
Property | Default value | Description |
---|---|---|
|
The test project path to the folder with story-files |
|
|
|
The comma-separated set of ant-patterns to match the stories to be included in the batch |
|
|
The comma-separated set of ant-patterns to match the stories to be excluded from the batch |
|
|
The name of the batch to display in the report |
|
|
The number of parallel threads used to run stories |
|
suite meta-filter |
The meta-filter used to filter the batch stories and scenarios |
|
|
If |
|
|
The max duration of the single story in the batch. |
|
|
If set to |
|
|
If set to |
|
|
If set to |
|
|
If set to |
Known Issues
Known issues mechanism allows to distinguish failures from known issues presented in the system under test.
JSON file should contain a valid JSON. If JSON is not valid known issues will be ignored and warning about invalid file will be printed. |
Properties
Property | Default value | Description |
---|---|---|
|
|
Defines the location of the known issues file |
|
|
Defines are potentially known issues detected |
Known Issues file
The file represents a collection of JSON elements describing known issues.
{
"VVD-1": { (1)
"type": "Internal",(2)
"assertionPattern": "Expected: a value equal to 200, Actual: \\[404\\]" (3)
"storyPattern": "Validate the Math", (4)
"scenarioPattern": "Validate equality", (5)
"stepPattern": "Then `1` is = `2`.+", (6)
"variablePatterns": { (7)
"var": "value-.*"
},
"failScenarioFast": true, (8)
"failStoryFast": true (9)
}
}
1 | [Mandatory] Identifier (This could be an ID of the issue in your bug tracking system) |
2 | [Mandatory] Issue type. One of: INTERNAL (application issue), EXTERNAL (3rd party issue), AUTOMATION (test automation issue) |
3 | [Mandatory] Assertion pattern describes which failed assertions should be considered as known |
4 | Story pattern to match failed story name |
5 | Scenario pattern to match failed scenario name |
6 | Step pattern to match failed step. |
7 | Variable patterns. Patterns to match variables. |
8 | Skip the rest of scenario in case of failure identified as this known issue |
9 | Skip the rest of story in case of failure identified as this known issue |
Use \\ for escaping of special characters in regular expressions |
Examples
-
To consider failed assertion known or potentially-known assertionPattern should match assertion failure, if defined variablePatterns should match also.
-
If assertionPattern/variablePatterns matched and you have story/scenario/step/Pattern defined, to consider issue known all of them should match.
-
If assertionPattern/variablePatterns matched and you have story/scenario/step/Pattern defined, when at least one not matched issue is considered potentially known.
Meta tags
Meta tags can be used to reach several goals:
-
Grouping of stories and scenarios in the report.
-
Dynamic selection of tests to execute by meta tags.
-
Management of capabilities for the particular tests.
Levels
Meta tags may be used at both story and scenario level. Meta tags specified at the story level are propagated to every scenario, if the same meta tag is used at the scenario level, then the scenario level value takes precedence.
!-- Story level
Meta:
@group Login
Scenario: Open Epam main page
!-- Scenario level
Meta:
@severity 1
@layout desktop
Given I am on a page with the URL 'https://www.epam.com/'
Then the page title is equal to 'EPAM | Enterprise Software Development, Design & Consulting'
Scenario: Open Google main page
Meta:
@severity 2
@layout desktop
@skip
Given I am on a page with the URL 'https://www.google.com/'
Then the page title is equal to 'Google'
Properties
The prorerty bdd.all-meta-filters
is used to override values of the default meta filters based on the tags: @skip
, @layout
, @browserWindowSize
and etc.
The property bdd.meta-filters
relies on groovy synax can be used to filters tests to be executed.
bdd.meta-filters=groovy: (group == 'Login' && !skip)
The following meta tags are available out of the box.
2D tags (@key value)
Tag Key | Allowed values | Description | Example |
---|---|---|---|
@severity |
Numeric values (range 1-5) |
Used to mark importance of the test where, most usually, the smallest number is the most important Test, the highest number is the least important. |
|
@layout |
|
Used to specify layout, using which marked story or scenario should be executed. In case if this meta tag is not specified, default ( |
|
@browserWindowSize |
Any browser window size in pixels, e.g. |
Used to specify browser window size. It can be applied on both story and scenario levels. In case, if browser size differs between adjacent scenarios, browser window will be simply resized to desired values, browser session will not be restarted. If the tag is not set for story or scenario, the browser window will be maximized to the screen size during local test execution. |
|
Reporting
User has possibility to link stories/scenarios to an external systems like TMS or bug-tracking systems.
There is a few steps to achieve this:
-
Configure link pattern via properties
-
Add metatag to the story/sceanrio meta
system.allure.link.issue.pattern=https://vividus.dev/issues/{}
Meta:
@issueId VVD-1
Scenario: Should check variables equals
Then `1` is = `1`

Configuration
Property | Meta tag | Description | Example |
---|---|---|---|
system.allure.link.issue.pattern |
@issueId |
Defines issue link pattern |
|
system.allure.link.requirement.pattern |
@requirementId |
Defines requirement link pattern |
|
system.allure.link.tms.pattern |
@testCaseId |
Defines test case link pattern |
Mutiple endpoints
It is possible to have multiple endpoints. To achieve this user should define system suffix for the properties and meta tags.
# Default property
system.allure.link.issue.pattern=https://vividus.prod/issues/{}
# Additional property with .dev suffix, that used to create links annotated with @isssueId.dev
system.allure.link.issue.dev.pattern=https://vividus.dev/issues/{}
Meta:
@issueId VVD-1
@issueId.dev DEV-1
Scenario: Should check variables equal
Then `1` is = `1`

Story Prioritization
In order to start stories in particular order it is possible to assign the stories a numeric priority, the stories with the higher priority start first.
To enable the feature create a file spring.xml
in the src/main/resources
directory and add the following XML data into that file.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans https://www.springframework.org/schema/beans/spring-beans.xsd"
default-lazy-init="false">
<bean class="org.vividus.bdd.priority.MetaBasedStoryExecutionPriority" factory-method="byNumericMetaValue">
<constructor-arg index="0" value="story_priority" />
</bean>
</beans>
The story_priority
value that corresponds to a meta name in stories can be changed to arbitrary value without spaces.
Taking story_priority
as an example the priority meta should be placed at the story level as the following example shows:
Meta: @story_priority 15
Scenario: Time consuming test
When I perform highly time consuming task