Applitools Plugin
The plugin provides the ability to perform various kinds of testing using the Applitools services such as Eyes, Ultrafast Grid and Execution Cloud.
Installation
-
Copy the below line to
dependencies
section of the projectbuild.gradle
filePlease make sure to use the same version for all VIVIDUS dependencies. Example 1. build.gradleimplementation(group: 'org.vividus', name: 'vividus-plugin-applitools', version: '0.6.10')
-
If the project was imported to the IDE before adding new dependency, re-generate the configuration files for the used IDE and then refresh the project in the used IDE.
General properties
The following properties should be configured prior to start working with any of the Applitools services.
The properties marked with bold are mandatory. |
Property Name | Default | Description | ||
---|---|---|---|---|
|
The URI of the rest server. |
|||
|
|
Allows users to execute tests but does not allow them to access the tests and test results, change data through the API, or make any changes to the tests. For more details see how to obtain API keys. |
||
|
Application under test |
The name of the application under test. |
Visual Testing
Applitools Eyes and Ultrafast Grid are UI validation tools powered by AI to help write & maintain visual tests and rapidly test across different browsers or devices for massive scalability.
Properties
General visual testing properties
The properties marked with bold are mandatory. |
Property Name | Default | Description |
---|---|---|
|
|
Allows users to access tests and test results, but does not allow them to execute tests, change data through the API, or make any changes to the tests. For more details see how to obtain API keys. Note if this key is not set, incorrect or does not have the necessary permission to read, screenshots of visual tests will not be available in the report and a warning message will be displayed. |
|
|
|
|
|
If |
|
|
If |
Cross-environment properties
See cross-environment testing for more details.
Property Name | Default | Description |
---|---|---|
|
|
The host OS running the application under test. |
|
|
The application running the application under test (e.g. Safari, Chrome). |
|
|
The required viewport size for the environment. |
|
|
The name of the baseline’s environment. |
Properties-based screenshot configuration for web applications
Properties-based screenshot configuration can be specified using web.screenshot.strategy.<YOUR_STRATEGY_NAME>.<PROPERTY_NAME>=<PROPERTY_VALUE>
pattern,
where:
-
YOUR_STRATEGY_NAME
- name of the custom strategy; -
PROPERTY_NAME
- name of the property you want to specify; -
PROPERTY_VALUE
- value of the property in the required format.
Keep in mind that properties-based configuration can be overridden by the step with custom step configuration.
Property Name | Acceptable values | Description |
---|---|---|
|
size in pixels |
Web header to cut (could be useful if your site has sticky header) |
|
size in pixels |
Web footer to cut (could be useful if your site has sticky footer) |
|
size in pixels |
Native header to cut |
|
size in pixels |
Native footer to cut |
|
|
Adds margins to coordinates, default one |
|
The duration in ISO-8601 Durations format. |
Timeout for scrolling during performing visual check |
|
locator |
Locator of element for performing scroll action during visual check, steps fails if an element by the locator does not exist |
|
|
Screenshot shooting strategy name ( |
|
size in pixels |
The size of top part of the final screenshot to cut. |
|
size in pixels |
The size of bottom part of the final screenshot to cut. |
|
size in pixels |
The size of left part of the final screenshot to cut. |
|
size in pixels |
The size of right part of the final screenshot to cut. |
|
string |
After configuration necessary properties use this one to enable your custom strategy |
Mind that cut-* modifications applied to the final screenshot, for example final stitched screenshot will be cut for the viewport-pasting strategy.
|
Modifiers to cut web header or web footer have the default value 100 px (cutting headers and footers up to 100 pixels high).
|
Common visual testing actions use cases
ESTABLISH
Creates a baseline image.
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName |batchName |baselineName |action |
|example.com|example-testing|main-app-page|ESTABLISH|
Established baseline on Applitools
![Establish view](_images/applitools_establish_view.png)
Established baseline details
![Establish step editor](_images/applitools_establish_step_editor.png)
COMPARE_AGAINST
Compares actual appearance of the page or element against the baseline. Make sure that the baseline you compare against actually exists.
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName |batchName |baselineName |action |elementsToIgnore|
|example.com|example-testing|main-app-page|COMPARE_AGAINST|By.tagName(h1) |
Comparison result against the established baseline on Applitools
![Compare view](_images/applitools_compare_view.png)
Comparison result details
![Compare step editor](_images/applitools_compare_step_editor.png)
CHECK_INEQUALITY_AGAINST
Work the same way as COMPARE_AGAINST
except it checks if actual appearance differs from the baseline.
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName |batchName |baselineName |action |elementsToIgnore|
|example.com|example-testing|main-app-page|CHECK_INEQUALITY_AGAINST|By.tagName(h1) |
Steps
Visual check with default Applitools configuration
Performs visual check on the Applitools visual testing plarform.
When I $actionType baseline `$testName` in batch `$batchName` with Applitools
-
$actionType
- The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`. -
$testName
- The name of the baseline (visual test name). -
$batchName
- The name of the batch (group of visual tests).
When I ESTABLISH baseline `main-app-page` in batch `example-testing` with Applitools
Visual check with custom Applitools configuration
Performs visual check on the Applitools visual testing platform with custom Applitools configuration.
Properties in the custom Applitools configuration provided in step take presendence over properties specified in configuration files.
It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.
When I run visual test with Applitools using:$applitoolsConfigurations
The expression parameters marked with bold are mandatory. |
-
$applitoolsConfigurations
- The Applitools configurations-
action
- The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`. -
batchName
- The name of the batch (group of visual tests). -
baselineName
- The name of the baseline (visual test name). -
matchLevel
- The match level. -
appName
- The name of the application under test. -
elementsToIgnore
- The comma separated list of locators of elements to ignore. -
areasToIgnore
- The comma separated list of locators of page areas to ignore. -
hostApp
- The application running the application under test (e.g. Safari, Chrome). -
hostOS
- The host OS running the application under test. -
viewportSize
- The required viewport size for the environment. -
baselineEnvName
- The name of the baseline’s environment. -
searchContext
- The locator of a search context -
readApiKey
- The key to execute tests. -
executeApiKey
- The key to access tests and test results. -
serverUri
- The URI of the rest server. -
accessibilityStandard
- The accessibility standard to check the page against. The required value format is<standard> - <level>
, where standard is eitherWCAG 2.0
orWCAG 2.1
and level is eitherAA
orAAA
. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account. -
disableBrowserFetching
- By default, isfalse
. Iftrue
, page resources for rendering on the UFG will be fetched from outside of the browser. -
layoutBreakpoints
- By default, istrue
. Iftrue
, the viewport width and height are changed to the specified device’s values prior to capturing the page source for rendering in UFG.
-
When I run visual test with Applitools using:
|appName |action |batchName |baselineName |matchLevel|
|Demo app|COMPARE_AGAINST|example-testing|main-page-app|EXACT |
Visual check with custom Applitools and screenshot configuration
Performs visual check on the Applitools visual testing platform with custom configuration and screenshot strategy.
Properties in the custom Applitools configuration provided in step take presendence over properties specified in configuration files.
It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.
When I run visual test with Applitools using:$applitoolsConfigurations and screenshot config:$screenshotConfiguration
The expression parameters marked with bold are mandatory. |
-
$applitoolsConfigurations
- The Applitools configurations-
action
- The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`. -
batchName
- The name of the batch (group of visual tests). -
baselineName
- The name of the baseline (visual test name). -
matchLevel
- The match level. -
appName
- The name of the application under test. -
elementsToIgnore
- The comma separated list of locators of elements to ignore. -
areasToIgnore
- The comma separated list of locators of page areas to ignore. -
hostApp
- The application running the application under test (e.g. Safari, Chrome). -
hostOS
- The host OS running the application under test. -
viewportSize
- The required viewport size for the environment. -
baselineEnvName
- The name of the baseline’s environment. -
searchContext
- The locator of a search context -
readApiKey
- The key to execute tests. -
executeApiKey
- The key to access tests and test results. -
serverUri
- The URI of the rest server. -
accessibilityStandard
- The accessibility standard to check the page against. The required value format is<standard> - <level>
, where standard is eitherWCAG 2.0
orWCAG 2.1
and level is eitherAA
orAAA
. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account. -
disableBrowserFetching
- By default, isfalse
. Iftrue
, page resources for rendering on the UFG will be fetched from outside of the browser. -
layoutBreakpoints
- By default, istrue
. Iftrue
, the viewport width and height are changed to the specified device’s values prior to capturing the page source for rendering in UFG.
-
-
$screenshotConfiguration
- The screenshot configurations.
Web screenshot configuration fields:
-
shootingStrategy
- The shooting strategy to use. -
webHeaderToCut
- The header to cut on each screenshot segment during scroll e.g. sticky header. -
webFooterToCut
- The footer to cut on each screenshot segment during scroll e.g. sticky footer. -
nativeHeaderToCut
- The header to cut on final screenshot. -
nativeFooterToCut
- The footer to cut on final screenshot. -
coordsProvider
- The coords provider used to get ignoring elements position during scroll. -
cutTop
- The size in pixels of top part of the image to cut. -
cutLeft
- The size in pixels of left part of the image to cut. -
cutRight
- The size in pixels of right part of the image to cut. -
cutBottom
- The size in pixels of bottom part of the image to cut.-
CEILING
- Based on JS API. -
WEB_DRIVER
- Based on WebDriver API.
-
-
scrollableElement
- The element to scroll. -
scrollTimeout
- The scroll timeout.
Mobile app screenshot configuration fields:
-
shootingStrategy
- The shooting strategy to use. -
cutTop
- The size in pixels of top part of the image to cut. -
cutLeft
- The size in pixels of left part of the image to cut. -
cutRight
- The size in pixels of right part of the image to cut. -
cutBottom
- The size in pixels of bottom part of the image to cut.
When I run visual test with Applitools using:
|appName |action |batchName |baselineName |matchLevel|
|Demo app|COMPARE_AGAINST|example-testing|main-page-app|EXACT |
and screenshot config:
|webFooterToCut|screenshotShootingStrategy|
|50 |SIMPLE |
Ultrafast Grid check with default Applitools configuration
Performs visual check on the Ultrafast Grid Applitools visual testing plarform.
When I $actionType baseline `$testName` in batch `$batchName` with Applitools UFG using matrix:$matrix
The step parameters marked with bold are mandatory. |
-
$actionType
- The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`. -
$testName
- The name of the baseline (visual test name). -
$batchName
- The name of the batch (group of visual tests). -
$matrix
- The matrix describing target platforms to run visual tests on.-
profile
- The profile name.-
Desktop
- the desktop profile with supported options:-
browser
- The browser. -
viewportSize
- The viewport size, e.g. 1920x1080
-
-
iOS
- the iOS profile with supported options:-
deviceName
- The device name. -
screenOrientation
- The screen orientation, eitherportrait
orlandscape
, default value isportrait
. -
version
- The iOS version, eitherlatest
orone version back
, default value islatest
.
-
-
Chrome Mobile Emulation
- the Chrome Mobile Emulation profile with supported options:-
deviceName
- The device name. -
screenOrientation
- The screen orientation, eitherportrait
orlandscape
, default value isportrait
.
-
-
-
When I ESTABLISH baseline `Check site on several configurations` in batch `Smoke Run` with Applitools UFG using matrix:
|profile |browser|viewportSize|deviceName|
|desktop |chrome |1920x1080 | |
|ios | | |iPhone X |
|chrome_mobile_emulation| | |Galaxy S10|
Ultrafast Grid check with custom Applitools configuration
Performs visual check on the Ultrafast Grid Applitools visual testing plarform.
The custom Applitools configuration provided in the step take precendence over values specified in properties.
It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.
When I run visual test with Applitools UFG using:$applitoolsConfigurations and matrix:$matrix
The step parameters marked with bold are mandatory. |
-
$applitoolsConfigurations
- The Applitools configurations-
action
- The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`. -
batchName
- The name of the batch (group of visual tests). -
baselineName
- The name of the baseline (visual test name). -
matchLevel
- The match level. -
appName
- The name of the application under test. -
elementsToIgnore
- The comma separated list of locators of elements to ignore. -
areasToIgnore
- The comma separated list of locators of page areas to ignore. -
hostApp
- The application running the application under test (e.g. Safari, Chrome). -
hostOS
- The host OS running the application under test. -
viewportSize
- The required viewport size for the environment. -
baselineEnvName
- The name of the baseline’s environment. -
searchContext
- The locator of a search context -
readApiKey
- The key to execute tests. -
executeApiKey
- The key to access tests and test results. -
serverUri
- The URI of the rest server. -
accessibilityStandard
- The accessibility standard to check the page against. The required value format is<standard> - <level>
, where standard is eitherWCAG 2.0
orWCAG 2.1
and level is eitherAA
orAAA
. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account. -
disableBrowserFetching
- By default, isfalse
. Iftrue
, page resources for rendering on the UFG will be fetched from outside of the browser. -
layoutBreakpoints
- By default, istrue
. Iftrue
, the viewport width and height are changed to the specified device’s values prior to capturing the page source for rendering in UFG.
-
-
$matrix
- The matrix describing target platforms to run visual tests on.-
profile
- The profile name.-
Desktop
- the desktop profile with supported options:-
browser
- The browser. -
viewportSize
- The viewport size, e.g. 1920x1080
-
-
iOS
- the iOS profile with supported options:-
deviceName
- The device name. -
screenOrientation
- The screen orientation, eitherportrait
orlandscape
, default value isportrait
. -
version
- The iOS version, eitherlatest
orone version back
, default value islatest
.
-
-
Chrome Mobile Emulation
- the Chrome Mobile Emulation profile with supported options:-
deviceName
- The device name. -
screenOrientation
- The screen orientation, eitherportrait
orlandscape
, default value isportrait
.
-
-
-
When I run visual test with Applitools UFG using:
|baselineName |batchName|action |
|Check site on several configurations|Smoke Run|ESTABLISH|
and matrix:
|profile |browser|viewportSize|deviceName|screenOrientation|version|
|desktop |firefox|1920x1080 | | | |
|ios | | |iPhone X |portrait |latest |
|chrome_mobile_emulation| | |Galaxy S10|landscape | |
When I run visual test with Applitools UFG using:
|baselineName |batchName |action |accessibilityStandard|
|Check site accessibility|Accessibility Run|ESTABLISH|WCAG 2.1 - AA |
and matrix:
|profile |browser|viewportSize|deviceName|
|desktop |firefox|1920x1080 | |
|ios | | |iPhone X |
|chrome_mobile_emulation| | |Galaxy S10|
Execution cloud
Execution Cloud is a self-healing test infrastructure for running resilient tests.
Properties
The properties marked with bold are mandatory. |
Property Name | Default | Description |
---|---|---|
|
|
The batch name used to group stories being run under one batch (i.e. suite name) |
configuration.profiles=applitools/web,web/desktop/chrome
applitools.server-uri=https://privatecloud.applitools.com/
applitools.execute-api-key=cee293b00ee1440bcd20
applitools.app-name=Grocery Store App
applitools.execution-cloud.batch-name=Smoke suite