Applitools Plugin

The plugin provides the ability to perform various kinds of testing using the Applitools services such as Eyes, Ultrafast Grid and Execution Cloud.

Installation

  1. Copy the below line to dependencies section of the project build.gradle file

    Please make sure to use the same version for all VIVIDUS dependencies.
    Example 1. build.gradle
    implementation(group: 'org.vividus', name: 'vividus-plugin-applitools', version: '0.6.13')

  2. If the project was imported to the IDE before adding new dependency, re-generate the configuration files for the used IDE and then refresh the project in the used IDE.

General properties

The following properties should be configured prior to start working with any of the Applitools services.

The properties marked with bold are mandatory.
Property Name Default Description

applitools.server-uri

ui.visual.applitools.server-uri property is deprecated in favor of applitools.server-uri and will be removed in VIVIDUS 0.7.0.

https://eyesapi.applitools.com

The URI of the rest server.

applitools.execute-api-key

ui.visual.applitools.execute-api-key property is deprecated in favor of applitools.execute-api-key and will be removed in VIVIDUS 0.7.0.

<empty>

Allows users to execute tests but does not allow them to access the tests and test results, change data through the API, or make any changes to the tests. For more details see how to obtain API keys.

applitools.app-name

ui.visual.applitools.app-name property is deprecated in favor of applitools.app-name and will be removed in VIVIDUS 0.7.0.

Application under test

The name of the application under test.

Visual Testing

Applitools Eyes and Ultrafast Grid are UI validation tools powered by AI to help write & maintain visual tests and rapidly test across different browsers or devices for massive scalability.

Properties

General visual testing properties

The properties marked with bold are mandatory.
Property Name Default Description

ui.visual.applitools.read-api-key

<empty>

Allows users to access tests and test results, but does not allow them to execute tests, change data through the API, or make any changes to the tests. For more details see how to obtain API keys. Note if this key is not set, incorrect or does not have the necessary permission to read, screenshots of visual tests will not be available in the report and a warning message will be displayed.

ui.visual.applitools.match-level

EXACT

See available match levels.

ui.visual.applitools.disable-browser-fetching

false

If true, page resources for rendering on the UFG will be fetched from outside of the browser.

ui.visual.applitools.layout-breakpoints

true

If true the viewport width and height are changed to the specified device’s values prior to capturing the page source for rendering in UFG.

Cross-environment properties

See cross-environment testing for more details.

Property Name Default Description

ui.visual.applitools.host-os

empty

The host OS running the application under test.

ui.visual.applitools.host-app

empty

The application running the application under test (e.g. Safari, Chrome).

ui.visual.applitools.viewport-size

empty

The required viewport size for the environment.

ui.visual.applitools.baseline-env-name

empty

The name of the baseline’s environment.

Properties-based screenshot configuration for web applications

Properties-based screenshot configuration can be specified using web.screenshot.strategy.<YOUR_STRATEGY_NAME>.<PROPERTY_NAME>=<PROPERTY_VALUE> pattern, where:

  • YOUR_STRATEGY_NAME - name of the custom strategy;

  • PROPERTY_NAME - name of the property you want to specify;

  • PROPERTY_VALUE - value of the property in the required format.

Keep in mind that properties-based configuration can be overridden by the step with custom step configuration.

Property Name Acceptable values Description

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.web-header-to-cut

size in pixels

Web header to cut (could be useful if your site has sticky header)

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.web-footer-to-cut

size in pixels

Web footer to cut (could be useful if your site has sticky footer)

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.native-header-to-cut

size in pixels

Native header to cut

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.native-footer-to-cut

size in pixels

Native footer to cut

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.coords-provider

WEB_DRIVER CEILING

Adds margins to coordinates, default one

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.scroll-timeout

The duration in ISO-8601 Durations format.

Timeout for scrolling during performing visual check

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.scrollable-element

locator

Locator of element for performing scroll action during visual check, steps fails if an element by the locator does not exist

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.shooting-strategy

SIMPLE VIEWPORT_PASTING

Screenshot shooting strategy name (VIEWPORT_PASTING by default)

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.cut-top

size in pixels

The size of top part of the final screenshot to cut.

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.cut-bottom

size in pixels

The size of bottom part of the final screenshot to cut.

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.cut-left

size in pixels

The size of left part of the final screenshot to cut.

web.screenshot.strategy.<YOUR_STRATEGY_NAME>.cut-right

size in pixels

The size of right part of the final screenshot to cut.

web.screenshot.strategy=YOUR_STRATEGY_NAME

string

After configuration necessary properties use this one to enable your custom strategy
Mind that cut-* modifications applied to the final screenshot, for example final stitched screenshot will be cut for the viewport-pasting strategy.
Modifiers to cut web header or web footer have the default value 100 px (cutting headers and footers up to 100 pixels high).

Common visual testing actions use cases

ESTABLISH

Creates a baseline image.

Example 2. Establish baseline
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action   |
|example.com|example-testing|main-app-page|ESTABLISH|

Established baseline on Applitools

Establish view

Established baseline details

Establish step editor

COMPARE_AGAINST

Compares actual appearance of the page or element against the baseline. Make sure that the baseline you compare against actually exists.

Example 3. Compare against established baseline
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action         |elementsToIgnore|
|example.com|example-testing|main-app-page|COMPARE_AGAINST|By.tagName(h1)  |

Comparison result against the established baseline on Applitools

Compare view

Comparison result details

Compare step editor

CHECK_INEQUALITY_AGAINST

Work the same way as COMPARE_AGAINST except it checks if actual appearance differs from the baseline.

Example 4. Check inequality established baseline
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action                  |elementsToIgnore|
|example.com|example-testing|main-app-page|CHECK_INEQUALITY_AGAINST|By.tagName(h1)  |

Steps

Visual check with default Applitools configuration

Performs visual check on the Applitools visual testing plarform.

When I $actionType baseline `$testName` in batch `$batchName` with Applitools

  • $actionType - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

  • $testName - The name of the baseline (visual test name).

  • $batchName - The name of the batch (group of visual tests).

Example 5. Perform visual check using default configuration
When I ESTABLISH baseline `main-app-page` in batch `example-testing` with Applitools

Visual check with custom Applitools configuration

Performs visual check on the Applitools visual testing platform with custom Applitools configuration.

Properties in the custom Applitools configuration provided in step take presendence over properties specified in configuration files.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools using:$applitoolsConfigurations
The expression parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

    • accessibilityStandard - The accessibility standard to check the page against. The required value format is <standard> - <level>, where standard is either WCAG 2.0 or WCAG 2.1 and level is either AA or AAA. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account.

    • disableBrowserFetching - By default, is false. If true, page resources for rendering on the UFG will be fetched from outside of the browser.

    • layoutBreakpoints - By default, is true. If true, the viewport width and height are changed to the specified device’s values prior to capturing the page source for rendering in UFG.

Example 6. Perform visual check using custom configuration
When I run visual test with Applitools using:
|appName |action         |batchName      |baselineName |matchLevel|
|Demo app|COMPARE_AGAINST|example-testing|main-page-app|EXACT     |

Visual check with custom Applitools and screenshot configuration

Performs visual check on the Applitools visual testing platform with custom configuration and screenshot strategy.

Properties in the custom Applitools configuration provided in step take presendence over properties specified in configuration files.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools using:$applitoolsConfigurations and screenshot config:$screenshotConfiguration
The expression parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

    • accessibilityStandard - The accessibility standard to check the page against. The required value format is <standard> - <level>, where standard is either WCAG 2.0 or WCAG 2.1 and level is either AA or AAA. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account.

    • disableBrowserFetching - By default, is false. If true, page resources for rendering on the UFG will be fetched from outside of the browser.

    • layoutBreakpoints - By default, is true. If true, the viewport width and height are changed to the specified device’s values prior to capturing the page source for rendering in UFG.

  • $screenshotConfiguration - The screenshot configurations.

Web screenshot configuration fields:

  • shootingStrategy - The shooting strategy to use.

  • webHeaderToCut - The header to cut on each screenshot segment during scroll e.g. sticky header.

  • webFooterToCut - The footer to cut on each screenshot segment during scroll e.g. sticky footer.

  • nativeHeaderToCut - The header to cut on final screenshot.

  • nativeFooterToCut - The footer to cut on final screenshot.

  • coordsProvider - The coords provider used to get ignoring elements position during scroll.

  • cutTop - The size in pixels of top part of the image to cut.

  • cutLeft - The size in pixels of left part of the image to cut.

  • cutRight - The size in pixels of right part of the image to cut.

  • cutBottom - The size in pixels of bottom part of the image to cut.

    • CEILING - Based on JS API.

    • WEB_DRIVER - Based on WebDriver API.

  • scrollableElement - The element to scroll.

  • scrollTimeout - The scroll timeout.

Mobile app screenshot configuration fields:

  • shootingStrategy - The shooting strategy to use.

  • cutTop - The size in pixels of top part of the image to cut.

  • cutLeft - The size in pixels of left part of the image to cut.

  • cutRight - The size in pixels of right part of the image to cut.

  • cutBottom - The size in pixels of bottom part of the image to cut.

Example 7. Perform visual check using custom Applitools and screenshot configuration
When I run visual test with Applitools using:
|appName |action         |batchName      |baselineName |matchLevel|
|Demo app|COMPARE_AGAINST|example-testing|main-page-app|EXACT     |
 and screenshot config:
|webFooterToCut|screenshotShootingStrategy|
|50            |SIMPLE                    |

Ultrafast Grid check with default Applitools configuration

Performs visual check on the Ultrafast Grid Applitools visual testing plarform.

When I $actionType baseline `$testName` in batch `$batchName` with Applitools UFG using matrix:$matrix
The step parameters marked with bold are mandatory.

  • $actionType - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

  • $testName - The name of the baseline (visual test name).

  • $batchName - The name of the batch (group of visual tests).

  • $matrix - The matrix describing target platforms to run visual tests on.

    • profile - The profile name.

      • Desktop - the desktop profile with supported options:

        • browser - The browser.

        • viewportSize - The viewport size, e.g. 1920x1080

      • iOS - the iOS profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

        • version - The iOS version, either latest or one version back, default value is latest.

      • Chrome Mobile Emulation - the Chrome Mobile Emulation profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

Example 8. Perform Ultrafast Grid check using default configuration
When I ESTABLISH baseline `Check site on several configurations` in batch `Smoke Run` with Applitools UFG using matrix:
|profile                |browser|viewportSize|deviceName|
|desktop                |chrome |1920x1080   |          |
|ios                    |       |            |iPhone X  |
|chrome_mobile_emulation|       |            |Galaxy S10|

Ultrafast Grid check with custom Applitools configuration

Performs visual check on the Ultrafast Grid Applitools visual testing plarform.

The custom Applitools configuration provided in the step take precendence over values specified in properties.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools UFG using:$applitoolsConfigurations and matrix:$matrix
The step parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

    • accessibilityStandard - The accessibility standard to check the page against. The required value format is <standard> - <level>, where standard is either WCAG 2.0 or WCAG 2.1 and level is either AA or AAA. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account.

    • disableBrowserFetching - By default, is false. If true, page resources for rendering on the UFG will be fetched from outside of the browser.

    • layoutBreakpoints - By default, is true. If true, the viewport width and height are changed to the specified device’s values prior to capturing the page source for rendering in UFG.

  • $matrix - The matrix describing target platforms to run visual tests on.

    • profile - The profile name.

      • Desktop - the desktop profile with supported options:

        • browser - The browser.

        • viewportSize - The viewport size, e.g. 1920x1080

      • iOS - the iOS profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

        • version - The iOS version, either latest or one version back, default value is latest.

      • Chrome Mobile Emulation - the Chrome Mobile Emulation profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

Example 9. Perform Ultrafast Grid check using custom configuration
When I run visual test with Applitools UFG using:
|baselineName                        |batchName|action   |
|Check site on several configurations|Smoke Run|ESTABLISH|
 and matrix:
|profile                |browser|viewportSize|deviceName|screenOrientation|version|
|desktop                |firefox|1920x1080   |          |                 |       |
|ios                    |       |            |iPhone X  |portrait         |latest |
|chrome_mobile_emulation|       |            |Galaxy S10|landscape        |       |
Example 10. Perform Ultrafast Grid check combined with Accessibility check
When I run visual test with Applitools UFG using:
|baselineName            |batchName        |action   |accessibilityStandard|
|Check site accessibility|Accessibility Run|ESTABLISH|WCAG 2.1 - AA        |
 and matrix:
|profile                |browser|viewportSize|deviceName|
|desktop                |firefox|1920x1080   |          |
|ios                    |       |            |iPhone X  |
|chrome_mobile_emulation|       |            |Galaxy S10|

Execution cloud

Execution Cloud is a self-healing test infrastructure for running resilient tests.

Profiles

applitools/web

Use in conjunction with web application profiles.

Properties

The properties marked with bold are mandatory.
Property Name Default Description

applitools.execution-cloud.batch-name

empty

The batch name used to group stories being run under one batch (i.e. suite name)
Example 11. configuration.properties
configuration.profiles=applitools/web,web/desktop/chrome

applitools.server-uri=https://privatecloud.applitools.com/
applitools.execute-api-key=cee293b00ee1440bcd20
applitools.app-name=Grocery Store App

applitools.execution-cloud.batch-name=Smoke suite