Applitools Plugin

The plugin provides the ability to perform various kinds of testing using the Applitools services such as Eyes, Ultrafast Grid and Execution Cloud.

Installation

  1. Copy the below line to dependencies section of the project build.gradle file

    Please make sure to use the same version for all VIVIDUS dependencies.
    Example 1. build.gradle
    implementation(group: 'org.vividus', name: 'vividus-plugin-applitools', version: '0.6.1')

  2. If the project was imported to the IDE before adding new dependency, re-generate the configuration files for the used IDE and then refresh the project in the used IDE.

General properties

The following properties should be configured prior to start working with any of the Applitools services.

The properties marked with bold are mandatory.
Property Name Default Description

applitools.server-uri

ui.visual.applitools.server-uri property is deprecated in favor of applitools.server-uri and will be removed in VIVIDUS 0.7.0.

https://eyesapi.applitools.com

The URI of the rest server.

applitools.execute-api-key

ui.visual.applitools.execute-api-key property is deprecated in favor of applitools.execute-api-key and will be removed in VIVIDUS 0.7.0.

<empty>

Allows users to execute tests but does not allow them to access the tests and test results, change data through the API, or make any changes to the tests. For more details see how to obtain API keys.

applitools.app-name

ui.visual.applitools.app-name property is deprecated in favor of applitools.app-name and will be removed in VIVIDUS 0.7.0.

Application under test

The name of the application under test.

Visual Testing

Applitools Eyes and Ultrafast Grid are UI validation tools powered by AI to help write & maintain visual tests and rapidly test across different browsers or devices for massive scalability.

Properties

General visual testing properties

The properties marked with bold are mandatory.
Property Name Default Description

ui.visual.applitools.read-api-key

<empty>

Allows users to access tests and test results, but does not allow them to execute tests, change data through the API, or make any changes to the tests. For more details see how to obtain API keys. Note if this key is not set, incorrect or does not have the necessary permission to read, screenshots of visual tests will not be available in the report and a warning message will be displayed.

ui.visual.applitools.match-level

EXACT

See available match levels.

Cross-environment properties

See cross-environment testing for more details.

Property Name Default Description

ui.visual.applitools.host-os

empty

The host OS running the application under test.

ui.visual.applitools.host-app

empty

The application running the application under test (e.g. Safari, Chrome).

ui.visual.applitools.viewport-size

empty

The required viewport size for the environment.

ui.visual.applitools.baseline-env-name

empty

The name of the baseline’s environment.

Common visual testing actions use cases

ESTABLISH

Creates a baseline image.

Example 2. Establish baseline
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action   |
|example.com|example-testing|main-app-page|ESTABLISH|

Established baseline on Applitools

Establish view

Established baseline details

Establish step editor

COMPARE_AGAINST

Compares actual appearance of the page or element against the baseline. Make sure that the baseline you compare against actually exists.

Example 3. Compare against established baseline
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action         |elementsToIgnore|
|example.com|example-testing|main-app-page|COMPARE_AGAINST|By.tagName(h1)  |

Comparison result against the established baseline on Applitools

Compare view

Comparison result details

Compare step editor

CHECK_INEQUALITY_AGAINST

Work the same way as COMPARE_AGAINST except it checks if actual appearance differs from the baseline.

Example 4. Check inequality established baseline
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action                  |elementsToIgnore|
|example.com|example-testing|main-app-page|CHECK_INEQUALITY_AGAINST|By.tagName(h1)  |

Steps

Visual check with default Applitools configuration

Performs visual check on the Applitools visual testing plarform.

When I $actionType baseline `$testName` in batch `$batchName` with Applitools

  • $actionType - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

  • $testName - The name of the baseline (visual test name).

  • $batchName - The name of the batch (group of visual tests).

Example 5. Perform visual check using default configuration
When I ESTABLISH baseline `main-app-page` in batch `example-testing` with Applitools

Visual check with custom Applitools configuration

Performs visual check on the Applitools visual testing platform with custom Applitools configuration.

Properties in the custom Applitools configuration provided in step take presendence over properties specified in configuration files.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools using:$applitoolsConfigurations
The expression parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

    • accessibilityStandard - The accessibility standard to check the page against. The required value format is <standard> - <level>, where standard is either WCAG 2.0 or WCAG 2.1 and level is either AA or AAA. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account.

Example 6. Perform visual check using custom configuration
When I run visual test with Applitools using:
|appName |action         |batchName      |baselineName |matchLevel|
|Demo app|COMPARE_AGAINST|example-testing|main-page-app|EXACT     |

Visual check with custom Applitools and screenshot configuration

Performs visual check on the Applitools visual testing platform with custom configuration and screenshot strategy.

Properties in the custom Applitools configuration provided in step take presendence over properties specified in configuration files.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools using:$applitoolsConfigurations and screenshot config:$screenshotConfiguration
The expression parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

    • accessibilityStandard - The accessibility standard to check the page against. The required value format is <standard> - <level>, where standard is either WCAG 2.0 or WCAG 2.1 and level is either AA or AAA. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account.

  • $screenshotConfiguration - The screenshot configurations.

Web screenshot configuration fields:

  • shootingStrategy - The shooting strategy to use.

  • webHeaderToCut - The header to cut on each screenshot segment during scroll e.g. sticky header.

  • webFooterToCut - The footer to cut on each screenshot segment during scroll e.g. sticky footer.

  • nativeHeaderToCut - The header to cut on final screenshot.

  • nativeFooterToCut - The footer to cut on final screenshot.

  • coordsProvider - The coords provider used to get ignoring elements position during scroll.

  • cutTop - The size in pixels of top part of the image to cut.

  • cutLeft - The size in pixels of left part of the image to cut.

  • cutRight - The size in pixels of right part of the image to cut.

  • cutBottom - The size in pixels of bottom part of the image to cut.

    • CEILING - Based on JS API.

    • WEB_DRIVER - Based on WebDriver API.

  • scrollableElement - The element to scroll.

  • scrollTimeout - The scroll timeout.

Mobile app screenshot configuration fields:

  • shootingStrategy - The shooting strategy to use.

  • cutTop - The size in pixels of top part of the image to cut.

  • cutLeft - The size in pixels of left part of the image to cut.

  • cutRight - The size in pixels of right part of the image to cut.

  • cutBottom - The size in pixels of bottom part of the image to cut.

Example 7. Perform visual check using custom Applitools and screenshot configuration
When I run visual test with Applitools using:
|appName |action         |batchName      |baselineName |matchLevel|
|Demo app|COMPARE_AGAINST|example-testing|main-page-app|EXACT     |
 and screenshot config:
|webFooterToCut|screenshotShootingStrategy|
|50            |SIMPLE                    |

Ultrafast Grid check with default Applitools configuration

Performs visual check on the Ultrafast Grid Applitools visual testing plarform.

When I $actionType baseline `$testName` in batch `$batchName` with Applitools UFG using matrix:$matrix
The step parameters marked with bold are mandatory.

  • $actionType - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

  • $testName - The name of the baseline (visual test name).

  • $batchName - The name of the batch (group of visual tests).

  • $matrix - The matrix describing target platforms to run visual tests on.

    • profile - The profile name.

      • Desktop - the desktop profile with supported options:

        • browser - The browser.

        • viewportSize - The viewport size, e.g. 1920x1080

      • iOS - the iOS profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

        • version - The iOS version, either latest or one version back, default value is latest.

      • Chrome Mobile Emulation - the Chrome Mobile Emulation profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

Example 8. Perform Ultrafast Grid check using default configuration
When I ESTABLISH baseline `Check site on several configurations` in batch `Smoke Run` with Applitools UFG using matrix:
|profile                |browser|viewportSize|deviceName|
|desktop                |chrome |1920x1080   |          |
|ios                    |       |            |iPhone X  |
|chrome_mobile_emulation|       |            |Galaxy S10|

Ultrafast Grid check with custom Applitools configuration

Performs visual check on the Ultrafast Grid Applitools visual testing plarform.

The custom Applitools configuration provided in the step take precendence over values specified in properties.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools UFG using:$applitoolsConfigurations and matrix:$matrix
The step parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

    • accessibilityStandard - The accessibility standard to check the page against. The required value format is <standard> - <level>, where standard is either WCAG 2.0 or WCAG 2.1 and level is either AA or AAA. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account.

  • $matrix - The matrix describing target platforms to run visual tests on.

    • profile - The profile name.

      • Desktop - the desktop profile with supported options:

        • browser - The browser.

        • viewportSize - The viewport size, e.g. 1920x1080

      • iOS - the iOS profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

        • version - The iOS version, either latest or one version back, default value is latest.

      • Chrome Mobile Emulation - the Chrome Mobile Emulation profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

Example 9. Perform Ultrafast Grid check using custom configuration
When I run visual test with Applitools UFG using:
|baselineName                        |batchName|action   |
|Check site on several configurations|Smoke Run|ESTABLISH|
 and matrix:
|profile                |browser|viewportSize|deviceName|screenOrientation|version|
|desktop                |firefox|1920x1080   |          |                 |       |
|ios                    |       |            |iPhone X  |portrait         |latest |
|chrome_mobile_emulation|       |            |Galaxy S10|landscape        |       |
Example 10. Perform Ultrafast Grid check combined with Accessibility check
When I run visual test with Applitools UFG using:
|baselineName            |batchName        |action   |accessibilityStandard|
|Check site accessibility|Accessibility Run|ESTABLISH|WCAG 2.1 - AA        |
 and matrix:
|profile                |browser|viewportSize|deviceName|
|desktop                |firefox|1920x1080   |          |
|ios                    |       |            |iPhone X  |
|chrome_mobile_emulation|       |            |Galaxy S10|

Execution cloud

Execution Cloud is a self-healing test infrastructure for running resilient tests.

Profiles

applitools/web

Use in conjunction with web application profiles.

Properties

The properties marked with bold are mandatory.
Property Name Default Description

applitools.execution-cloud.batch-name

empty

The batch name used to group stories being run under one batch (i.e. suite name)
Example 11. configuration.properties
configuration.profiles=applitools/web,web/desktop/chrome

applitools.server-uri=https://privatecloud.applitools.com/
applitools.execute-api-key=cee293b00ee1440bcd20
applitools.app-name=Grocery Store App

applitools.execution-cloud.batch-name=Smoke suite