Applitools Plugin

The plugin provides the ability to perform visual testing using the Applitools visual testing platform.

Installation

Example 1. build.gradle
implementation(group: 'org.vividus', name: 'vividus-plugin-applitools', version: '0.5.0')

Properties

General properties

The properties marked with bold are mandatory.
Property Name Default Description

ui.visual.applitools.server-uri

https://eyesapi.applitools.com

The URI of the rest server.

ui.visual.applitools.execute-api-key

empty

Allows users to execute tests but does not allow them to access the tests and test results, change data through the API, or make any changes to the tests. For more details see how to obtain API keys.

ui.visual.applitools.read-api-key

<empty>

Allows users to access tests and test results, but does not allow them to execute tests, change data through the API, or make any changes to the tests. For more details see how to obtain API keys. Note if this key is not set, incorrect or does not have the necessary permission to read, screenshots of visual tests will not be available in the report and a warning message will be displayed.

ui.visual.applitools.match-level

EXACT

See available match levels.

ui.visual.applitools.app-name

<empty>

The name of the application under test.

Cross-environment properties

See cross-environment testing for more details.

Property Name Default Description

ui.visual.applitools.host-os

empty

The host OS running the application under test.

ui.visual.applitools.host-app

empty

The application running the application under test (e.g. Safari, Chrome).

ui.visual.applitools.viewport-size

empty

The required viewport size for the environment.

ui.visual.applitools.baseline-env-name

empty

The name of the baseline’s environment.

Common visual testin actions use cases

ESTABLISH

Creates a baseline image.

Example 2. Establish baseline
Given I am on a page with the URL 'https://example.com/'
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action   |
|example.com|example-testing|main-app-page|ESTABLISH|

Established baseline on Applitools

Establish view

Established baseline details

Establish step editor

COMPARE_AGAINST

Compares actual appearance of the page or element against the baseline. Make sure that the baseline you compare against actually exists.

Example 3. Compare against established baseline
Given I am on a page with the URL 'https://example.com/'
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action         |elementsToIgnore|
|example.com|example-testing|main-app-page|COMPARE_AGAINST|By.tagName(h1)  |

Comparison result against the established baseline on Applitools

Compare view

Comparison result details

Compare step editor

CHECK_INEQUALITY_AGAINST

Work the same way as COMPARE_AGAINST except it checks if actual appearance differs from the baseline.

Example 4. Check inequality established baseline
Given I am on a page with the URL 'https://example.com/'
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action                  |elementsToIgnore|
|example.com|example-testing|main-app-page|CHECK_INEQUALITY_AGAINST|By.tagName(h1)  |

Steps

Visual check with default Applitools configuration

Performs visual check on the Applitools visual testing plarform.

When I $actionType baseline `$testName` in batch `$batchName` with Applitools

  • $actionType - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

  • $testName - The name of the baseline (visual test name).

  • $batchName - The name of the batch (group of visual tests).

Example 5. Perform visual check using default configuration
When I ESTABLISH baseline `main-app-page` in batch `example-testing` with Applitools

Visual check with custom Applitools configuration

Performs visual check on the Applitools visual testing plarform with custom Applitools configuration.

Properties in the custom Applitools configuration provided in step take presendence over properties specified in configuration files.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools using:$applitoolsConfigurations
The expression parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

Example 6. Perform visual check using custom configuration
When I run visual test with Applitools using:
|appName |action         |batchName      |baselineName |matchLevel|
|Demo app|COMPARE_AGAINST|example-testing|main-page-app|EXACT     |

Visual check with custom Applitools and screenshot configuration

Performs visual check on the Applitools visual testing plarform with custom configuration and screenshot strategy.

Properties in the custom Applitools configuration provided in step take presendence over properties specified in configuration files.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools using:$applitoolsConfigurations and screenshot config:$screenshotConfiguration
The expression parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

  • $screenshotConfiguration - The screenshot configurations.

Web screenshot configuration fields:

  • shootingStrategy - The shooting strategy to use.

  • webHeaderToCut - The header to cut on each screenshot segment during scroll e.g. sticky header.

  • webFooterToCut - The footer to cut on each screenshot segment during scroll e.g. sticky footer.

  • nativeHeaderToCut - The header to cut on final screenshot.

  • nativeFooterToCut - The footer to cut on final screenshot.

  • coordsProvider - The coords provider used to get ignoring elements position during scroll.

  • cutTop - The size in pixels of top part of the image to cut.

  • cutLeft - The size in pixels of left part of the image to cut.

  • cutRight - The size in pixels of right part of the image to cut.

  • cutBottom - The size in pixels of bottom part of the image to cut.

    • CEILING - Based on JS API.

    • WEB_DRIVER - Based on WebDriver API.

  • scrollableElement - The element to scroll.

  • scrollTimeout - The scroll timeout.

Mobile app screenshot configuration fields:

  • shootingStrategy - The shooting strategy to use.

  • cutTop - The size in pixels of top part of the image to cut.

  • cutLeft - The size in pixels of left part of the image to cut.

  • cutRight - The size in pixels of right part of the image to cut.

  • cutBottom - The size in pixels of bottom part of the image to cut.

Example 7. Perform visual check using custom Applitools and screenshot configuration
When I run visual test with Applitools using:
|appName |action         |batchName      |baselineName |matchLevel|
|Demo app|COMPARE_AGAINST|example-testing|main-page-app|EXACT     |
 and screenshot config:
|webFooterToCut|screenshotShootingStrategy|
|50            |SIMPLE                    |