Applitools Plugin

The plugin provides the ability to perform visual testing using the Applitools visual testing platform.

Installation

  1. Copy the below line to dependencies section of the project build.gradle file

    Example 1. build.gradle
    implementation(group: 'org.vividus', name: 'vividus-plugin-applitools', version: '0.5.10')

  2. If the project was imported to the IDE before adding new plugin, re-generate the configuration files for the used IDE and then refresh the project in the used IDE.

Properties

General properties

The properties marked with bold are mandatory.
Property Name Default Description

ui.visual.applitools.server-uri

https://eyesapi.applitools.com

The URI of the rest server.

ui.visual.applitools.execute-api-key

empty

Allows users to execute tests but does not allow them to access the tests and test results, change data through the API, or make any changes to the tests. For more details see how to obtain API keys.

ui.visual.applitools.read-api-key

<empty>

Allows users to access tests and test results, but does not allow them to execute tests, change data through the API, or make any changes to the tests. For more details see how to obtain API keys. Note if this key is not set, incorrect or does not have the necessary permission to read, screenshots of visual tests will not be available in the report and a warning message will be displayed.

ui.visual.applitools.match-level

EXACT

See available match levels.

ui.visual.applitools.app-name

<empty>

The name of the application under test.

Cross-environment properties

See cross-environment testing for more details.

Property Name Default Description

ui.visual.applitools.host-os

empty

The host OS running the application under test.

ui.visual.applitools.host-app

empty

The application running the application under test (e.g. Safari, Chrome).

ui.visual.applitools.viewport-size

empty

The required viewport size for the environment.

ui.visual.applitools.baseline-env-name

empty

The name of the baseline’s environment.

Common visual testing actions use cases

ESTABLISH

Creates a baseline image.

Example 2. Establish baseline
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action   |
|example.com|example-testing|main-app-page|ESTABLISH|

Established baseline on Applitools

Establish view

Established baseline details

Establish step editor

COMPARE_AGAINST

Compares actual appearance of the page or element against the baseline. Make sure that the baseline you compare against actually exists.

Example 3. Compare against established baseline
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action         |elementsToIgnore|
|example.com|example-testing|main-app-page|COMPARE_AGAINST|By.tagName(h1)  |

Comparison result against the established baseline on Applitools

Compare view

Comparison result details

Compare step editor

CHECK_INEQUALITY_AGAINST

Work the same way as COMPARE_AGAINST except it checks if actual appearance differs from the baseline.

Example 4. Check inequality established baseline
Given I am on page with URL `https://example.com/`
When I run visual test with Applitools using:
|appName    |batchName      |baselineName |action                  |elementsToIgnore|
|example.com|example-testing|main-app-page|CHECK_INEQUALITY_AGAINST|By.tagName(h1)  |

Steps

Visual check with default Applitools configuration

Performs visual check on the Applitools visual testing plarform.

When I $actionType baseline `$testName` in batch `$batchName` with Applitools

  • $actionType - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

  • $testName - The name of the baseline (visual test name).

  • $batchName - The name of the batch (group of visual tests).

Example 5. Perform visual check using default configuration
When I ESTABLISH baseline `main-app-page` in batch `example-testing` with Applitools

Visual check with custom Applitools configuration

Performs visual check on the Applitools visual testing platform with custom Applitools configuration.

Properties in the custom Applitools configuration provided in step take presendence over properties specified in configuration files.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools using:$applitoolsConfigurations
The expression parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

    • accessibilityStandard - The accessibility standard to check the page against. The required value format is <standard> - <level>, where standard is either WCAG 2.0 or WCAG 2.1 and level is either AA or AAA. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account.

Example 6. Perform visual check using custom configuration
When I run visual test with Applitools using:
|appName |action         |batchName      |baselineName |matchLevel|
|Demo app|COMPARE_AGAINST|example-testing|main-page-app|EXACT     |

Visual check with custom Applitools and screenshot configuration

Performs visual check on the Applitools visual testing platform with custom configuration and screenshot strategy.

Properties in the custom Applitools configuration provided in step take presendence over properties specified in configuration files.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools using:$applitoolsConfigurations and screenshot config:$screenshotConfiguration
The expression parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

    • accessibilityStandard - The accessibility standard to check the page against. The required value format is <standard> - <level>, where standard is either WCAG 2.0 or WCAG 2.1 and level is either AA or AAA. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account.

  • $screenshotConfiguration - The screenshot configurations.

Web screenshot configuration fields:

  • shootingStrategy - The shooting strategy to use.

  • webHeaderToCut - The header to cut on each screenshot segment during scroll e.g. sticky header.

  • webFooterToCut - The footer to cut on each screenshot segment during scroll e.g. sticky footer.

  • nativeHeaderToCut - The header to cut on final screenshot.

  • nativeFooterToCut - The footer to cut on final screenshot.

  • coordsProvider - The coords provider used to get ignoring elements position during scroll.

  • cutTop - The size in pixels of top part of the image to cut.

  • cutLeft - The size in pixels of left part of the image to cut.

  • cutRight - The size in pixels of right part of the image to cut.

  • cutBottom - The size in pixels of bottom part of the image to cut.

    • CEILING - Based on JS API.

    • WEB_DRIVER - Based on WebDriver API.

  • scrollableElement - The element to scroll.

  • scrollTimeout - The scroll timeout.

Mobile app screenshot configuration fields:

  • shootingStrategy - The shooting strategy to use.

  • cutTop - The size in pixels of top part of the image to cut.

  • cutLeft - The size in pixels of left part of the image to cut.

  • cutRight - The size in pixels of right part of the image to cut.

  • cutBottom - The size in pixels of bottom part of the image to cut.

Example 7. Perform visual check using custom Applitools and screenshot configuration
When I run visual test with Applitools using:
|appName |action         |batchName      |baselineName |matchLevel|
|Demo app|COMPARE_AGAINST|example-testing|main-page-app|EXACT     |
 and screenshot config:
|webFooterToCut|screenshotShootingStrategy|
|50            |SIMPLE                    |

Ultrafast Grid check with default Applitools configuration

Performs visual check on the Ultrafast Grid Applitools visual testing plarform.

When I $actionType baseline `$testName` in batch `$batchName` with Applitools UFG using matrix:$matrix
The step parameters marked with bold are mandatory.

  • $actionType - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

  • $testName - The name of the baseline (visual test name).

  • $batchName - The name of the batch (group of visual tests).

  • $matrix - The matrix describing target platforms to run visual tests on.

    • profile - The profile name.

      • Desktop - the desktop profile with supported options:

        • browser - The browser.

        • viewportSize - The viewport size, e.g. 1920x1080

      • iOS - the iOS profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

        • version - The iOS version, either latest or one version back, default value is latest.

      • Chrome Mobile Emulation - the Chrome Mobile Emulation profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

Example 8. Perform Ultrafast Grid check using default configuration
When I ESTABLISH baseline `Check site on several configurations` in batch `Smoke Run` with Applitools UFG using matrix:
|profile                |browser|viewportSize|deviceName|
|desktop                |chrome |1920x1080   |          |
|ios                    |       |            |iPhone X  |
|chrome_mobile_emulation|       |            |Galaxy S10|

Ultrafast Grid check with custom Applitools configuration

Performs visual check on the Ultrafast Grid Applitools visual testing plarform.

The custom Applitools configuration provided in the step take precendence over values specified in properties.

It’s possible to perform several visual testing checks at once by passing several Applitools configuration into step.

When I run visual test with Applitools UFG using:$applitoolsConfigurations and matrix:$matrix
The step parameters marked with bold are mandatory.

  • $applitoolsConfigurations - The Applitools configurations

    • action - The `ESTABLISH`, `COMPARE_AGAINST` or `CHECK_INEQUALITY_AGAINST`.

    • batchName - The name of the batch (group of visual tests).

    • baselineName - The name of the baseline (visual test name).

    • matchLevel - The match level.

    • appName - The name of the application under test.

    • elementsToIgnore - The comma separated list of locators of elements to ignore.

    • areasToIgnore - The comma separated list of locators of page areas to ignore.

    • hostApp - The application running the application under test (e.g. Safari, Chrome).

    • hostOS - The host OS running the application under test.

    • viewportSize - The required viewport size for the environment.

    • baselineEnvName - The name of the baseline’s environment.

    • searchContext - The locator of a search context

    • readApiKey - The key to execute tests.

    • executeApiKey - The key to access tests and test results.

    • serverUri - The URI of the rest server.

    • accessibilityStandard - The accessibility standard to check the page against. The required value format is <standard> - <level>, where standard is either WCAG 2.0 or WCAG 2.1 and level is either AA or AAA. This check uses Applitools Contrast Advisor that helps in identifying potential contrast violations in both the text and images of your web and mobile applications, helping to ensure your technology is more accessible. Please make sure Applitools Contrast Advisor feature is enabled in your Applitools account.

  • $matrix - The matrix describing target platforms to run visual tests on.

    • profile - The profile name.

      • Desktop - the desktop profile with supported options:

        • browser - The browser.

        • viewportSize - The viewport size, e.g. 1920x1080

      • iOS - the iOS profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

        • version - The iOS version, either latest or one version back, default value is latest.

      • Chrome Mobile Emulation - the Chrome Mobile Emulation profile with supported options:

        • deviceName - The device name.

        • screenOrientation - The screen orientation, either portrait or landscape, default value is portrait.

Example 9. Perform Ultrafast Grid check using custom configuration
When I run visual test with Applitools UFG using:
|baselineName                        |batchName|action   |
|Check site on several configurations|Smoke Run|ESTABLISH|
 and matrix:
|profile                |browser|viewportSize|deviceName|screenOrientation|version|
|desktop                |firefox|1920x1080   |          |                 |       |
|ios                    |       |            |iPhone X  |portrait         |latest |
|chrome_mobile_emulation|       |            |Galaxy S10|landscape        |       |
Example 10. Perform Ultrafast Grid check combined with Accessibility check
When I run visual test with Applitools UFG using:
|baselineName            |batchName        |action   |accessibilityStandard|
|Check site accessibility|Accessibility Run|ESTABLISH|WCAG 2.1 - AA        |
 and matrix:
|profile                |browser|viewportSize|deviceName|
|desktop                |firefox|1920x1080   |          |
|ios                    |       |            |iPhone X  |
|chrome_mobile_emulation|       |            |Galaxy S10|