How to create VIVIDUS automated tests with the help of AI

VIVIDUS offers MCP server that exposes core VIVIDUS functionalities to LLMs via the MCP protocol, enabling generation of automated test scenarios leveraging those capabilities.

LLM will access only those steps that are available from plugins included into your test project.

Install MCP server

  1. Copy the below line to dependencies section of the project build.gradle file in your test project

    VIVIDUS MCP server is available starting from version 0.6.16-SNAPSHOT and higher.
    Example 1. build.gradle
    implementation(group: 'org.vividus', name: 'vividus-mcp-server')

  2. Make sure to use Build System 3.0

  3. Update gradlew script in the root of your test project to the latest version:

    1. Go to https://github.com/vividus-framework/vividus-starter/blob/main/gradlew

    2. Copy content of the gradlew script

    3. Replace the content of the gradlew script in the root of the test project with the copied content

  4. If the project was imported to the IDE before adding new dependency, re-generate the configuration files for the used IDE and then refresh the project in the used IDE.

Configure AI Assistant

GitHub Copilot in VS Code

  1. Install Visual Studio Code

  2. Open the project with VIVIDUS tests in Visual Studio Code

  3. Set up Copilot in VS Code if it is not done yet

  4. Create a new file at path /.vscode/mcp.json in the root of the project

  5. Add the following configuration into mcp.json file and save it

    • macOS / Linux

    • Windows

    {
  "servers": {
    "vividus": {
      "command": "/Users/Bob/Workspace/vividus-sample-tests/gradlew", (1)
      "args": [
        "startMcpServer",
        "-q",
        "-p",
        "/Users/Bob/Workspace/vividus-sample-tests" (2)
      ]
    },
    "playwright": { (3)
      "command": "npx",
      "args": [
        "@playwright/mcp@latest"
      ]
    }
  }
}
    {
  "servers": {
    "vividus": {
      "command": "c:\\Users\\Bob\\Workspace\\vividus-sample-tests\\gradlew.bat", (1)
      "args": [
        "startMcpServer",
        "-q",
        "-p",
        "c:\\Users\\Bob\\Workspace\\vividus-sample-tests" (2)
      ]
    },
    "playwright": { (3)
      "command": "npx",
      "args": [
        "@playwright/mcp@latest"
      ]
    }
  }
}
    1 The absolute path to the gradlew or gradlew.bat executable contained in the tests project
    2 The absolute path to the tests project
    3 The Playwright MCP server is optional and can be used to control browser and generate tests for web-applications

  6. Start the MCP servers by clicking on the ▷ Start buttons on the top of the servers names

  7. Use Copilot in agent mode to generate automated scenarios

Claude Desktop

Claude is a next generation AI assistant built by Anthropic and trained to be safe, accurate, and secure to help you do your best work.

  1. Install Claude Desktop

  2. Open Claude Desktop and navigate to Developer tab in application Settings

  3. Click Edit Config which opens claude_desktop_config.json file that contains MCP server configs

  4. Add the following configuration into claude_desktop_config.json file and save it

    • macOS / Linux

    • Windows

    {
  "mcpServers": {
    "vividus": {
      "command": "/Users/Bob/Workspace/vividus-sample-tests/gradlew", (1)
      "args": [
          "startMcpServer",
          "-q",
          "-p",
          "/Users/Bob/Workspace/vividus-sample-tests" (2)
      ]
    },
    "playwright": { (3)
      "command": "npx",
      "args": [
          "@playwright/mcp"
      ]
    }
  }
}
    {
  "mcpServers": {
    "vividus": {
      "command": "c:\\Users\\Bob\\Workspace\\vividus-sample-tests\\gradlew.bat", (1)
      "args": [
          "startMcpServer",
          "-q",
          "-p",
          "c:\\Users\\Bob\\Workspace\\vividus-sample-tests" (2)
      ]
    },
    "playwright": { (3)
      "command": "npx",
      "args": [
          "@playwright/mcp"
      ]
    }
  }
}
    1 The absolute path to the gradlew or gradlew.bat executable contained in the tests project
    2 The absolute path to the tests project
    3 The Playwright MCP server is optional and can be used to control browser and generate tests for web-applications

  5. Restart Claude Desktop application

Web-application test automation with GitHub Copilot in VS Code

Prerequisites

  1. Switch to chat agent mode

  2. Choose the preferred Language Model (the list of the available models can differ depending on your license)

  3. Configure and start the MCP servers

Common Prompt Files

A prompt file stores stable instructions and constraints for the AI agent. Use it to fix style, allowed steps, locator conventions, and validation rules so every generated test follows team standards.

Benefits of prompt file usage:

  • Centralized rules for test generation.

  • Reusable across test creation sessions.

  • Easier auditing and refactoring when the rules change.

Setup prompt files

  1. Create a .github/prompts folder (if it doesn’t exist)

  2. Place the ready-made prompt files into the created folder

    The prompt file must have the .prompt.md extension (e.g., test-template.prompt.md). This extension ensures proper recognition by VS Code and GitHub Copilot

  3. Specify the path to your tests folder in the prompt.

  4. Use a prompt file in chat.

    Example 2. Prompt example test-template.prompt.md:
    Used steps should strictly follow the steps' structure, grammar, and syntax.
Use the provided VIVIDUS steps from the input list exactly as they are given.
Steps should begin with Given, When, or Then. DO NOT add additional words. Only parameters can be added.
Use real locators for real elements that from provided URL. DO NOT create non-existent locators.
Use Locator types id(idValue), xpath(xpathValue).

Writing Tests with AI Assistance

Example 3. Prompt with specifically defined scope of testing scenario:
Open `https://example.com/` page.
Close any opened pop-ups.
Check Search functionality.

This will generate a test with:

  • Browser initialization.

  • URL navigation.

  • Pop-up handling.

  • Search functionality verification.

Example 4. Prompt with general Web testing scenarios (scenarios list generated by AI):
Open `https://example.com/` page and write test automation scenarios demonstrating Vividus tool capabilities for web application testing.

This will generate multiple scenarios covering:

  • Page navigation.

  • Basic element interactions.

  • Key elements verification.

Best Practices for AI Test Generation

  1. Be Specific: Include exact URLs and scope details.

  2. State Requirements: Mention specific validations needed.

  3. Indicate Patterns or Approaches.

  4. Store Common Prompt Logic: Save common prompt logic in a separate prompt file to avoid copy-pasting across different chats and minimize session context expiration effects.

FAQ

Q: How to execute my tests automatically after creation?

Please be aware of token limits. Set up auto-execution only if you are confident and polish your prompt file.

Extension of prompt test-template.prompt.md file to include auto-run of test cases after confirmation in chat:

After generating the test script:
1. Confirm test execution in agent chat by `"Yes, execute the tests"`.
2. Execute test using сommand: ./gradlew runStories
3. Automatically execute test with:
   - Chrome desktop profile.
   - Newly created story files.
   - Real-time execution logs.
4. Provide results in the chat showing:
   - Execution status.
   - Test results summary.
   - Any failures or errors.
   - Link to a detailed report.
Tests will only run after explicit confirmation to ensure the script is ready for execution.

Q: How to set up recursive updates of steps by AI?

Please be aware of token limits. Send only the relevant story (add context).

Due to token limitations, break down large test suites:

  • Update scenarios in small batches (3-5 at a time).

  • Use intermediate commits.

  • Request focused updates.

Example 5. Prompt to add recursive test scenarios update:
Update test scenarios to include enhanced logging and error handling, focusing on the login flow

Q: How to combine several cases written by AI in different files?

Combining Strategies:

  • Extract common steps to separate scenario.

  • Create composite steps.

  • Use story includes.

Example 6. Prompt for file combination:
Combine these test scenarios into a single story file, extract common steps into separate scenario, and create composite steps for repeated actions.

Q: How to refactor existing code with AI help?

Please be aware of token limits. If you need a full refactor, split the work into multiple passes.

Refactoring Approaches:

  • Share existing code with AI (add context).

  • Request specific improvements.

  • Apply design patterns.

Example 7. BDD
Refactor these test scenarios to follow BDD approach and extract common steps into composite steps.
Example 8. DDD
Refactor these test scenarios to follow DDD approach and add examples tables to add variety of test data.

Q: What to do when AI starts hallucinating after previously working well?

Symptoms:

  • Generated steps don’t match Vividus step list.

  • Incorrect locator formats (not using id/xpath as specified).

  • Invalid syntax or made-up parameters.

  • Attempts to execute non-existent commands.

  • Creates invalid file structures.

Immediate Actions:

  1. Stop current generation.

  2. Clear the conversation.

  3. Start fresh with URL and requirements (use test-template.prompt.md or provide details in chat).