AWS CloudWatch Plugin

The plugin provides functionality to fetch the log events from Amazon CloudWatch Logs.

Installation

  1. Copy the below line to dependencies section of the project build.gradle file

    Please make sure to use the same version for all VIVIDUS dependencies.
    Example 1. build.gradle
    implementation('org.vividus:vividus-plugin-aws-cloudwatch')

  2. If the project was imported to the IDE before adding new dependency, re-generate the configuration files for the used IDE and then refresh the project in the used IDE. == Configuration

Authentication

The plugin attempts to find AWS credentials by using the default credential provider chain. The provider chain looks for credentials using the provided below options one by one starting from the top. If credentials are found at some point, the search stops and further options are not evaluated.

  1. The AWS credentials scoped to either current scenario or story (configured via the corresponding step).

  2. Environment variables: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (the optional variable for session token is AWS_SESSION_TOKEN).

  3. The properties: system.aws.accessKeyId and system.aws.secretKey (the optional property for session token is system.aws.sessionToken).

  4. Web Identity Token credentials from the environment or container.

  5. In the default credentials file (the location of this file varies by platform).

  6. Credentials delivered through the Amazon EC2 container service if the AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable is set and security manager has permission to access the variable.

  7. In the instance profile credentials, which exist within the instance metadata associated with the IAM role for the EC2 instance. This step is available only when running your application on an Amazon EC2 instance, but provides the greatest ease of use and best security when working with Amazon EC2 instances.

  8. If the plugin still hasn’t found credentials by this point, client creation fails with an exception.

See the official "Working with AWS Credentials" guide to get more details.

Region Selection

The plugin attempts to find AWS region by using the default region provider chain. The provider chain looks for a region using the provided below options one by one starting from the top. If region is found at some point, the search stops and further options are not evaluated.

  1. Environment variable: AWS_REGION.

  2. The property: system.aws.region.

  3. AWS shared configuration file (usually located at ~/.aws/config).

  4. Use the Amazon EC2 instance metadata service to determine the region of the currently running Amazon EC2 instance.

  5. If the plugin still hasn’t found a region by this point, client creation fails with an exception.

See the official "AWS Region Selection" guide to get more details.

Steps

Start listening for the log events

Starts listening for the Amazon CloudWatch log events by capturing the current moment in time. The captured moment is later used as the start of the time range by the Wait for the log events in a log stream step.

The step is useful when the moment the events of interest may start appearing is not known upfront, e.g. the events are triggered by some business actions performed after this step.

When I start listening for CloudWatch log events

Wait for the log events in a log stream

Polls the specified log stream of the specified Amazon CloudWatch log group until at least one log event matching the filter pattern is found since the listening was started or the timeout expires, and then saves all the found events to the variable. The step is failed if no matching event appears within the timeout.

The listening must be started beforehand by the Start listening for the log events step. The time range grows on each polling attempt from the moment the listening was started up to the current moment. This handles the Amazon CloudWatch ingestion delay: the events produced right before the step execution are picked up as soon as they become available.

When I wait `$duration` with `$pollingDuration` polling until at least one event matching `$pattern` pattern appears in CloudWatch log group `$logGroupName` and log stream `$logStreamName` and save them to $scopes variable `$variableName`

  • $duration - The maximum time to wait for the events, in ISO-8601 duration format.

  • $pollingDuration - The interval between the polling attempts, in ISO-8601 duration format.

  • $pattern - The filter pattern to match the events against, if it’s empty any event since the listening was started is matched.

  • $logGroupName - The name of the Amazon CloudWatch log group to fetch the events from.

  • $logStreamName - The name of the Amazon CloudWatch log stream to fetch the events from.

  • $scopes - The comma-separated set of the variables scopes.

  • $variableName - The variable name to store the events. The events are sorted from the newest to the oldest and are accessible via zero-based index and the event key, e.g. ${my-var[0].message} returns the message of the most recent found event. The available keys are:

    • eventId - the ID of the event

    • timestamp - the time the event occurred at

    • message - the data contained in the event

Example 2. Wait until the specific application instance logs any event after performing a business action
When I start listening for CloudWatch log events
!-- Perform the business action that is expected to produce the log event, e.g. trigger a request
When I wait `PT2M` with `PT10S` polling until at least one event matching `` pattern appears in CloudWatch log group `/aws/logs/my-application` and log stream `application.log` and save them to scenario variable `applicationEvents`
Then `${applicationEvents[0].message}` is not equal to ``
Example 3. Wait until the application logs the processed order
When I start listening for CloudWatch log events
!-- Perform the business action that places the order
When I wait `PT2M` with `PT10S` polling until at least one event matching `"Order ${orderId} is processed"` pattern appears in CloudWatch log group `/aws/lambda/vividus-function` and log stream `vividus-function-instance-1` and save them to scenario variable `orderEvents`
Then `${orderEvents[0].message}` matches `.*is processed.*`