Parquet Plugin
The plugin provides the ability to create Parquet files.
Installation
-
Copy the below line to
dependencies
section of the projectbuild.gradle
filePlease make sure to use the same version for all VIVIDUS dependencies. Example 1. build.gradleimplementation(group: 'org.vividus', name: 'vividus-plugin-parquet', version: '0.6.1')
-
If the project was imported to the IDE before adding new dependency, re-generate the configuration files for the used IDE and then refresh the project in the used IDE.
Properties
Property Name | Acceptable values | Default | Description |
---|---|---|---|
|
Specifies the format of a CSV file to create a Parquet file from. |
Expressions
The expression parameters marked with bold are mandatory. |
convertCsvToParquetFile
Creates temporary Parquet file, writes CSV data records to the file using Avro schema and returns its path.
#{convertCsvToParquetFile($csv-resource-path, $avro-schema-resource-path)}
-
$csv-resource-path
- The path to a resource containing CSV data. -
$avro-schema-resource-path
- The path to a resource containing Avro schema.
Example 2. Initialize a variable with a path to a Parquet file
Given I initialize scenario variable `parquet-path` with value `#{convertCsvToParquetFile(data/pets.csv, data/pets-schema.avsc)}`