Table Transformers

VIVIDUS provides a set of pre-registered table transformers. Also plugins may register own table transformers (they are documented in the corresponding plugins articles).

FROM_LANDSCAPE

FROM_LANDSCAPE transformer is designed to transform the table from a landscape form.

Example 1. Usage of FROM_LANDSCAPE transformer
{transformer=FROM_LANDSCAPE}
|header 1|value 11| ... | value m1|
...
|header n|value 1n| .... |value mn|

RESOLVING_EXPRESSIONS_EAGERLY

RESOLVING_EXPRESSIONS_EAGERLY transformer processes expressions which are used inside ExamplesTable eagerly, i.e. at the table loading stage. If expressions generating random data are used in the table with RESOLVING_EXPRESSIONS_EAGERLY transfomer, the random data will be generated once and each placeholder will be replaced with this value. While in regular flow placeholders are replaced with the expression and data is generated at step execution stage, in other words the same placeholders will be replaced by different randomly generated datasets.

Example 2. Usage of RESOLVING_EXPRESSIONS_EAGERLY transformer
{transformer=RESOLVING_EXPRESSIONS_EAGERLY}
|firstName                     |lastName                     |
|#{generate(Address.firstName)}|#{generate(Address.lastName)}|

RESOLVING_SELF_REFERENCES_EAGERLY

RESOLVING_SELF_REFERENCES_EAGERLY transformer enables eagerly resolution of references to one column from another column within single ExamplesTable ("eagerly" means at the table loading stage). Circular references will result in error.

Example 3. Usage of RESOLVING_SELF_REFERENCES_EAGERLY transformer
{transformer=RESOLVING_SELF_REFERENCES_EAGERLY}
|column1|column2        |column3             |
|A      |foo<column3>bar|before<column1>after|

FORMATTING

FORMATTING transformer formats the table according to the max width of each column. Also this transformer removes the commented lines.

Example 4. Usage of FORMATTING transformer
{transformer=FORMATTING}
|key|
|value|
|-- |value|
|longValue|

REPLACING

REPLACING transformer replaces all substrings inside table values with replacement according to the given parameters.

Parameter Description

replacing

the sequence of chars to be replaced in the table cells

replacement

the replacing sequence of chars
Example 5. Usage of REPLACING transformer
{transformer=REPLACING, replacing=k, replacement=l}
|key  |
|vakue|

SORTING

SORTING transformer is designed to sort the table rows by the values in the columns.

Parameter Description

byColumns

column names ordered according to the priority of applying sorting (the column names are joined using |)
Example 6. Usage of SORTING transformer
{transformer=SORTING, byColumns=key|key1}
|key   |key1  |
|value3|value4|
|value1|value2|

ITERATING

ITERATING transformer generates simple one-column table with header iterator and numbers as values: 0, 1, 2…​ .

Parameter Description

limit
The property is deprecated and will be removed in VIVIDUS 0.6.0. Use startInclusive and endInclusive instead

threshold value, so values are generated from 0 to (limit - 1).

startInclusive

value specifying which number to start generating values from, it should be less than or equal to endInclusive value

endInclusive

threshold value specifying at which number to stop generating values, it should be greater than or equal to startInclusive value
Example 7. Usage of ITERATING transformer
{transformer=ITERATING, startInclusive=0, endInclusive=4}

REPEATING

REPEATING transformer generates ExamplesTable by copy-pasting the initial table the desired number of times.

Parameter Description

times

the number of times to copy-paste the initial table

The following REPEATING transformer:

{transformer=REPEATING, times=3}
|column1|column2|
|A      |C      |
|B      |D      |

will result in the following ExamplesTable:

|column1|column2|
|A      |C      |
|B      |D      |
|A      |C      |
|B      |D      |
|A      |C      |
|B      |D      |

FILTERING

FILTERING transformer filters the table using the specified parameters.

Parameter Description

byMaxColumns

the maximum number of columns to keep

byMaxRows

the maximum number of rows to keep

byRandomRows

the number of random rows to keep

byColumnNames

the names of the columns to keep separated by semicolon

byRowIndexes

the zero-based indices of the rows to keep separated by semicolon
byMaxColumns and byColumnNames are competing properties and only one can be specified at the same time. The same restriction is applied to the properties byMaxRows, byRandomRows and byRowIndexes.
In case if byColumnNames contains nonexistent table columns the exception with list of the invalid columns will be thrown.
Example 8. Usage of FILTERING transformer with filtering by column names and by max number of rows
{transformer=FILTERING, byColumnNames=key1;key3, byMaxRows=1}
|key1  |key2  |key3  |
|value1|value2|value3|
|value4|value5|value6|
Example 9. Usage of FILTERING transformer with filtering by max number of columns and by row indexes
{transformer=FILTERING, byMaxColumns=2, byRowIndexes=0;2}
|key1  |key2  |key3  |
|value1|value2|value3|
|value4|value5|value6|
|value7|value8|value9|
Example 10. Usage of FILTERING transformer with filtering by column names and the number of random rows to keep
{transformer=FILTERING, byColumnNames=key2;key3, byRandomRows=2}
|key1  |key2  |key3  |
|value1|value2|value3|
|value4|value5|value6|
|value7|value8|value9|

DISTINCTING

DISTINCTING transformer generates ExamplesTable from the specified columns, deleting the rows with repeating values.

Parameter Description

byColumnNames

the names of the columns to keep separated by semicolon

The following DISTINCTING transformer:

{transformer=DISTINCTING, byColumnNames=column1;column2}
|column1|column2|column3|
|A      |C      |0      |
|B      |D      |1      |
|A      |C      |1      |
|A      |C      |0      |
|B      |D      |1      |

will result in the following ExamplesTable:

|column1|column2|
|A      |C      |
|B      |D      |

MERGING

MERGING transformer merges several tables into one.

Parameter Description

mergeMode

rows or columns

tables

paths to examples tables or transformers to merge

fillerValue

value to fill new cells while merging tables with different number of rows (in columns mode) or columns (in rows mode)

  • to merge tables with different number of rows or columns use fillerValue parameter to fill new cells

  • to merge tables in rows mode they must have the same numbers of columns + the same headers(keys)

  • to merge tables in columns mode they must have the same number of rows + unique headers(keys)

  • number of unique tables paths must be more than 1

  • when using transformers in tables parameter it is necessary to escape brackets, commas and semicolon in them: for unary nested- "", for double - "\" etc.

  • empty tables are allowed
Example 11. Usage of MERGING transformer with 'rows' merge mode
{transformer=MERGING, mergeMode=rows, tables=story/tables/test1.table;story/tables/test2.table}
Example 12. Usage of MERGING transformer with 'columns' merge mode
{transformer=MERGING, mergeMode=columns, tables=story/tables/test1.table;story/tables/test2.table;story/tables/test3.table}
Example 13. Usage of MERGING transformer with nested tables
{transformer=MERGING, mergeMode=columns, fillerValue=null, tables=\{transformer=FROM_EXCEL\, path=TestTemplate.xlsx\, sheet=Data\, addresses=A2\;A3\, column=test1\};\{transformer=FROM_EXCEL\, path=TestTemplate.xlsx\, sheet=Data\, range=B2:B4\, column=test2\}}
Example 14. Usage of nested MERGING transformers
{
 transformer=MERGING,
 mergeMode=rows,
 fillerValue=null,
 tables=
 \{
   transformer=MERGING\,
   mergeMode=columns\,
   tables=
        \\{
           transformer=FROM_EXCEL\\,
           path=TestTemplate.xlsx\\,
           sheet=Data\\,
           addresses=E6\\,
           column=text1
        \\}
        \;
        \\{
           transformer=FROM_EXCEL\\,
           path=TestTemplate.xlsx\\,
           sheet=Data\\,
           addresses=E7\\,
           column=text2
        \\}
 \}
 ;
 \{
   transformer=FROM_EXCEL\,
   path=TestTemplate.xlsx\,
   sheet=Data\,
   addresses=E8\,
   column=text1
 \}
}
Example 15. Usage of MERGING transformer with table body
{transformer=MERGING, mergeMode=columns, tables=/data/some-table.table}
|column1|
|value1 |

JOINING

JOINING transformer joins values from rows/columns of the table.

Parameter Description

joinMode

rows or columns

joinedColumn

[columns mode only] name of the new column with joined values

columnsToJoin

[columns mode only] colon-separated list of the columns to join
JOINING transformer should be the last while joining columns/rows of the table which created using another transformer.
Example 16. Usage of JOINING transformer with columns join mode
Scenario: Verify JOINING transformer in default columns mode
Then `<joinedColumn>` is equal to `A B`
Examples:
{transformer=JOINING, joinMode=columns, joinedColumn=joinedColumn}
|column1|column2|
|A      |B      |
Example 17. Usage of JOINING transformer with columns join mode and definition of columns to join
Scenario: Verify JOINING transformer in configured columns mode
Then `<joinedColumn>` is equal to `B D`
Examples:
{transformer=JOINING, joinMode=columns, joinedColumn=joinedColumn, columnsToJoin=column2;column4}
|column1|column2|column3|column4|
|A      |B      |C      |D      |
Example 18. Usage of JOINING transformer with rows join mode
Scenario: Verify JOINING transformer in default rows mode
Then `<column1>` is equal to `A B`
Then `<column2>` is equal to `C D`
Examples:
{transformer=JOINING, joinMode=rows}
|column1|column2|
|A      |C      |
|B      |D      |
Example 19. Usage of JOINING transformer in the chain of transformers
{transformer=MERGING, mergeMode=columns, tables=
  \{transformer=FROM_EXCEL\, path=/TestTemplate.xlsx\, sheet=Mapping\, range=A4:A5\, column=header1\, joinValues=true\};
  \{transformer=FROM_EXCEL\, path=/TestTemplate.xlsx\, sheet=Mapping\, range=B4:B5\, column=header2\, joinValues=true\}
}
{transformer=JOINING, joinMode=columns, joinedColumn=header}
Example 20. Usage of JOINING transformer with table body containing path to table file
{transformer=JOINING, joinMode=columns, joinedColumn=header}
/test1.table

CARTESIAN_PRODUCT

CARTESIAN_PRODUCT transformer creates a cartesian product from several tables.

Parameter Description

tables

paths to examples tables or transformers to create a cartesian product

  • headers of input tables must be different

  • number of input tables must be more than 1

  • if any of tables is empty - the resulting table will also be empty
Example 21. Usage of CARTESIAN_PRODUCT transformer
{transformer=CARTESIAN_PRODUCT, tables=story/tables/test1.table;story/tables/test2.table}
Example 22. Usage of CARTESIAN_PRODUCT transformer with table body
{transformer=CARTESIAN_PRODUCT, tables=/data/some-table.table}
|column1|
|value1 |
Example 23. Usage of CARTESIAN_PRODUCT transformer with nested transformers
{transformer=CARTESIAN_PRODUCT, tables=\{transformer=REPEATING\, times=100\};\{transformer=FROM_CSV\, csvPath=/data/csv.csv\}}

INDEXING

INDEXING transformer adds a column with the rows indices in the specified order. Indices are zero-based

Parameter Description

order

The mandatory indedxing order: ASCENDING or DESCENDING.
Example 24. ASCENDING order example
{transformer=INDEXING, order=ASCENDING}
|key   |
|value |
|value2|
Example 25. ASCENDING order resulting table
|key   |index|
|value |0    |
|value2|1    |
Example 26. DESCENDING order example
{transformer=INDEXING, order=DESCENDING}
|key   |
|value |
|value2|
Example 27. DESCENDING order resulting table
|key   |index|
|value |1    |
|value2|0    |

INNER_JOIN

INNER_JOIN transformer combines rows from two tables whenever there are matching values between the columns.

Parameter Description

leftTableJoinColumn

the column name for matching in the left table

rightTableJoinColumn

the column name for matching in the right table

tables

ExamplesTable-s to join

  • left table is the first ExamplesTable declared in tables parameter,

  • right table is the second ExamplesTable declared in tables parameter or table body put under the transformer definition.

  • The number of used ExamplesTable-s must be equal to 2 (left and right).

  • The column names of input tables must be different (except the column names for matching).

  • If any of tables is empty - the resulting table will also be empty.
Example 28. Usage of INNER_JOIN transformer with table body
{transformer=INNER_JOIN, leftTableJoinColumn=joinID, rightTableJoinColumn=joinID, tables=/tables/some-table.table}
|joinID|column2|
|5     |row25  |
|3     |row23  |
|1     |row21  |

where ExamplesTable from /tables/some-table.table:

|joinID|column1|
|1     |row11  |
|2     |row12  |
|3     |row13  |
|4     |row14  |

will result in the following ExamplesTable:

|column1|joinID|column2|
|row11  |1     |row21  |
|row13  |3     |row23  |
Example 29. Usage of INNER_JOIN transformer with the same column names for matching
{transformer=INNER_JOIN, leftTableJoinColumn=joinID, rightTableJoinColumn=joinID, tables=/tables/test1.table;/tables/test2.table}
Example 30. Usage of INNER_JOIN transformer with the different column names for matching
{transformer=INNER_JOIN, leftTableJoinColumn=joinID1, rightTableJoinColumn=joinID2, tables=/tables/test1.table;/tables/test2.table}
Example 31. Usage of INNER_JOIN transformer with nested tables
{
 transformer=INNER_JOIN,
 leftTableJoinColumn=joinID,
 rightTableJoinColumn=joinID,
 tables=
 \{
   transformer=FROM_EXCEL\,
   path=TestTemplate.xlsx\,
   sheet=Data\, addresses=A2\;A3\,
   column=joinID
 \};
 \{
   transformer=FROM_EXCEL\,
   path=TestTemplate.xlsx\,
   sheet=Data\, range=B2:B4\,
   column=joinID
 \}
}