Page History
| Table of Contents | |
|---|---|
|
|
Overview
This step type of transformation duplicates an input dataset to create identical output datasets. Under normal circumstances, a single step can only be connected to one other step. But using the split transformation step, you can connect one step to multiple steps. This step does not need to be configured, and you can split a step multiple times using this.
This is done by using the Split step in the Data Transformation module. This step does not need to be configured, and can be used multiple times to split a data set.
Note: This is a built-in step, and therefore will be available in the Transformations List by default.
Step Configuration
Follow the instructions below to configure a split step:
- Expand the Transformation Steps button on the left side of the Transformation Flow builder, to view a list of transformation steps.
- Drag the Split step from the list of transformation steps.
- Connect this step to the previous step in the flow.
- Add more steps to the flow, and one by one connect them to the split step.
For example, you can split data from a single data source and store it in multiple databases.
...
Complete Example
...
In this example, we will cover how to create a simple transformation flow that involves a split transformation step. Therefore, this transformation Our flow will involve extracting data from a data source, splitting it into two parts, with each part getting a different type of transformation applied to it. Each of the results are then saved into separate databasesoutputs. You could always include additional steps in your own transformation flowflows.
Click on the Create button in the top-right corner.
Then select Transformation Flow.
Note If you do not see this option, you may not have security access to transformation flows. Learn how to get access here.
You will be taken to the transformation flow builder.
Hover your cursor over the input steps button on the left side. A panel with a list of all data extraction steps will appear.
Drag one of these steps onto the canvas. For this procedure, we will first use the single table step as an example. (Click here to learn about all the different input steps.)
On doing so, a popup will appear to load data from a data source.
Click on the data source that you require.
Then choose the database table, and click on Submit.
The selected table's fields will appear in the transformation flow panel to be configured.
Select only the fields that you want data to be extracted from.
You can make further changes to the step, such as renaming it, adding a description, etc.
Once you’re done with the step configuration, click on the Apply button.
Warning On doing so, the data preview panel will display the data extracted from the configured database table.
Now follow the below steps to add another input step to import and configure another data source, for example data from a report.
- Drag the report step from the input step panel. Then select a report by providing its name in the popup window.
Once you are ready to split your data, extend the transformation steps panel, and drag in the Split step.
- Then create a connection between the database input step (or the previous step) and this split step.
There is no need to configure the split step, however you can still choose which fields will be carried onto the next steps through the Field tab.
Now you can add multiple steps simultaneously and perform different transformations using the same dataset.
For this example, we will aggregate this dataset and separately create a custom calculated field on its copy.
- Drag in the Aggregate step from the transformation steps panel, and connect the split step to it.
Click on the aggregate step, and in the transformation flow panel, select the aggregations to be applied on each of the fields. Then click Apply.
- The result of this step will appear in the data preview panel.
- Next, include the calculated field transformation step to the flow, and connect the split step to it.
Click on this newly added step and configure it through the transformation flow panel. First click Add Item, and then in the popup window, create a custom calculated field that you want your data to generate.
- You can confirm the validation of your formula by using the Validate button.
- Then click on the Save button.
The result of this step will be generated in the data preview panel.
Now we will save the data from each of these steps into separate output steps
Click on Apply when configuring the report fields in the transformation flow panel.The data preview panel will then display the configured report's data.
You can also apply transformations to fields directly in the data preview panel. Click here to learn more about this.
In order to split a step in other steps
Once you are ready to save your data into a writeable database, follow the steps below.
Extend the output steps panel by hovering on its icon, and drag the SQL database output step onto the canvas.
Connect the merge step (or the previous step in your flow) to the aggregate step to this output step by creating a connection.
Note: By default, the output step will be highlighted as red to signify that it contains errors. This is because it has not been configured yet.
And then configure the output step through the panel on the right-side. Click here to learn more about configuring this step.
- Similarly, add another SQL database output step, and create a connection with the calculated field step.
Configure this output step as well.
You can now execute the draft flow by clicking on the run button in the top header menu. (This does a quick execution of the data rows in the data preview panel.)
Or save the flow for a full execution. To do that, click on the Publish button.
Then provide details in the popup that appears, such as providing a proper name, and selecting rights to access the flow.
Finally, click on the Save button.
When the report's fields appear in the next popup, click Add Report.
...