Overview
This step duplicates an input dataset to create identical output datasets. Under normal circumstances, a single step can only be connected to one other step. But using the split transformation step, you can connect one step to multiple steps. This step does not need to be configured, and you can split a step multiple times using this.
Splitting Data in a Transformation Flow
In this example, we will cover how to create a simple transformation flow that involves a split transformation step. Our flow example will involve extracting data from a data source, splitting it into two parts, with each part getting a different type of transformation applied to it. Each of the results are then saved into separate databases. You could always include additional steps in your own transformation flow.
Click on the Create button in the top-right corner.
Then select Transformation Flow.
If you do not see this option, you may not have security access to transformation flows. Learn how to get access here.
You will be taken to the transformation flow builder.
Hover your cursor over the input steps button on the left side. A panel with a list of all data extraction steps will appear.
Drag one of these steps onto the canvas. For this procedure, we will first use the single table step as an example. (Click here to learn about all the different input steps.)
On doing so, a popup will appear to load data from a data source.
Click on the data source that you require.
Then choose the database table, and click on Submit.
The selected table's fields will appear in the transformation flow panel to be configured.
Select only the fields that you want data to be extracted from.
You can make further changes to the step, such as renaming it, adding a description, etc.
Once you’re done with the step configuration, click on the Apply button.
You must click on the Apply button to save the configuration details. Otherwise, if you click somewhere else first, you will lose the configuration details.
On doing so, the data preview panel will display the data extracted from the configured database table.
Now follow the below steps to add another input step to import and configure another data source, for example data from a report.
- Drag the report step from the input step panel.
- Then select a report by providing its name in the popup window.
When the report's fields appear in the next popup, click Add Report.
Click on Apply when configuring the report fields in the transformation flow panel.
The data preview panel will then display the configured report's data.
You can also apply transformations to fields directly in the data preview panel. Click here to learn more about this.
In order to split a step in other steps
Once you are ready to save your data into a writeable database, follow the steps below.
Extend the output steps panel by hovering on its icon, and drag the SQL database output step onto the canvas.
Connect the merge step (or the previous step in your flow) to the output step by creating a connection.
Note: By default, the output step will be highlighted as red to signify that it contains errors. This is because it has not been configured yet.
And then configure the output step through the panel on the right-side. Click here to learn more about configuring this step.
You can now execute the draft flow by clicking on the run button in the top header menu. (This does a quick execution of the data rows in the data preview panel.)
Or save the flow for a full execution. To do that, click on the Publish button.
Then provide details in the popup that appears, such as providing a proper name, and selecting rights to access the flow.
Finally, click on the Save button.