Click on the Create button on the top-right corner.
Then select the Transformation Flow option.
You will be taken to the transformation builder.
Hover your cursor over the Input Steps button on the left side to make the panel appear. (Click here to learn more about each of these input steps.)
Then drag an input step from this panel onto the canvas. (For this procedure, we will use the single database table as an example.)
On doing so, a popup will appear, requiring information on the selected data source. (The steps within this popup will differ for each type of input step.)
Select the data source, and then a single table (since our selected input was a single table).
Then click on the Submit button.
Next, click on the newly created input step in the canvas. Its details will appear in the configure flow panel.
Note: Clicking on a blank space in the canvas will show the configurations of the entire flow, but clicking on a specific step shows only that step’s configurations.
If required, you can rename the input step. Click on the Details tab in the Configuration panel, and provide a new name, or even an updated description.
Click on the Configure tab and select only the fields that you want data to be extracted from.
Click on the Add Filter button to apply filters to control the data being extracted, when the Filter popup appears.
Once you’re done with the configuration, click on the Apply button.
Warning
You must click on the Apply button to save the configuration details. If you do not click on this button, and click anywhere else first, you will lose the config details.
On doing so, the data preview panel will display the data extracted from the configured data source. A maximum row setting of 200 applies.
You can apply transformations to fields directly from the data panel. Click here to learn more.
A new Fields tab will also appear in the Configuration panel. Use this to manage the data fields that are extracted.
Expand
title
Part 2: Transformation step connection and configuration
Next, expand the transformation step panel by hovering on its icon in the step builder, and then dragging a transformation step onto the canvas.
Image Added
A transformation step will appear on the canvas.
Image Added
Next, create a connection between the input step and the transformation step. Hover over an input step and connecting points will form.
Drag a point and connect it to the transformation step to form a relationship between them. (Click here to learn about rules involving connections, and how to break existing connections.)
Image Added
Now you can configure the transformation step, by clicking on its icon and using the configuration panel. (Each transformation step type will be configured differently.Learn more about how to do that here.))
Save changes using the Apply button.
Image Added
Note: When a transformation is made to the data, the Transformation tab will appear in the configuration panel.
Expand
title
Part 3: Output step configuration and connection
Warning
Warning: Extra caution should be taken when configuring the output database step so that any existing source data does not get accidentally overwritten. The output to database step has a variety of options which can be destructive to data. For example, it can drop tables, truncate (delete) data, add news rows and update existing data. It is recommended that a test database with test data is used when developing transformation flows. Further, the output data step is disabled by default when in edit mode (Run in Preview mode option). If you are confident that the database you are using is a test database, or that the underlying data is safe to delete, then and only then should you change this setting. If you have any concerns about this, please first contact your system administrator.
Extend the output step panel by hovering on its icon, and drag an output step onto the canvas.
Next, connect the transformation step to the output step by repeating steps 20 and 21.
Then configure the output step through the panel on the right-side.
Click here to learn more about configuring this step.
You can also view the flow’s success status in the run logs tab in the configuration panel.
Click on the run button in the top header menu to execute the flow. The step will only be run for the data that is available in the preview (ie. limited to 200 rows). By default, any output to database step will not be run. This behaviour can be overridden by enabling the Run in Preview mode option in the Output to Database step. This should only be done subject to the warnings listed earlier on this page.
