Understanding the Transformation Flow Builder

1.

Navigation menu

The buttons here enable you to save, publish and delete a transformation flow.

• Flow: Using this button, you can save (through the Save or Save As options) the flow as a draft, or even delete it. Draft flows can be executed from the Browse page.
• Publish: This lets you save and publish your flow. Scheduled flows have to be published to run.

2.

Steps

There are three types of steps that can be included in a transformation flow. Hover over their individual buttons or click on them to view a list of the selected step.

• Input steps
• Transformation steps
• Output steps

These are connected to one another by connections. 

3.

Middle header menu

• Settings: Configure settings related to the transformation builder. Includes the following options:

  • • Preview Row Limit: To specify the maximum number of rows that are to be processed for each step while in the Design Mode. For performance reasons, this limit should not exceed 1,000,000.

    • Rows Displayed: To specify the maximum number of rows to be displayed in the data preview panel. This limit should not exceed 200.

    • Error Threshold: To specify the error limit. If this value is reached, then the step will be considered a failure. To disable this setting, simply enter -1 here.

• Schedule: Configure a batch schedule for the transformation flow.
• Transformation Summary: Provides details of the transformation flow.
• Run: Use the run button to do a preview execution of the data transformation flow. Note, that this is not a full execution, but only executes a limited amount of rows, as specified in the Preview Row Limit setting. This is ideal for testing the flow. You can change the amount of rows executed with a maximum limit of 1,000,000.

4.

Transformation flow panel

This panel provides details of the overall transformation flow, or a selected step.

Transformation flow: If no step is selected, details of the entire transformation flow are seen here. This includes the flow's run logs, and activity stream.

Step configuration: If a step is selected, this panel is used to configure its settings. The tabs in this panel will vary according to the selected step.

5.

Data preview panel

This shows the extracted and/or transformed data. Every time a change is made, it can be instantly viewed here.

6.

Transformation builder canvas

This canvas is used to create the transformation flow. Drag steps here to include them in the flow, and create connections between them.

7.

Canvas positioner

The map view can be used to move the canvas around - this is useful when there are large numbers of steps on the canvas. You can also use this to re-centre the canvas.