Step Helper Methods

public Set<String> getInputFlowUuids()

public Set<String> getOutputFlowUuids()

These methods fetch all input/output flow UUIDs. Since most steps don’t have multiple inputs or outputs, getFirstInputFlow()and getFirstOutputFlow()may be more useful.

public ETLStep getInputStep(String inFlowUuid)

public ETLStep getOutputStep(String outFlowUuid)

public List<ETLStepMetadataFieldBean> getMetadataFields(String outFlowUuid)

public ETLStepMetadataFieldBean addNewGeneratedField(ETLStepMetadataFieldBean field, String optionKey)

This method is used for removing existing default metadata fields. However, it's not very safe to use this to remove default fields from an input step, as this may irreversibly corrupt the step.

protected void excludeDefaultField(String fieldUUID)

A default metadata field may be excluded from the flow using this method. 

A default metadata field may be included to the flow using this method. It restores the field at the specified position (starting from 0).

public Map<String, String> getDefaultToInputFieldMap()

public Map<String, String> getInputToDefaultFieldMap()

This method returns a step option which is stored as a String in the Yellowfin configuration database. The API provides a few convenience methods which convert the option to a specific data type. For example, getStepOptionValueJSONObject() converts an option to a com.google.gson.JsonObject and getStepIntegerValue() converts it to an Integer.

public byte[] getFile(String optionKey)

public String getFileName(String optionKey)

public String getText(String optionKey)

Step Options which reference files and CLOBs may be retrieved using these methods. These objects are fetched from tables in Yellowfin's configuration database.

public void parseData(Map<String, Object> data)

This method is used to validate and/or convert inputs from the front-end. The step’s configuration UI is represented by a ParameterPanelCollection which is composed of Parameter objects. These are basic building blocks such as a text box or a drop-down. There may be more complicated Parameter classes such as field-match and file-input. The input data from a parameter may have to be parsed before storing it as a step option. The field-match parameter may, for example, return separate lists for matched fields and excluded fields. To save them as separate step options, parseData() must modify the corresponding entry in data and put it back. This is shown in the example below.

@Override
public void parseData(Map<String, Object> dataMap) {
    for(String key: dataMap.keySet()) {
        if ("MATCH_FIELDS".equals(key) && dataMap.get(key) != null) {
            // Fields are matched using "from" and "to" elements in the data string.
            // Convert [{from=uuid1, to=uuid2}, {from=uuid3, to=uuid4}] to
            // LEFT_FIELD0=uuid1, RIGHT_FIELD0=uuid2, 
            // LEFT_FIELD1=uuid3, RIGHT_FIELD1=uuid4
            List<?> allJoins = (List<?>) dataMap.get(key);
            int i = 0;
            for (Object joinObj : allJoins) {
                if (joinObj instanceof Map) {
                    @SuppressWarnings("unchecked")
                    Map<String, String> join = (Map<String, String>)joinObj;
                    
                    dataMap.put("LEFT_FIELD"+i, join.get("from"));
                    dataMap.put("RIGHT_FIELD"+i, join.get("to"));
                    i++;
                }
            }
        }
    }
    
    // This shouldn't be saved as a StepOption
    dataMap.remove("MATCH_FIELDS");
}