Data Extraction Methods

 

 

Overview

The Data Transformation module supports different ways of extracting data from sources and loading it into Yellowfin, using the input steps. This section will show you how to use and configure each of these steps.

 

 

 

Configuring Input Steps

Delimited File

 

1. Drag the Delimited File option from the input step list. 
   
   
   
   

2. Configure the step to extract data from the file. First select the file protocol.

3.  If the file to be imported is available locally, choose the Local option and specify its path.

   If you have selected to import a local file, the file should be local on your server.

   
   
   

4. But to use a file that's available on a network, select the correct protocol (Network Share, FTP, or SFTP), then provide additional configurations to connect to the network.
   
   
   
   
   
   

5. You can also provide additional configuration settings by enabling the Advanced Settings toggle.
   
   
   
   

   • First row contains unique titles: Enable this toggle switch if the first row of the file contains titles or column names. If switched on, the system will read these values as the column field titles.
   • Field separator: Choose a character from the list to use it to separate values in the CSV, or enter a custom one after selecting the Other option.
   • Decimal separator: Choose a character from the list to set it as the decimal separator, or enter a custom one after selecting the Other option.
   • File encoding: Select a file encoding option.
   • Precision: Use this to specify how many rows of the CVS file Yellowfin will examine in order to define the field types and size.
     • Low Precision is fast, and only examines the first 1000 rows of the file.
     • High Precision is slower, depending on the size of the file, as it examines all rows.

6. Finally, click Apply. Data extracted from the file will appear in the data preview panel.
   
   
   
   
   

---

 

Freehand SQL

 

1. Drag the Freehand SQL option.
   
   
   
   
2. Select the data source that you want to use.
   
   
   
   
3. Write a SQL query to extract the required data.
   
   
   
   
4. You can click on the Validate option to check the validity of the SQL you provided. This ensures there are no errors in the query.
5. Click on the Submit button when you are sure your query is correct.
6. Then click on Apply in the Configure panel. The extracted data will appear in the data preview panel.
   
   
   
   
7. To change the freehand SQL query, click the Edit SQL option in the Configure panel.

 

 

 

---

 

Report

1. Drag the Report input step onto the transformation canvas.
2. Search for the report that you want to extract data from, by entering and then clicking on its name.
   
   
   
   
3. Once a report is selected, its column field names will be displayed.
   
   
   
   
4. Click on the Add Report button.
5. The report fields can be customized in the Configure panel.
   
   
   
   
6. You can remove fields that you don’t want included.
7. If the report has any embedded user-prompt filters, they will also appear here. You can apply these filters when extracting data.
8. Then click Apply. You can see the extracted data from the report in the data preview panel. 
   
   
   

 

 

---

 

Single Database Table

1. Drag the Single Table option onto the canvas.
   
   
   
   

2. On the appearance of the popup, select the database you want to use.
   
   
   
   

3. Then choose a table. And click on the Submit button. 
   
   

   
   
   

4. The details of this step can be configured through the configure panel. 
   
   
   
   

5. Select only the fields that you want data to be extracted from. (You can unselect any unwanted fields.)

6. Click on the Add Filter button to apply filters to control the data being extracted through the Filter popup. 

7. Once you’re done with the configuration, click on the Apply button. 

8. On doing so, the data preview panel will display the data extracted from the configured data source.
   
   

   

 

 

---

 

 

Third-party Connectors

1. Drag the Third-party Table Connector option from the input step onto the canvas.
   
   
   
   
2. A list of all third-party data sources will appear. (You can also click on the Not here? link at the bottom to create a new third-party source.)
   
   
   
   
3. Choose the data source that you want to use.
4. Then select a table from the chosen data source.
5. Finally, click Submit.
   
   
   
   
6. Next choose the fields that are to be extracted.
7. If the connector has any mandatory filters, provide details for them. For example, when extracting data from a Google Analytics table, you will need to provide the start and end dates to filter your data.
8. Click on the Apply button. The extracted data will appear in the data preview panel.