Configuring the Data Iterator node

  1. Select the Data Iterator start node. On the General tab, specify this information:
    Configuration name

    Select the configuration to use for this activity. To use an existing configuration, select a name from the list. If you leave the field blank, the default configuration is used.

    Input method

    Select File if the input data is read from a file, Data if it is from a variable, or Data Lake if it is from Data Lake tables using Compass queries.

    Input file/Input data

    If the input method is File, this field specifies the file name to read.

    If the input method is Data, this field must contain a reference to a ProcessFlow variable, for example, <!myVariable>).

    Parse by

    Specify the method by which the data stream is parsed. These are the available options:

    • Line to read the data by line.

      Lines are terminated by a line feed ('\n'), a carriage return ('\r'), or a carriage return followed immediately by a linefeed.

    • Delimiter String to split the data by a specified delimiter string, for example a comma.

    • Length to read a specified number of characters at a time.

    Starting position to read

    A position in the data stream where the read begins. The default is zero, which means to start at the beginning of the file or variable.

    Maximum read iterations

    The maximum number of read iterations to perform on the file. Leave this field blank or specify 0 for unlimited iterations.

    Bytes to read per iteration

    For the Length parse method only. This field defines the number of characters to read at a time. The activity reads the next n characters or to the end of the input stream.

    Delimiter string

    For the Delimiter String parse method only. This field defines the delimiter string used to split the input stream. The activity reads up to the next instance of the specified delimiter string or to the end of the input stream.

    Ignore trailing delimiter

    For the Line and Delimiter String parse methods. This field specifies whether to ignore any trailing delimiter at the end of the stream. If the check box is selected (True) and there is a delimiter immediately before the end of the input stream, the delimiter is ignored. If the check box is cleared (False) and there is a delimiter immediately before the end of the input stream, the delimiter is interpreted as indicating an empty record at the end.

    Accumulate output variables

    Specifies whether records are output into separate variables as they are parsed. If the check box is selected (True), each record is saved in the activity variable activityName_outputDataN, where activityName is the name of the activity and N is the record number.

    Compass query
    Specify the compass query to run.
    Iteration option
    Select how to process the output data retrieved from the Data Lake. Select Write to file to export all data directly to a file. Select By row to process each row individually and convert each column into a variable.
    Filename
    Specify the file name to use.
    Max number of rows
    Specify the maximum number of rows to write to the file.
    Max wait time (Minutes)
    Specify the maximum wait time in minutes before the system times out during data retrieval. The default value is 5.
    Field Delimiter
    Specify the character that separates fields in the output file. The default value is |.
    Note: If you specify a delimiter character enclosed in double quotes, the output data is also enclosed in double quotes. For example, “data1”,”data2”.
    Line Delimiter
    Specify the sequence that separates rows in the output file. The default value is \n.
    Include header
    Select this check box to include the column header in the output file.
    Keep work unit active
    Select this check box to prevent the thread from being dropped.
  2. On the On Error tab, specify this information:
    What should happen on an error?

    Select from the available options.

    Notify

    Select this check box to send a notification when an error occurs. If this option is selected, you must fill out the notification details.

    Custom log entry

    Select this check box to have a custom log entry in addition to the system-generated log entry.

  3. Select the Data Iterator end node. On the Break Condition tab, specify any valid true or false conditions that you must satisfy to end the iteration.
    Note: 

    If a variable is used as part of the condition, you must remove the surrounding brackets <! and >.