VERIFIED SOLUTION i
X

How to change settings to generate the correct flat file in Spectrum/Portrait Miner

UPDATED: August 7, 2017


By default, Portrait Miner uses the first 100 rows of data to determine (instantiate) the file format (number of fields, type of separator, NULL character etc) and the first 1000 rows to determine field types.  These defaults can be modified via the Edit-Preferences menu in Portrait Miner:

Flat File Database - Preferences

Portrait Miner does not always generate the correct flat file description - for example when a flat file is sparsely populated, contains a lot of dates in different formats and if the data is sorted such that larger values appear later in the file.

Increasing the number of rows used to determine the flat file description will improve the accuracy, though will have a slight affect on performance.

Portrait Miner will continue to build a focus even when errors or warnings are generated, allowing up to 1,000,000 of each by default.  These default settings can also be reduced, so issues with the data description are flagged up sooner.

When using the Portrait Miner gui to build a new focus, clicking on the Show Format Information button in Step 1 of the New Focus wizard will display details of the format and field types chosen.

The file format can be overridden by clicking on the Set Format... button in Step 2 of the New Focus wizard and making alternate selections:

Set flat file format

Finally individual field types can be modified in Step 3 of the New Focus wizard, changing field types, specifying lengths of string and date formats as required:

Change field types in Step 3 of New Focus wizard


When working at the command line, the qsgenfdd Data Build Command can be used to generate a new fdd file.  The help text for this tool is below:

C:\>qsgenfdd -h

Create a flat data description (FDD) file for a text file (overwriting
the FDD file if it already exists).

Required:
   -input <text file>
      Generate an FDD file for this text file.

Options:
   -comment <string>
      Include this comment in the generated FDD file.
   -dateformat <date format>
      Describe date fields as using this date format.
   -datemarker <string>
      Describe date fields as using this quoting character.
   -defaultday <day number>
      Describe date fields as having this day number (for date formats
      that do not include a day component).
   -defaultmonth <month number>
      Describe date fields as having this month number (for date formats
      that do not include a month component).
   -headers
      Describe the text file as having an initial header line.
   -null <string>
      Describe the null value as being represented by this string.
   -output <FDD file>
      Write the flat data description to this file instead of to <input>.fdd,
      where <input> is the basename of the input file.
   -separator <character>
      Describe the text file as having fields separated by this character.
   -stringmarker <character>
      Describe string fields as using this quoting character.
   -template <FDD file>
      Base the description on the format specified by this FDD file. (Any
      other command-line options override or extend this format.)

Use -helpall to see a full list that includes standard options.

Environment Details

Product Feature: Focus building

Downloads

  • No Downloads