Changing the text analyzer

The text analyzer determines how data is indexed and searched.

You can choose from these options: standard or whitespace analyzer. The standard analyzer is the default analyzer in Function Search. You might prefer a switch to whitespace analyzer depending on the type of M3BE specific data and preferred searches.
  • Standard analyzer
    • The standard analyzer provides grammer-based tokenization which is based on the Unicode Text Segmentation algorithm. The algorithm works well for most languages and is described in the Unicode Standards Annex #29. See https://unicode.org/reports/tr29/.

      For example: The 2 QUICK Brown-Foxes jumped over the lazy dog's bone.

      The statement produces these terms or indexed data: [ the, 2, quick, brown, foxes, jumped, over, the, lazy, dog's, bone ] wherein Brown-Foxes is divided and indexed into two terms.

  • Whitespace analyzer
    • The whitespace analyzer breaks text into terms whenever the analyzer encounters a whitespace character.

      For example: The 2 QUICK Brown-Foxes jumped over the lazy dog's bone.

      The statement produces these terms or indexed data: [ The, 2, QUICK, Brown-Foxes, jumped, over, the, lazy, dog's, bone. ] wherein Brown-Foxes is indexed as one term.

      Another example is 3/4x14" Tube. You must use the whitespace analyzer to get the correct result when searching for the slash sign and quotation marks as the standard analyzer algorithm divides terms and removes special characters.

Use the procedure to change the text analyzer:

  1. In the Function Search management pages, click Settings.
  2. Click Update.
  3. Select Standard or Whitespace in the Text Analyzer drop-down.
  4. Click Update.