Lucene: generateWordParts vs splitOnCaseChange

Question

I am investigating WordDelimiterFilterFactory

I am confusing about generateWordParts and splitOnCaseChange parameters.

from java doc:
generateWordParts:

/**
   * Causes parts of words to be generated:
   * <p>
   * "PowerShot" =&gt; "Power" "Shot"
   */
  public static final int GENERATE_WORD_PARTS = 1;

splitOnCaseChange:

 /**
   * If not set, causes case changes to be ignored (subwords will only be generated
   * given SUBWORD_DELIM tokens)
   */
  public static final int SPLIT_ON_CASE_CHANGE = 64;

Can you show example which clarify the difference?

P.S.

Also I don't understanf menaning of SUBWORD_DELIM

Answer 1

First: The WordDelimiterFilter has been superseded by the Word Delimiter Graph Filter (which plays nice with phrase queries).

generateWordParts considers more than just case differences. I.e. foo-bar is split into foo and bar. Split on case change wouldn't do anything here, leaving a single foo-bar token.

The SUBWORD_DELIM references the types parameter, where you can include a file that defines which characters that can be assumed to split a token into subwords:

types
(optional) The pathname of a file that contains character => type mappings, which enable customization of this filter’s splitting behavior. Recognized character types: LOWER, UPPER, ALPHA, DIGIT, ALPHANUM, and SUBWORD_DELIM.

I assume you could use a SUBWORD_DELIM character as '|' or '.', where you'd split a word into two tokens if it contained either of those characters.