PhoenixmlDb.XQuery

StandardFullTextAnalyzer

The built-in analyzer, with no dependency. In order: word boundaries by Unicode Standard Annex #29 (Unicode 6.3 data); when stemming English, the possessive "'s" removed; lowercase unless using case sensitive; stop words removed, leaving a gap in the positions; when stemming English, the Porter stemmer. A word longer than 255 UTF-16 characters is not a token but takes a position.

#StandardFullTextAnalyzer

Namespace: PhoenixmlDb.XQuery.FullText

The built-in analyzer, with no dependency. In order: word boundaries by Unicode Standard Annex #29 (Unicode 6.3 data); when stemming English, the possessive "'s" removed; lowercase unless

using case sensitive

; stop words removed, leaving a gap in the positions; when stemming English, the Porter stemmer. A word longer than 255 UTF-16 characters is not a token but takes a position.

It produces the same tokens, positions and offsets as the Lucene.Net 4.8 analysis chain that XQuery 3.1 and earlier used (StandardTokenizer, EnglishPossessiveFilter, LowerCaseFilter, StopFilter, PorterStemFilter), so an index built with the earlier release stays valid. Stemming applies only when the language is absent or English ("en", "english", "en-…"); the default stop words are the English list for every language.

#Properties

Name Description
Capabilities
DefaultStopWords The default stop words: a short English list.
Instance The shared instance; the analyzer has no state.
Name

#Methods

#Analyze(String,PhoenixmlDb.XQuery.FullText.FtAnalysisRequest)

#IsDefaultStopWord(String)

Whether

, lowercased, is a default stop word.

#LowerCase(String)

Each code point lowercased by the invariant culture; the UTF-16 length can change only for a supplementary character.

#WithoutDiacritics(String)

The term without combining marks: decomposed, marks removed, recomposed.

#WithoutPossessive(String)

A trailing "'s" (apostrophe, right single quotation mark or fullwidth apostrophe; s either case) removed.

#Fields

Name Description
MaxTokenLength The longest word, in UTF-16 characters, that is a token.