PhoenixmlDb.XQuery
StandardFullTextAnalyzer
The built-in analyzer, with no dependency. In order: word boundaries by Unicode Standard Annex #29 (Unicode 6.3 data); when stemming English, the possessive "'s" removed; lowercase unless using case sensitive; stop words removed, leaving a gap in the positions; when stemming English, the Porter stemmer. A word longer than 255 UTF-16 characters is not a token but takes a position.
#StandardFullTextAnalyzer
Namespace: PhoenixmlDb.XQuery.FullText
The built-in analyzer, with no dependency. In order: word boundaries by Unicode Standard Annex #29 (Unicode 6.3 data); when stemming English, the possessive "'s" removed; lowercase unless
using case sensitive
; stop words removed, leaving a gap in the positions; when stemming English, the Porter stemmer. A word longer than 255 UTF-16 characters is not a token but takes a position.
It produces the same tokens, positions and offsets as the Lucene.Net 4.8 analysis chain that XQuery 3.1 and earlier used (StandardTokenizer, EnglishPossessiveFilter, LowerCaseFilter, StopFilter, PorterStemFilter), so an index built with the earlier release stays valid. Stemming applies only when the language is absent or English ("en", "english", "en-…"); the default stop words are the English list for every language.
#Properties
| Name | Description |
|---|---|
Capabilities
|
|
DefaultStopWords
|
The default stop words: a short English list. |
Instance
|
The shared instance; the analyzer has no state. |
Name
|
#Methods
#Analyze(String,PhoenixmlDb.XQuery.FullText.FtAnalysisRequest)
#IsDefaultStopWord(String)
Whether
, lowercased, is a default stop word.
#LowerCase(String)
Each code point lowercased by the invariant culture; the UTF-16 length can change only for a supplementary character.
#WithoutDiacritics(String)
The term without combining marks: decomposed, marks removed, recomposed.
#WithoutPossessive(String)
A trailing "'s" (apostrophe, right single quotation mark or fullwidth apostrophe; s either case) removed.
#Fields
| Name | Description |
|---|---|
MaxTokenLength
|
The longest word, in UTF-16 characters, that is a token. |