LuceneStandardTokenizer Class

Definition

Namespace:: Azure.Search.Documents.Indexes.Models

Assembly:: Azure.Search.Documents.dll

Package:: Azure.Search.Documents v11.6.1

Package:: Azure.Search.Documents v11.7.0-beta.7

Source:: LuceneStandardTokenizer.cs

Source:: LuceneStandardTokenizer.cs

Important

Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

Breaks text following the Unicode Text Segmentation rules. This tokenizer is implemented using Apache Lucene.

public class LuceneStandardTokenizer : Azure.Search.Documents.Indexes.Models.LexicalTokenizer

public class LuceneStandardTokenizer : Azure.Search.Documents.Indexes.Models.LexicalTokenizer, System.ClientModel.Primitives.IJsonModel<Azure.Search.Documents.Indexes.Models.LuceneStandardTokenizer>, System.ClientModel.Primitives.IPersistableModel<Azure.Search.Documents.Indexes.Models.LuceneStandardTokenizer>

type LuceneStandardTokenizer = class
    inherit LexicalTokenizer

type LuceneStandardTokenizer = class
    inherit LexicalTokenizer
    interface IJsonModel<LuceneStandardTokenizer>
    interface IPersistableModel<LuceneStandardTokenizer>

Public Class LuceneStandardTokenizer
Inherits LexicalTokenizer

Public Class LuceneStandardTokenizer
Inherits LexicalTokenizer
Implements IJsonModel(Of LuceneStandardTokenizer), IPersistableModel(Of LuceneStandardTokenizer)

Inheritance: Object

LexicalTokenizer
LuceneStandardTokenizer

Implements: IJsonModel<LuceneStandardTokenizer> IPersistableModel<LuceneStandardTokenizer> IPersistableModel<T>

Constructors

LuceneStandardTokenizer(String)

Initializes a new instance of LuceneStandardTokenizer.

Properties

MaxTokenLength	The maximum token length. Default is 255. Tokens longer than the maximum length are split. The maximum token length that can be used is 300 characters.
Name	The name of the tokenizer. It must only contain letters, digits, spaces, dashes or underscores, can only start and end with alphanumeric characters, and is limited to 128 characters. (Inherited from LexicalTokenizer)

Methods

JsonModelWriteCore(Utf8JsonWriter, ModelReaderWriterOptions)

Explicit Interface Implementations

IJsonModel<LexicalTokenizer>.Create(Utf8JsonReader, ModelReaderWriterOptions)	Reads one JSON value (including objects or arrays) from the provided reader and converts it to a model. (Inherited from LexicalTokenizer)
IJsonModel<LexicalTokenizer>.Write(Utf8JsonWriter, ModelReaderWriterOptions)	Writes the model to the provided Utf8JsonWriter. (Inherited from LexicalTokenizer)
IJsonModel<LuceneStandardTokenizer>.Create(Utf8JsonReader, ModelReaderWriterOptions)	Reads one JSON value (including objects or arrays) from the provided reader and converts it to a model.
IJsonModel<LuceneStandardTokenizer>.Write(Utf8JsonWriter, ModelReaderWriterOptions)	Writes the model to the provided Utf8JsonWriter.
IPersistableModel<LexicalTokenizer>.Create(BinaryData, ModelReaderWriterOptions)	Converts the provided BinaryData into a model. (Inherited from LexicalTokenizer)
IPersistableModel<LexicalTokenizer>.GetFormatFromOptions(ModelReaderWriterOptions)	Gets the data interchange format (JSON, Xml, etc) that the model uses when communicating with the service. (Inherited from LexicalTokenizer)
IPersistableModel<LexicalTokenizer>.Write(ModelReaderWriterOptions)	Writes the model into a BinaryData. (Inherited from LexicalTokenizer)
IPersistableModel<LuceneStandardTokenizer>.Create(BinaryData, ModelReaderWriterOptions)	Converts the provided BinaryData into a model.
IPersistableModel<LuceneStandardTokenizer>.GetFormatFromOptions(ModelReaderWriterOptions)	Gets the data interchange format (JSON, Xml, etc) that the model uses when communicating with the service.
IPersistableModel<LuceneStandardTokenizer>.Write(ModelReaderWriterOptions)	Writes the model into a BinaryData.

Applies to