Elasticsearch Text Analysis FAQ & Answers

Explicit mapping uses => to define one-way expansion: 'laptop, notebook => computer' means searching 'laptop' or 'notebook' matches documents containing 'computer', but not vice versa. Implicit mapping (comma-separated without =>) creates bidirectional equivalence: 'laptop, notebook, computer' means all three terms match each other. Best practice: use explicit mappings for precision control. Implicit mappings can cause unexpected matches and degrade search precision when one term is much broader than others.

The standard analyzer is the default analyzer in Elasticsearch and includes: (1) No character filters. (2) Standard tokenizer - grammar-based tokenization using Unicode Text Segmentation algorithm (Unicode Standard Annex #29), works well for most languages. (3) Two token filters: lowercase filter and stop token filter (disabled by default, empty stopwords list). Configurable parameters: max_token_length (default 255) and stopwords (default empty).

The Synonyms Management API was introduced in Elasticsearch 8.10 as a technical preview and became stable in Elasticsearch 8.13. It allows managing synonyms in an internal system index via REST API without file management. Key endpoints: PUT /_synonyms/{set_id} to create/update, GET /_synonyms/{set_id} to retrieve, DELETE /_synonyms/{set_id} to remove. When a synonyms set is updated, search analyzers using it are automatically reloaded on all indices.

Elasticsearch provides two tokenizers for partial word matching: (1) ngram tokenizer - breaks text into words at specified characters, then generates sliding window n-grams. Example: 'quick' with min_gram=2, max_gram=3 produces [qu, ui, ic, ck, qui, uic, ick]. Good for substring matching. (2) edge_ngram tokenizer - generates n-grams anchored to the start of each word. Example: 'quick' produces [q, qu, qui, quic, quick]. Ideal for autocomplete/search-as-you-type functionality.

No. If updateable is set to true for a synonym or synonym_graph token filter, the corresponding analyzer can ONLY be used as a search_analyzer. It cannot be used for indexing. Attempting to use an updateable synonym filter in an index analyzer will result in an error. This restriction exists because search-time synonym expansion requires different token graph handling than index-time.

The keyword tokenizer is a no-op tokenizer that outputs the entire input string as a single token, unchanged. Use cases: (1) Exact match fields where you want the whole value searchable as one unit (email addresses, product codes, IDs). (2) Combined with token filters like lowercase for case-insensitive exact matching. (3) Structured data that should not be broken into words. For keyword fields that need no analysis at all, use the keyword field type instead of text with keyword tokenizer.

Use the escaped_tags parameter to specify HTML tags that should NOT be stripped. Example configuration: {"type": "html_strip", "escaped_tags": ["b", "i", "em", "strong"]}. This preserves bold and italic tags while stripping all other HTML. The filter still decodes HTML entities like & to & even for preserved tags. Useful when you need to index HTML content but want to preserve certain formatting tags for display.

When Elasticsearch security features are enabled, you must have the 'manage' index privilege for the target data stream, index, or index alias to use the _reload_search_analyzers API. This is a relatively high privilege level - the same privilege required for operations like force merge, refresh, and close/open index.

The uax_url_email tokenizer is like the standard tokenizer but recognizes URLs and email addresses as single tokens instead of breaking them apart. Standard tokenizer breaks '[email protected]' into ['user', 'example.com'] and 'https://example.com/path' into multiple tokens. The uax_url_email tokenizer keeps these as single tokens: ['[email protected]'] and ['https://example.com/path']. Essential for indexing content where searching by complete URLs or email addresses is required.

Search-time synonym expansion adds approximately 1-2 milliseconds of query latency compared to index-time synonyms. This overhead is negligible for most applications. The tradeoff is worth it because search-time synonyms: (1) require no reindexing when synonyms change, (2) produce smaller indexes (no duplicate term storage), (3) allow updateable: true for zero-downtime synonym updates. Only consider index-time synonyms for extreme low-latency requirements (sub-10ms queries) where every millisecond matters.