동의어 및 어간 추출

Users rarely type the exact words stored in your documents. They search running but your text says run, or they type laptop when the doc says notebook. Two analysis techniques bridge this gap: stemming and synonyms.

Stemming reduces words to a common root form. running, runs, and ran may all become run. This means a query matches regardless of the grammatical form used.

Elasticsearch ships algorithmic stemmers like porter_stem and the language-aware stemmer filter. They apply rules to strip suffixes quickly without a dictionary.

Dictionary stemmers such as hunspell use real word lists for more accurate, linguistically correct roots. They are slower and need dictionary files but avoid over-stemming.

Stemming has failure modes:

• Over-stemming: unrelated words map to the same root (e.g. universe and university).
• Under-stemming: related words fail to share a root.

Use keyword_marker to protect specific words from stemming.

Synonyms map words with the same meaning to each other. Searching tv can match television. They are applied via a synonym token filter in the analyzer chain.

Synonym rules come in two styles:

• Equivalent (tv, television): all terms are interchangeable.
• Explicit (i-pod => ipod, music player): the left maps to the right only.

Synonyms can be applied when indexing or when searching. Search-time synonyms (via synonym_graph) are preferred because you can update the list without re-indexing the whole corpus.

Multi-word synonyms like ny, new york need the graph-aware synonym_graph filter at search time to be tokenized correctly. The older synonym filter mishandles phrases.

A typical chain applies synonyms first, then stemming, after lowercasing. Order matters: stem after expanding synonyms so all variants get normalized consistently.

Always verify your chain with the _analyze API to confirm the produced tokens match your expectations before relying on it in production.

You learned to widen search recall:

• Stemming reduces word forms to a shared root; watch for over/under-stemming.
• Synonyms map equivalent terms; equivalent vs explicit rules behave differently.
• Prefer synonym_graph at search time for editable, multi-word-safe synonyms.
• Verify analyzer output with the _analyze API.