Mind the Morphology: Morphology-Aware Tokenizer Adaptation under Constrained Fine-Tuning

Nhan Tran, Hung To and Hai Nguyen Thi Tuyet
Posts and Telecommunications Institute of Technology
SOICT 2026 · Oral Presentation · Springer CCIS Proceedings
Code

Why this matters

Tokenizers are often treated as fixed preprocessing. MorpheL makes tokenizer adaptation morphology-aware, helping multilingual models reuse meaningful pieces across inflected and derived word forms when the model cannot fully fine-tune.

Research at a glance

6

XNLI languages evaluated

4–8%

lower fragmentation vs. WordPiece

16.24

English–Turkish BLEU

The contribution

  • Morpheme-aware adaptation without annotations: MorpheL uses language-specific candidate positions and corpus-level Mutual Information, without gold morpheme boundaries or an external morphological analyzer.
  • Controlled adaptation: The study isolates tokenizer quality across classifier-only, embeddings-plus-classifier, and full fine-tuning regimes using the same XLM-R backbone and 32K vocabulary budget.
  • Beyond classification: An English–Turkish OPUS-100 experiment tests whether the tokenizer transfers to neural machine translation.
  • Reproducible research: The implementation, configurations, and reproducibility artifacts are publicly available.

Where MorpheL helps most

LanguageAccuracy gainMacro-F1 gainSetting
Turkish+1.27+2.80Constrained adaptation
Swahili+0.49+3.01Constrained adaptation
Russian+0.47+1.23Constrained adaptation

The strongest gains concentrate in morphologically richer languages, especially when the encoder has limited room to adapt.

Machine translation

16.24 BLEU

Highest English–Turkish BLEU on OPUS-100 among BPE, WordPiece, Unigram, and MorpheL.

+1.20 over BPE · +3.53 over WordPiece · +1.08 over Unigram

Mechanism evidence

35.95 → 36.04

On Turkish, MI-guided selection rises far above random boundaries in Macro-F1; the full configuration adds the final improvement.

31.93 random boundaries · 35.95 MI-only · 36.04 full MorpheL

Takeaway: MorpheL shows that tokenizer design can be an actionable lever for low-resource multilingual adaptation—especially for morphologically rich languages under constrained fine-tuning.