Stemming

Stemming reduces a word to its stem, an approximate root form, by mechanically stripping prefixes and suffixes according to fixed rules. It doesn't look anything up; it just chops.

That speed comes at a cost. Feed a stemmer historical and history and both collapse toward a stem like histori; final, finally, and finalized all collapse to final. The stems aren't always real English words, they're just short and consistent.

Sometimes stemming lands on a real, meaningful word anyway: going, goes, and go all reduce to go. When that happens it's a genuine win, same root, same meaning, less vocabulary for a model to learn separately.

Because it's cheap to run at scale, stemming is a reasonable default whenever the exact wording of the output doesn't matter, only the downstream classification does, spam filtering, or predicting a review's star rating from its text. The mangled stems in the middle are never shown to a human, so they don't need to be real words.