Install NLTK tool kit and perform stemming.
Solution :
DESCRIPTION:
NLTK : The Natural Language Toolkit (NLTK) is a Python library for natural language processing. It contains modules for tokenization, stemming, lemmatization, tagging, parsing and many text corpora.
Stemming : Stemming reduces a word to its stem (root form) by removing suffixes, so that related words such as connect, connected, connection are treated as one term. The stem need not be a real word (studies → studi). NLTK provides several stemmers: PorterStemmer (the classic rule-based algorithm), SnowballStemmer (Porter2, available for many languages) and LancasterStemmer (more aggressive).
Installation of NLTK:
Python must be installed first. Open the command prompt (Windows) or the terminal (Linux / macOS) and run:
# Windows / Linux / macOS
pip install nltk
# if pip is not found (Linux / macOS)
pip3 install nltk
# check the installation
pip show nltk
PROGRAM: ( nltk_stemming.py )
import nltk
from nltk.stem import PorterStemmer, SnowballStemmer
from nltk.tokenize import wordpunct_tokenize
print("NLTK version:", nltk.__version__)
porter = PorterStemmer()
snowball = SnowballStemmer("english")
words = ["running", "flies", "studies", "happily", "caresses",
"ponies", "connection", "plastered"]
print()
print("{:<12} {:<12} {:<12}".format("Word", "Porter", "Snowball"))
print("-" * 36)
for w in words:
print("{:<12} {:<12} {:<12}".format(w, porter.stem(w), snowball.stem(w)))
# stemming the words of a sentence
sentence = "The children were running and connecting the cables"
tokens = wordpunct_tokenize(sentence)
print("\nOriginal :", tokens)
print("Stemmed :", [porter.stem(t) for t in tokens])
OUTPUT:
$ python nltk_stemming.py
NLTK version: 3.9.1
Word Porter Snowball
------------------------------------
running run run
flies fli fli
studies studi studi
happily happili happili
caresses caress caress
ponies poni poni
connection connect connect
plastered plaster plaster
Original : ['The', 'children', 'were', 'running', 'and', 'connecting', 'the', 'cables']
Stemmed : ['the', 'children', 'were', 'run', 'and', 'connect', 'the', 'cabl']
The NLTK version number on your computer may be different. PorterStemmer and SnowballStemmer need no extra data download.
๐ป Run This Python Program
Install NLTK tool kit and perform stemming.
The selected Python program is loaded
automatically into the editor.
Edit the program, provide input through
STDIN if required
and click Run.
nltk.download()).
If NLTK is not available in the embedded compiler, use
Copy Program and run the program on your
computer (pip install nltk) or in Google Colab.