๐Ÿ’ฌ
B.TECH III YEAR โ€ข II SEMESTER โ€ข R22

Natural Language Processing - Lab Manual

Natural Language Processing Laboratory โ€ข Program Solution

๐Ÿ“˜ R22 Regulation ๐ŸŽ“ III Year ๐Ÿ“š II Semester ๐Ÿ’ฌ NLP Using Python ๐Ÿงช Lab Program
Selected Program

Write a Python Program to perform following tasks on text.

โ† All NLP Programs

Solution :

DESCRIPTION:

Tokenization :
Tokenization is the process of breaking a text into smaller units called tokens, usually words or terms. It helps in analyzing text by separating meaningful elements from a sentence. Tokenization is a basic and essential step in text preprocessing for natural language processing. It simplifies further text analysis tasks.

Stop Word Removal :
Stop word removal is the process of eliminating commonly used words such as is, the, and, in that carry little meaningful information. These words are removed to reduce text size and improve processing efficiency. It helps focus on important words that contribute to the actual meaning of the text.

PROGRAM: ( tokenization_stopword_removal.py )

 
# To read text from user
text = input("Enter the text: ")

# Convert text to lowercase
text = text.lower()

# Tokenization
tokens = clean_text.split()

print("\nTokens after Tokenization:")
print(tokens)

# Stop word removal

# List of stop words
stop_words = [
    "is", "am", "are", "was", "were", "be",
	"a", "an", "the", "and", "or", "but", "if", "in", "on",
    "at", "to", "for", "with", "by", "of", "as", "from"
]

filtered_tokens = []
for word in tokens:
    if word not in stop_words:
        filtered_tokens.append(word)

print("\nTokens after Stop Word Removal:")
print(filtered_tokens)


OUTPUT:

 
$ python nlpw1.py
Enter the text: NLP is a powerful tool to work with text data

Tokens after Tokenization:
['nlp', 'is', 'a', 'powerful', 'tool', 'to', 'work', 'with', 'text', 'data']

Tokens after Stop Word Removal:
['nlp', 'powerful', 'tool', 'work', 'text', 'data']



๐Ÿ’ป Run This Python Program

Write a Python Program to perform following tasks on text.
The selected Python program is loaded automatically into the editor. Edit the program, provide input through STDIN if required and click Run.

The compiler is embedded using OneCompiler. Your Python program runs inside the embedded Python environment; Study Glance does not execute Python code directly on your web server. If the program asks for input, type the value in the STDIN box before clicking Run. Programs that use the NLTK library need the package and its data (nltk.download()). If NLTK is not available in the embedded compiler, use Copy Program and run the program on your computer (pip install nltk) or in Google Colab.
โ† Back to NLP Lab Programs ๐Ÿ“š All Lab Manuals