๐Ÿ’ฌ
B.TECH III YEAR โ€ข II SEMESTER โ€ข R22

Natural Language Processing - Lab Manual

Natural Language Processing Laboratory โ€ข Program Solution

๐Ÿ“˜ R22 Regulation ๐ŸŽ“ III Year ๐Ÿ“š II Semester ๐Ÿ’ฌ NLP Using Python ๐Ÿงช Lab Program
Selected Program

Create a Sample list for at least 5 words with ambiguous sense and Write a Python program to implement WSD.

โ† All NLP Programs

Solution :

DESCRIPTION:

Word Sense Disambiguation (WSD) is the task of finding the correct meaning (sense) of an ambiguous word from the context in which it is used. For example, bank means a financial institution in “deposit money in the bank” and the side of a river in “sat on the bank of the river”.

The program uses the simplified Lesk algorithm: a sample list of ambiguous words is created, each with two or more senses and a short definition (gloss). For a given sentence, the words of the sentence (without stop words) are compared with the words of every definition, and the sense with the largest overlap is selected. If no word overlaps, the first sense is returned.

PROGRAM: ( wsd.py )

 
import re

# Simplified Lesk algorithm for Word Sense Disambiguation (WSD):
# choose the sense whose definition has the most words in common with the sentence.

stop_words = {"a", "an", "the", "is", "are", "was", "were", "in", "on", "at", "to", "of",
              "and", "or", "with", "for", "by", "that", "it", "he", "she", "his", "her",
              "from", "into", "this", "as", "after", "when", "where", "we", "i"}

# sample list of ambiguous words and their senses
senses = {
    "bank": [
        ("financial institution", "an organization where people deposit money, keep a savings account and take a loan"),
        ("river side", "the sloping land along the side of a river or a stream"),
    ],
    "bat": [
        ("animal", "a small flying animal that sleeps upside down in a cave and hunts insects at night"),
        ("sports equipment", "a wooden stick used to hit the ball in cricket or baseball"),
    ],
    "bass": [
        ("fish", "an edible fish that lives in a lake or the sea and is caught with a fishing rod"),
        ("music", "a low sound or a musical instrument that plays low notes in a band"),
    ],
    "spring": [
        ("season", "the season of the year after winter when flowers bloom and the weather becomes warm"),
        ("coil", "a coil of metal wire that returns to its shape after it is pressed or stretched"),
    ],
    "crane": [
        ("bird", "a tall bird with a long neck and long legs that stands in water"),
        ("machine", "a tall machine with a long arm used to lift and move heavy loads at a building site"),
    ],
    "bark": [
        ("tree covering", "the rough outer covering of the trunk of a tree"),
        ("dog sound", "the loud sound that a dog makes"),
    ],
}

def words_of(text):
    return set(w for w in re.findall(r"[a-z]+", text.lower()) if w not in stop_words)

def disambiguate(word, sentence):
    context = words_of(sentence) - {word}
    best_sense, best_gloss, best_score = senses[word][0][0], senses[word][0][1], 0
    for sense, gloss in senses[word]:
        score = len(context & words_of(gloss))
        if score > best_score:
            best_sense, best_gloss, best_score = sense, gloss, score
    return best_sense, best_gloss, best_score

tests = [
    ("bank",   "He went to the bank to deposit money and take a loan"),
    ("bank",   "We sat on the bank of the river and watched the stream"),
    ("bat",    "The bat flew out of the cave at night to hunt insects"),
    ("bass",   "He caught a big bass in the lake with his fishing rod"),
    ("spring", "The flowers bloom in spring when the weather becomes warm"),
    ("crane",  "The crane lifted the heavy load at the building site"),
    ("bark",   "The dog started to bark loudly"),
    ("bark",   "The bark of the old tree is rough"),
]

for word, sentence in tests:
    sense, gloss, score = disambiguate(word, sentence)
    print("Word     :", word)
    print("Sentence :", sentence)
    print("Sense    :", sense)
    print("Meaning  :", gloss)
    print("Overlap  :", score)
    print()

OUTPUT:

 
$ python wsd.py
Word     : bank
Sentence : He went to the bank to deposit money and take a loan
Sense    : financial institution
Meaning  : an organization where people deposit money, keep a savings account and take a loan
Overlap  : 4

Word     : bank
Sentence : We sat on the bank of the river and watched the stream
Sense    : river side
Meaning  : the sloping land along the side of a river or a stream
Overlap  : 2

Word     : bat
Sentence : The bat flew out of the cave at night to hunt insects
Sense    : animal
Meaning  : a small flying animal that sleeps upside down in a cave and hunts insects at night
Overlap  : 3

Word     : bass
Sentence : He caught a big bass in the lake with his fishing rod
Sense    : fish
Meaning  : an edible fish that lives in a lake or the sea and is caught with a fishing rod
Overlap  : 4

Word     : spring
Sentence : The flowers bloom in spring when the weather becomes warm
Sense    : season
Meaning  : the season of the year after winter when flowers bloom and the weather becomes warm
Overlap  : 5

Word     : crane
Sentence : The crane lifted the heavy load at the building site
Sense    : machine
Meaning  : a tall machine with a long arm used to lift and move heavy loads at a building site
Overlap  : 3

Word     : bark
Sentence : The dog started to bark loudly
Sense    : dog sound
Meaning  : the loud sound that a dog makes
Overlap  : 1

Word     : bark
Sentence : The bark of the old tree is rough
Sense    : tree covering
Meaning  : the rough outer covering of the trunk of a tree
Overlap  : 2


๐Ÿ’ป Run This Python Program

Create a Sample list for at least 5 words with ambiguous sense and Write a Python program to implement WSD.
The selected Python program is loaded automatically into the editor. Edit the program, provide input through STDIN if required and click Run.

The compiler is embedded using OneCompiler. Your Python program runs inside the embedded Python environment; Study Glance does not execute Python code directly on your web server. If the program asks for input, type the value in the STDIN box before clicking Run. Programs that use the NLTK library need the package and its data (nltk.download()). If NLTK is not available in the embedded compiler, use Copy Program and run the program on your computer (pip install nltk) or in Google Colab.
โ† Back to NLP Lab Programs ๐Ÿ“š All Lab Manuals