All You Need to Know to Build an AI Chatbot With NLP in Python

Preprocessing plays an important role in enabling machines to understand words that are important to a text and removing those that are not necessary.

Cosine similarity determines the similarity score between two vectors. In NLP, the cosine similarity score is determined between the bag of words vector and query vector.

Stemming means the removal of a few characters from a word, resulting in the loss of its meaning. For e.g., stemming of “moving” results in “mov” which is insignificant. On the other hand, lemmatization means reducing a word to its base form. For e.g., “studying” can be reduced to “study” and “writing” can be reduced to “write”, which are actual words.