Have you ever considered the complexity involved in programming text translators, virtual personal assistants, and/or even chatbots? Making machines communicate coherently (and naturally) with humans is challenging and involves, among other techniques, NLP.
Natural Language Processing (NLP) is a sub-area of AI, which can be simplistically conceptualized as a “translator” or tool used to help the machine understand and process natural human language. It seeks "solutions" to problems that require the computational processing of a natural language. Among other techniques, NLP involves named entity recognition (a type of identification of proper names, locations, and other relevant information in text), sentiment analysis, and the generation of natural language itself.
SEE ALSO: “Customer service automation: what is it, what are the advantages and how to do it?”
SEE ALSO: “Sales bot: what is it, how to create one and how to use it to sell more?”
However, although already quite developed and widely used (with numerous applications), the Natural Language Processing still faces complex challenges and obstacles and continues to improve. This is primarily due to the fact that human language is constantly evolving. The Oxford English Dictionary, for example, included 700 new words in its 2021 update. The Portuguese language alone has approximately 370,000 words!
Beyond this complex and ever-changing scenario, another formidable challenge is context. "Consider that within a specific language context, taking into account nuances such as speech intonation, word choice, and pause intervals, the human brain is capable of processing the meaning and significance of a text in milliseconds. Machines, on the other hand, even with advanced NLP models, it has difficulty recognizing all these points”, recalls Simone Faquineli, Artificial Intelligence Specialist at Brazilian Matrix.
Words with double meanings (also called polysemy; there are countless in our vocabulary) make life difficult for machines, because their meaning depends precisely on the context. Another complication is slang and expressions – widely used in everyday life, when used by users interacting with technology, they can lead to confusion and errors.The term "velho" or "véio" is synonymous with "cara" in many situations; remember that "cara" in this sense means "friend," and although a masculine noun, it can be used for women. "Velho," therefore, in a given context, would not even remotely represent a person of advanced age. Feel the drama?!
Another challenge of NLP is grammatical errors. The machine is programmed based on standard language, accurately following spelling and grammar. Users, on the other hand, are prone to errors and/or abbreviations. "When configuring our chatbots, we take into account the most common typos, agreement, and/or grammar errors, which greatly aids the machine's understanding. Furthermore, what we know as 'machine learning' helps the technology improve with each new interaction, making it smarter," adds Simone.
Machine learning is precisely what has enabled (and continues to enable) such significant developments in Artificial Intelligence technologies. It would be virtually impossible to empower a machine with all the countless linguistic and semantic possibilities and variations that exist. The technology's operation and interactions ensure a much more general and faithful absorption of the language's reality, including, for example, regionalisms.
Still, semantic challenges like human irony and sarcasm still pose obstacles—even for highly trained AIs. Conversations with Voice Assistants and Customer Service Chatbots, for example, can not always capture complaints, such as “What wonderful service!” "In these cases, what we do to mitigate such confusion is, through sentiment analysis and a scoring system, program the machine to read the previous and subsequent context, observe punctuation and identify emojis in order to capture the real meaning of the conversation," Simone points out.
What then are the perspectives and predictions that we can make for the evolution of Natural Language Processing as a subfield of Artificial Intelligence?
It is a fact that NLP techniques are becoming more advanced every day and their models are becoming more intelligent. Virtual assistants, translators, chatbots, and other technologies are also increasingly trained, surprising us. However, it's also a fact that linguistic complexity is only increasing. New techniques, subfields, and technologies promise to emerge over time to address such complexity and humanity's latent desire to communicate with machines. Simone concludes, "(...) we are prepared and equipped with the experience and technology to keep up with this evolution and what's to come."
READ ALSO: “Setting schedules and breaks in OMINI”
