Evaluating the Performance of Cloud Based NLP APIs for Sentiment and Emotion Detection in Sarcastic Text

Abstract

Sentiment and emotion analysis has become an integral component of Natural Language Processing (NLP), widely applied across domains such as social media monitoring and customer feedback analytics. Despite significant advancements, sarcasm detection remains a persistent challenge, as sarcastic expressions often convey meanings opposite to their literal content, posing difficulties not only for human interpretation but even more so for automated systems. This thesis evaluates the capabilities of four widely used cloud-based NLP APIs—Google Cloud NLP, AWS Comprehend, IBM Watson, and Azure Text Analytics—in interpreting sentiment and emotion in sarcastic text, using the publicly available News Headlines Dataset for Sarcasm Detection. Preprocessing involved tokenizing headlines, removing noise such as punctuation, stopwords, and special characters, and formatting the data into JSON and CSV formats for API compatibility, while custom Python scripts automated the request-response workflow and collected structured output logs. Performance assessment was conducted through quantitative metrics—sentiment and emotion labeling accuracy, average response latency, and estimated cost per 15,000 samples—and qualitative factors such as ease of use, setup complexity, maintenance requirements, and analysis of common misinterpretations. The results indicate that the accuracy for all four APIs degrades significantly in the presence of sarcasm, often misinterpreting overtly positive or negative language at face value and missing the underlying ironic intent. These findings suggest that organizations working with sarcasm-heavy content—such as satirical journalism, online reviews, or political commentary—must account for these limitations and consider hybrid solutions, such as rule-based post-processing or context-aware enhancements. Ultimately, this research contributes empirical evidence to the discourse on sentiment analysis in complex linguistic environments, highlighting that while commercial NLP APIs offer scalable and user-friendly tools, they remain insufficient for nuanced text interpretation without supplementary strategies; future research could explore integrating API outputs with classifiers fine-tuned on sarcasm-specific datasets or leveraging transformer-based architectures better equipped to capture contextual subtleties.