What is sentiment analysis? Using NLP and ML to extract meaning

Section 4 addresses multiple challenges faced by researchers during sentiment and emotion analysis. We have provided experiments also with the Lexicon-based approach (LBA), Naïve Bayes (NB), and SVM using BOW representation, for comparison with our neural networks model (combined Conv1D + LSTM) trained by deep learning. The results of classic methods of machine learning are poor but still in most cases much better than the probability of random selection equal 0.166 in the multiclassification task with 6 classes. This table showed that the best model is the neural networks model (combined Conv1D + LSTM). This best detection model was used in a web application for recognition of the emotion type from texts as posts or comments and in a conversation of a ChatBot with a human.

The most common datasets are SemEval, Stanford sentiment treebank (SST), international survey of emotional antecedents and reactions (ISEAR) in the field of sentiment and emotion analysis. SemEval and SST datasets have various variants which differ in terms of domain, size, etc. ISEAR was collected from multiple respondents who felt one of the seven emotions (mentioned in the table) in some situations. The table shows that datasets include mainly the tweets, reviews, feedbacks, stories, etc. A dimensional model named valence, arousal dominance model (VAD) is used in the EmoBank dataset collected from news, blogs, letters, etc.

Sometimes the text can be long and contains multiple emotions or it is on the border of multiple emotions. The main problem is the recognition of negative emotions, where it is difficult to determine whether it is anger, sadness, or fear. This represents a big problem since, e.g., various psychological problems such as depression can also play a role in the emotion of sadness. For this reason, machines cannot replace psychological care for seniors, but they can provide them with entertaining company. At the forefront of techniques employed for emotion detection stands sentiment analysis, also recognized as opinion mining.

Businesses may use automated sentiment sorting to make better and more informed decisions by analyzing social media conversations, reviews, and other sources. The CNN is a type of deep neural network that uses the mathematical function convolution, which can be understood as multiplying two functions. These filters are matrices, usually square, which are used in convolutional layers (Goodfellow et al., 2016). CNNs were initially used for image processing, where square convolution filters with odd dimensions move through the image as illustrated in Figure 2.

5, which includes spelling mistakes, new slang, and incorrect use of grammar. These challenges make it difficult for machines to perform sentiment and emotion analysis. ”, 'why' is misspelled as 'y,' 'you' is misspelled as 'u,' and 'soooo' is used to show more impact.

In situations where the dataset is vast, the deep learning approach performs better than machine learning. Recurrent neural networks, especially the LSTM model, are prevalent in sentiment and emotion analysis, as they can cover long-term dependencies and extract features very well. At the same time, it is important to keep in mind that the lexicon-based approach and machine learning approach (traditional approaches) are also evolving and have obtained better outcomes. Also, pre-processing and feature extraction techniques have a significant impact on the performance of various approaches of sentiment and emotion analysis. Emotional recognition has arisen as an essential field of study that can expose a variety of valuable inputs.

Learn

Natural language processing (NLP) is a subfield of computer science and artificial intelligence (AI) that uses machine learning to enable computers to understand and communicate with human language. One of the best things about Authenticx is that the users don’t have to understand how natural language processing works in order to take advantage of the incredible insights it can bring to their business. Authenticx provides complex data in a way that is easy to understand, presenting important information at the click of a button. IBM Watson NLP Library for Embed, powered by Intel processors and optimized with Intel software tools, uses deep learning techniques to extract meaning and meta data from unstructured data. If you’ve made it this far then it’s fair to say that there’s a strong possibility that you’re interested in exploring the benefits that Lettria’s sentiment analysis could bring to your project or organization.

Following code can be used as a standard workflow which helps us extract the named entities using this tagger and show the top named entities and their types (extraction differs slightly from spacy). Spacy had two types of English dependency parsers based on what language models you use, you can find more details here. Based on language models, you can use the Universal Dependencies Scheme or the CLEAR Style Dependency Scheme also available in NLP4J now.

Building a natural language processing app that uses Hex, HuggingFace, and a simple TF-IDF model to do sentiment analysis, emotion detection, and question detection on natural language text. Sentiment analysis is analytical technique that uses statistics, natural https://chat.openai.com/ language processing, and machine learning to determine the emotional meaning of communications. Future trends in sentiment analysis include enhanced contextual understanding, emotion recognition, and improved accuracy through advanced NLP techniques.

I have covered several topics around NLP in my books “Text Analytics with Python” (I’m writing a revised version of this soon) and “Practical Machine Learning with Python”. Our algorithm analyzes the text to identify the adverbs and adjectives that are modifiers of meaning within a text. Once this is complete and a sentiment is detected within each statement, the algorithm then assigns a source and target to each sentence. Both statements are clearly positive and there’s no real requirement for any great contextual understanding.

Text Analytics with Python - A Practical Real-World Approach to Gaining Actionable Insights from…

Another work (Ahmed et al., 2022) used EEG signals for training models for emotion classification. To solve the problem with asymmetry in different brain regions, they used AsMap and convolutional neural network (CNN) model with the highest accuracy of 97.1%. The second type of data – voice/speech was used for training various machine learning models for vocal emotion recognition in work (Dogdu et al., 2022). From many methods, sequential minimal optimization (SMO), multilayer perceptron (MLP) neural network and logistic regression (LOG) showed better performance (reaching to 87.85, 84.00 and 83.74% accuracies).

The fundamental emotions are the only main features as the text contains the fundamental emotions whose values will be the likelihoods of the emotional state in the sentence. Mapping the emotion string to mathematical values is completed based on data gathering formats. Emotion detection is completed by extracting emotional keywords from the text. These keywords match the knowledge base or the vocabulary like Thesaurus to discover emotional expressions. Detecting a person’s emotional state by analyzing someone’s written text seems challenging.

For instance, it may be particularly important for therapists to identify negative emotions to better understand client avoidance of negative emotions (e.g., Acceptance and Commitment Therapy; Hayes et al., 2006). Using a methodology like this may help supervisors use their time more efficiently and listen to portions of psychotherapy sessions that go beyond those that were selected by their supervisees. Our paper relies on a broad conceptualization of emotion wherein raters were simply asked to rate the positive or negative sentiment of a set of text.

A vital topic of study that can reveal a range of relevant inputs has emerged called emotional recognition. There are various ways of articulating emotions, such as voice and facial expressions, written language, and gestures. The identification of emotions in a written document is essentially a matter of content categorization, incorporating ideas from natural language processing and the disciplines of profound learning. Tanana et al (2016) used naïve coders to identify basic valence of emotion (e.g. positive, negative, neutral) among a large corpus of utterances from psychotherapy sessions. Focus on valence allowed for comparison with existing computer science models of sentiment (Pang & Lee, 2008), and positive and negative emotion categories in dictionary-based programs (Tausczik & Pennebaker, 2010).

Input text can be encoded into word vectors using counting techniques such as Bag of Words (BoW) , bag-of-ngrams, or Term Frequency/Inverse Document Frequency (TF-IDF). Choose a sentiment analysis model that’s aligned with your objectives, size, and quality of training data, your desired level of accuracy, and the resources available to you. The most common models include the rule-based model and a machine learning model. Contextual understanding is a critical aspect of natural language processing (NLP) and sentiment analysis. It refers to the ability of NLP systems and algorithms to grasp the nuances and meaning of words, phrases, or sentences within the broader context in which they are used. In other words, it involves interpreting language not just based on individual words but also considering how those words interact with each other and the surrounding text.

In an emotion detection dataset, it’s best to have as much data as possible that has a broad representation of all races, genders, accents, and ages. This is especially true for healthcare software due to the fact that nearly every person in every population is going to need a healthcare provider at some point in their lives. If the dataset does not contain information for the algorithm to learn from, it is likely to be inaccurate. To gain a more complete understanding of the emotions of a sentence, Lettria uses deep learning to identify the context of the sentiments within a text. So we’ve given you a little background on how natural language processing works and what syntactic analysis is, but we know that you’re here to have a better understanding of sentiment analysis and its applications. How sentiment analysis works, Lettria’s approach to sentiment analysis, and some key use cases.

This type of sentiment analysis can be applied to developing chatbots for efficient conversation routing or helping marketers identify the right B2B campaign for their target audience. In conclusion, sentiment analysis is a powerful tool that helps us decipher the emotions and opinions hidden within the vast ocean of text data. Its applications are limitless, and as technology advances, so will our ability to understand and harness the power of sentiment analysis in various fields. As NLP technology continues to evolve, sentiment analysis is expected to become more context-aware and capable of understanding nuances in human emotion. In the age of information, we are inundated with vast amounts of text data every day.

In sentiment analysis, polarity is the primary concern, whereas, in emotion detection, the emotional or psychological state or mood is detected. Sentiment analysis is exceptionally subjective, whereas emotion detection is more objective and precise. After selecting a sentiment, every piece of text is assigned a sentiment score based on it.

But you’ll need a team of data scientists and engineers on board, huge upfront investments, and time to spare. We focused on the categorical approach and defined the set of six emotions (joy, sadness, anger, fear, love, and surprise) for detection from the text. As was said, the concept of detection is closely related to the concept of categorization or classification. Thus, classification methods of supervised machine learning are a logical choice for emotion detection. In text processing, the Naïve Bayes classifier (as baseline method), SVM, and NN were proven as suitable and precise enough in our experiments.

I am assuming you are aware of the CRISP-DM model, which is typically an industry standard for executing any data science project. Typically, any NLP-based problem can be solved by a methodical workflow that has a sequence of steps. In this article, we will be working with text data from news articles on technology, sports and world news. I will be covering some basics on how to scrape and retrieve these news articles from their website in the next section. When I started delving into the world of data science, even I was overwhelmed by the challenges in analyzing and modeling on text data.

We already looked at how we can use sentiment analysis in terms of the broader VoC, so now we’ll dial in on customer service teams. Social media and brand monitoring offer us immediate, unfiltered, and invaluable information on customer sentiment, but you can also put this analysis to work on surveys and customer support interactions. Real-time sentiment analysis allows you to identify potential PR crises and take immediate action before they become serious issues. Or identify positive comments and respond directly, to use them to your benefit. Brand monitoring offers a wealth of insights from conversations happening about your brand from all over the internet.

Next, we will iterate over each model name and load the model using the [transformers]() package. Building their own platforms can give companies an edge over the competition, says Dan Simion, vice president of AI and analytics at Capgemini. The group analyzes more than 50 million English-language tweets every single day, about a tenth of Twitter’s total traffic, to calculate a daily happiness store. Collect quantitative and qualitative information to understand patterns and uncover opportunities. We will find the probability of the class using the predict_proba() method of Random Forest Classifier and then we will plot the roc curve.

This means that your work will not suffer from the silo effect that is the undoing of many NLP projects. There’s a good chance that you’ve already run campaigns that have included surveys and other initiatives to help you get feedback from leads and customers. Social media monitoring and customer service responses can play a key role in improving brand loyalty, but it also helps you to identify the areas of your brand that are performing the best and those that require attention. Now, say you’re really enjoying this article and decide to leave a comment saying ‘I really like reading’ then you would still return a positive sentence, but the addition of ‘really’ would increase the value of the emotion to .66. Lettria’s API uses resources from psychology and the 8 primary emotions modelled in Putichik’s wheel of emotions (joy, sadness, fear, anger, attract, surprise, and anticipation). That is to say that there are many different scenarios, subtleties, and nuances that can impact how a sentence is processed.

Let’s now leverage this model to shallow parse and chunk our sample news article headline which we used earlier, “US unveils world’s most powerful supercomputer, beats China”. Besides these four major categories of parts of speech , there are other categories that occur frequently in the English language. These include pronouns, prepositions, interjections, conjunctions, determiners, and many others. Furthermore, each POS tag like the noun (N) can be further subdivided into categories like singular nouns (NN), singular proper nouns (NNP), and plural nouns (NNS). Lemmatization is very similar to stemming, where we remove word affixes to get to the base form of a word.

In order to compare our model performance with human reliability, we computed the average pairwise Cohen’s Kappa for course (positive, negative, neutral) human sentiment ratings. Based on this computation, our best model exceeded human performance on a test set by 14%. This may be surprising that a model can exceed human performance on this task, but we should note that we are comparing to the average human-human agreement (some rater-pairs had agreement as high as .54). Moreover, the model we tested has exceeded the performance of individual humans on a number of NLP tasks (Devlin et al, 2018).

The group’s full texts are detected by different human emotions based on text analysis; the measurement function is zero. The complete classification accuracy is obtained from the recall and F measure of different human emotions. The effect of emotions is detected by various parameters of the word clustering approach in the first how do natural language processors determine the emotion of a text? group. In the second group, the emotional Classification is compared with results when using various characteristics and coefficients. According to the text analysis, the provinces’ analysis’s detection results vary with different emotions. Each lateral row is the actual outcome, and the result obtained is every lateral row.

We provide a more comprehensive description of each model below than the original study to aid with the comparison of otherwise complex models. Prior results from Tanana et al (2016) are presented in Table 1, along with results from the current study. A GPU is composed of hundreds of cores that can handle thousands of threads in parallel. GPUs have become the platform of choice to train ML and DL models and perform inference because they can deliver 10X higher performance than CPU-only platforms.

The whole article was given a sentiment score, followed by entity-level sentiment. I recorded the score of the entire article, as well as the score of the company name as an entity. Now we encounter semantic role labeling (SRL), sometimes called "shallow parsing." SRL identifies the predicate-argument structure of a sentence – in other words, who did what to whom.

Discover how we analyzed the sentiment of thousands of Facebook reviews, and transformed them into actionable insights. You can use it on incoming surveys and support tickets to detect customers who are ‘strongly negative’ and target them immediately to improve their service. Zero in on certain demographics to understand what works best and how you can improve. Get an understanding of customer feelings and opinions, beyond mere numbers and statistics. Understand how your brand image evolves over time, and compare it to that of your competition.

Using an NLP sentiment analysis dataset, Authenticx is able to provide healthcare organizations with information on the reason the patient called and how they felt about it. It can track the caller’s sentiment through the call thanks to a natural language processing sentiment analysis Python code. This allows the organization to identify how their caller is feeling throughout the course of their call and if they feel satisfied by the end – whether or not their issue received their desired solution. These challenges highlight the complexity of human language and communication.

It enables AI to imitate how humans learn and has revolutionized the field of sentiment analysis in many ways. With ML, algorithms can be trained on labeled data (supervised learning) or it can identify patterns in unlabeled data (unsupervised learning). It also allows advanced neural networks to extract complex data from text through deep learning. Current approaches to natural language processing are based on deep learning, a type of AI that examines and uses patterns in data to improve a program's understanding. SentiWordNet (Esuli and Sebastiani 2006) and Valence Aware Dictionary and Sentiment Reasoner (VADER) (Hutto and Gilbert 2014) are popular lexicons in sentiment.

Consider words like "New York" that should be treated as a single token rather than two separate words or contractions that could be improperly split at the apostrophe. You can foun additiona information about ai customer service and artificial intelligence and NLP. Please note that in this appendix, we will show you how to add the Sentiment transformer. However, we don't recommend that you run this on Aquarium, as Aquarium provides a small environment; the experiment might not finish on time or might not give you the expected results.

Word embedding is commonly used for several NLP functions, including computer translation, interpretation of emotions, and question answering. The techniques of NLP enhance the efficiency of learning approaches by combining semantical and syntactic language characteristics. Emotion is conveyed in different forms, such as face and voice, gestures, and written language. Emotion can be observed with text emotion recognition, and it is a matter of information classification involving natural language processing and deep learning principles. Findings demonstrate that the suggested approach is a very promising choice for emotion recognition due to its powerful ability to learn raw data features directly.

• NLP Architect by Intel is a Python library for deep learning topologies and techniques.
• Here’s a quite comprehensive list of emojis and their unicode characters that may come in handy when preprocessing.
• Sentiment analysis is the automated interpretation and classification of emotions (usually positive, negative, or neutral) from textual data such as written reviews and social media posts.
• Many researchers implemented the proposed models on their dataset collected from Twitter and other social networking sites.

If you aren’t listening to your customers wherever they speak about you then you are missing out on invaluable insights and information. That means that social media platforms are areas where your leads, customers, or former customers will be sharing their honest opinions about your product and services. Natural language processing allows computers to interpret and understand language through artificial intelligence. Over the past 50 years it has developed into one of the most advanced and common applications for artificial intelligence and forms the backbone of everything from your email spam filters to the chatbots you interact with on websites.

Overcoming them requires advanced NLP techniques, deep learning models, and a large amount of diverse and well-labelled training data. Despite these challenges, sentiment analysis continues to be a rapidly evolving field with vast potential. Finally, in order to test a more recent innovation in NLP we used the Bidirectional Encoder Representations and Transformations (BERT; Devlin et al, 2018).

Or start learning how to perform sentiment analysis using MonkeyLearn’s API and the pre-built sentiment analysis model, with just six lines of code. Then, train your own custom sentiment analysis model using MonkeyLearn’s easy-to-use UI. Sentiment analysis can be used on any kind of survey – quantitative and qualitative – and on customer support interactions, to understand the emotions and opinions of your customers. Tracking customer sentiment over time adds depth to help understand why NPS scores or sentiment toward individual aspects of your business may have changed. More recently, new feature extraction techniques have been applied based on word embeddings (also known as word vectors).

On social media, people usually communicate their feelings and emotions in effortless ways. As a result, the data obtained from these social media platform's posts, audits, comments, remarks, and criticisms are highly unstructured, making sentiment and emotion analysis difficult for machines. As a result, pre-processing is a critical stage in data cleaning since the data quality significantly impacts many approaches that follow pre-processing. The organization of a dataset necessitates pre-processing, including tokenization, stop word removal, POS tagging, etc. (Abdi et al. 2019; Bhaskar et al. 2015). Some of these pre-processing techniques can result in the loss of crucial information for sentiment and emotion analysis, which must be addressed. Sentiment analysis is perhaps one of the most popular applications of NLP, with a vast number of tutorials, courses, and applications that focus on analyzing sentiments of diverse datasets ranging from corporate surveys to movie reviews.

For instance, in the business world, vendors use social media platforms such as Instagram, YouTube, Twitter, and Facebook to broadcast information about their product and efficiently collect client feedback (Agbehadji and Ijabadeniyi 2021). People’s active feedback is valuable not only for business marketers to measure customer satisfaction and keep track of the competition but also for consumers who want to learn more about a product or service before buying it. Sentiment analysis assists marketers in understanding their customer's perspectives better so that they may make necessary changes to their products or services (Jang et al. 2013; Al Ajrawi et al. 2021).

Sentiment analysis evaluates text, often product or service reviews, to categorize sentiments as positive, negative, or neutral. This process is vital for organizations, as it helps gauge customer satisfaction with their offerings. Companies across various sectors, including sales, finance, and healthcare, can understand and improve user experiences by analyzing large volumes of customer feedback. Companies use sentiment analysis to evaluate customer messages, call center interactions, online reviews, social media posts, and other content. Sentiment analysis can track changes in attitudes towards companies, products, or services, or individual features of those products or services. NLP uses either rule-based or machine learning approaches to understand the structure and meaning of text.

Contact Apptension and take the first step towards transforming your business with innovative digital solutions. Confidently take action with insights that close the gap between your organization and your customers. Pull customer interaction data across vendors, products, and services into a single source of truth. Gain a deeper level understanding of contact center conversations with AI solutions. Unsolicited feedback is an unbiased, renewable source of customer insights that surfaces what’s truly top of mind for the customer in their own words.

The precision of most forms of emotions has increased, and the uncertainty of emotion is mitigated by integrating audible and text psychological functionality. Experimental findings indicate that modal fusion may effectively minimize emotional confusion and enhance emotional sensitivity. Well-known NLP Python library with pre-trained models for entity recognition, dependency parsing, and text classification. It is the preferred choice for many developers because of its intuitive interface and modular architecture. This advanced text mining technique can reveal the hidden thematic structure within a large collection of documents. Sophisticated statistical algorithms (LDA and NMF) parse through written documents to identify patterns of word clusters and topics.

In the current study, we extended findings from Tanana et al (2016) by comparing the sentiment model to LIWC and an innovative NLP model, BERT. We hypothesized that the prior NLP models from Tanana et al (2016) would outperform LIWC, and that BERT would outperform all models. At present, text-based methods for evaluating emotion in psychotherapy are reliant on dictionary-based methods. Mergenthaler (1996) was one of the first researchers to create a quantitative method for measuring emotional expression in psychotherapy. Mergenthaler and Bucci (1999) hypothesized that key moments in the psychotherapy process involved client expression of both high emotional content and high verbal abstraction.

Emotions constitute intricate mental states that reflect our feelings, often conveyed through language, tone, and even facial expressions. Emotion detection with NLP entails the meticulous analysis of textual data, encompassing written content and spoken words, aiming to discern the emotional tone or sentiment embedded within these expressions. Natural Language Processing (NLP) is all about leveraging tools, techniques and algorithms to process and understand natural language-based data, which is usually unstructured like text, speech and so on. In this series of articles, we will be looking at tried and tested strategies, techniques and workflows which can be leveraged by practitioners and data scientists to extract useful insights from text data. This article will be all about processing and understanding text data with tutorials and hands-on examples. That’s where natural language processing with sentiment analysis can ensure that you are extracting every bit of possible knowledge and information from social media.

Identifying the emotions of the text plays a vital role in human-computer interaction (HCI). An individual’s speech can convey emotions, facial expressions, and written texts called facial, text-based, and speech emotions. Adequate work has been performed on facial and speech emotion detection, and a text-based emotional recognition system also needs to draw researchers. Identifying human emotions in the text in computational linguistics is becoming progressively significant from an application perspective.

When the journalist was writing the article, they used their own opinions within, whether it was conscious or unconscious bias. In the same way, the labelling process for training data has to start with humans. The results of the research show that 8 times out of 10, Google’s NLP tool agreed with my own classification of sentiment. It would be interesting to find out how they trained the sentiment analysis model, but in true Google style, they have not released any information about the detailed structure. This score can help you to understand the level of emotional content within the text. Google explains that you can distinguish truly neutral articles or documents, as they will have a low magnitude score.

All the big cloud players offer sentiment analysis tools, as do the major customer support platforms and marketing vendors. Conversational AI vendors also include sentiment analysis features, Sutherland says. In conclusion, Sentiment Analysis stands at the intersection of NLP and AI, offering valuable insights into human emotions and opinions. As organizations increasingly recognize the importance of understanding sentiments, the application of sentiment analysis continues to grow across diverse industries. People usually express their anger or disappointment in sarcastic and irony sentences, which is hard to detect (Ghanbari-Adivi and Mosleh 2019). For instance, in the sentence, “This story is excellent to put you in sleep,” the excellent word signifies positive sentiment, but in actual the reviewer felt it quite dull.

Sentiment analysis uses computational techniques to determine the emotions and attitudes within textual data. Natural language processing (NLP) and machine learning (ML) are two of the major approaches that are used. The next step is to establish features to help the model identify sentiments. This process involves the creation, transformation, extraction, and selection of the features or variables most suitable for creating an accurate machine learning algorithm. Machine language and deep learning approaches to sentiment analysis require large training data sets. Commercial and publicly available tools often have big databases, but tend to be very generic, not specific to narrow industry domains.

Let’s use this now to get the sentiment polarity and labels for each news article and aggregate the summary statistics per news category. No surprises here that technology has the most number of negative articles and world the most number of positive articles. Sports might Chat GPT have more neutral articles due to the presence of articles which are more objective in nature (talking about sporting events without the presence of any emotion or feelings). Let’s dive deeper into the most positive and negative sentiment news articles for technology news.

When new pieces of feedback come through, these can easily be analyzed by machines using NLP technology without human intervention. These neural networks try to learn how different words relate to each other, like synonyms or antonyms. It will use these connections between words and word order to determine if someone has a positive or negative tone towards something. There are different machine learning (ML) techniques for sentiment analysis, but in general, they all work in the same way.

Statistical programs designed to conduct sentiment analysis can consume hundreds of hours of data, and create analyses of sessions almost immediately, whereas behavioral coding often takes months and requires a multitude of human resources. Mental health researchers have already demonstrated the capacity of developing and training more complex NLP models (see Imel et al, 2019). Aspect-based sentiment analysis goes one level deeper to determine which specific features or aspects are generating positive, neutral, or negative emotion. Businesses can use this insight to identify shortcomings in products or, conversely, features that generate unexpected enthusiasm. Emotion analysis is a variation that attempts to determine the emotional intensity of a speaker around a topic. Text mining, also known as text data mining or text analytics, sits at the crossroads of data analysis, machine learning, and natural language processing.

Currently, a major focus in NLP is developing methods that correctly identify the emotion related phenomenon in passages using only the written words - often called sentiment analysis in computer science (for a review, see Pang & Lee, 2008). This field is broad, including classification of emojis (Read, 2005), tone of movie reviews (Socher, Pennington, Huang, Ng & Manning, 2011), and product reviews (Nasukawa & Yi, 2003). This study compared existing sentiment models (Tanana et a, 2016) with the LIWC coding system, as well as an innovative deep learning technique BERT (Devlin et al, 2018). We found that the newer NLP method (BERT), which can leverage large existing language datasets, outperformed the prior n-gram and RNN models from Tanana et al, as well as the commonly used dictionary model LIWC.