Human Affective States Ontology for Sentiment Analysis

Tracking #: 1917-3130

Authors: 
Rana Abaalkhail
Benjamin Guthier1
Abdulmotaleb El Saddik

Responsible editor: 
Lora Aroyo

Submission type: 
Ontology Description
Abstract: 
Social media provides a platform where users share an enormous amount of information about events, products,experiences and more. This information may contain user sentiments and feelings.Sentiment analysis helps monitor and analyze the opinions of users. An ontology has the ability to express the concepts shared, as well as their relationships, in a semantically rich representation. This strong feature enables an ontology to be applied in the area of sentiment analysis. In this paper, we propose the development of a Human Affective States Ontology, which we will refer to as HASO. We employ HASO to the problem of sentiment analysis. We argue that this ontology can compete with state of the art machine learning approaches to detect the sentiment contained in textual data. By using HASO, we classify the sentiment found in the SemEval-2017 dataset and compare our results with those obtained by the teams that participated in this task. The results of our work show the effectiveness of the proposed ontology (HASO) in capturing sentiment, especially when compared to machine learning approaches.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
Reject

Solicited Reviews:
Click to Expand/Collapse
Review #1
Anonymous submitted on 23/Aug/2018
Suggestion:
Major Revision
Review Comment:

The paper proposes a Human Affective States Ontology (called HASO), represented in OWL, for modeling human emotions and sentiment. It is inspired by several psychological theories (e.g., Ekman, Cowie, OCC, etc.) and embeds many of the existing sentiment and emotion lexicons (e.g., WordNet, SenticNet, Harvard General Enquirer, etc.). The authors use Methontology for building the ontology, several tools to check its validity, and a custom modularization procedure in Protégé. Using a standard data set for tweets classifications and the SentiStength method of aggregating word sentiment, the authors show that the proposed ontology-based approach works better than some of the existing, classical machine learning methods.

The ontology seems to be very extensive ensuring a good coverage of the emotion and sentiment concepts. Also, based on the evaluation results, the authors show the usefulness of the proposed ontology. Nevertheless, it is not clear to me how the authors reconciled the various psychological theories and sentiment lexicons (how did you deal with redundancy and conflicts?). It is also not clear how the ontology relates to domain specific ontologies as proposed in [1,2,3] where complex axioms are able to capture domain specific sentiment (e.g., “cold” in “cold pizza” denotes negative sentiment, while “cold” in “cold beer” denotes positive sentiment). Such discussions need to be part of the related work.

[1] Mauro Dragoni, Soujanya Poria, Erik Cambria: OntoSenticNet: A Commonsense Ontology for Sentiment Analysis. IEEE Intelligent Systems 33(3): 77-85 (2018)
[2] Kim Schouten, Flavius Frasincar, Franciska de Jong: Ontology-Enhanced Aspect-Based Sentiment Analysis. ICWE 2017: 302-320
[3] Kim Schouten, Flavius Frasincar: Ontology-Driven Sentiment Analysis of Product and Service Aspects. ESWC 2018: 608-623

Why the deep learning solutions have been dismissed from the evaluation? I can imagine that these have a good performance and you might have good reasons for not using them in the comparison, but these reasons have to be given in the paper otherwise the evaluation becomes dubious. Also, you seem to be silent on the various senses a word can have, yes, you do classify a word under multiple concepts, but how do you know which concept to consider when processing text?

You seem to separate “recognition” from state without a direct link between them (see Figure 1). How do you know how to identify a particular state from a particular “recognition” (e.g., while there might be many words that denote mood, some denote bad mood while others denote good mood). You make use of HEO ontology (I assume this is the Human Emotion Ontology) but this is not properly introduced and referenced in the paper. Regarding the modularity task, should here not be some criterium on semantic cohesion when the modules are defined (counting the number of lines or axioms seems insufficient to me).

It is not clear why the sentiments associated with the emojis from reference [25] (paper) available at http://people.few.eur.nl/hogenboom/files/EmoticonSentimentLexicon.zip were not directly used (instead of converting the emojis to their word equivalent expressions and then determining the sentiment), is your solution working better (some analysis needs to be given)? Which were the used SPARQL queries?

The paper misses numerous references, e.g., HEO, Ekman model, Douglas-Cowie category model, OCC model, Drummond emotion vocabulary, Frijda Action Tendency model, NeOn dictionary, etc. There are several style issues present sometimes in the paper: there should be a space before/after references in text (unless they are the last token), there should be no space before a footnote number, there should be no space before a “.” or “,” , etc.

Other comments:
-page 2: “How Net dictionary”, which Net dictionary do you mean?
-page 8: ill formed sentence starting with small letter “Since recognizing the need…”
-page 9: “we divide” instead of “We divide”
-page 10: “which is clarified” instead of “which clarified”
-page 10: “Figure shows”, which figure?
-page 12: ill formed sentence “...have different representation in another domain and context [28].”

Review #2
Anonymous submitted on 08/Sep/2018
Suggestion:
Major Revision
Review Comment:

The authors present, from one hand, the development of a Human Affective States Ontology (HASO), and from the other one, its application for sentiment analysis (e.g., Tweet Polarity Calculation). HASO provides knowledge and a common vocabulary for human affective states (emotion, mood, sentiment) in a machine-readable format. Some experimental results using SemEval-2017 dataset shows the effectiveness of the proposed approach for sentiment classification w.r.t. to some machine learning approaches.

Some aspects of idea behind the paper are quite interesting, but its presentation and technical qualities should be strongly improved. In my opinion, critical factors are the description of the framework for the ontology management and experiments. Some suggestions are in the following reported to enhance quality of the paper.

The Introduction section should be enriched with a motivating example for the work that better highlights importance of sentiment analysis for SNA applications.

The Related Work Section should be enriched by considering other more recent data mining techniques (also based on deep learning approaches) for sentiment analysis. In addition, several ontology-based approaches quite similar to the proposed one have not be considered (e.g. by Colace et al.: “Probabilistic Approaches for Sentiment Analysis: Latent Dirichlet Allocation for Ontology Building and Sentiment Extraction” and “Terminological ontology learning and population using latent Dirichlet allocation”) in the analysis of the literature.

The authors present in the paper only the ontology structure but more information should be detailed about methodology and process used to build the ontology. The authors should add more details and several examples. To this end, it could be useful to present the framework at the basis of HASO with several implementation details and process workflow (from analysis of data sources to ontology population/updating).

Can the effectiveness of the model be compared with other approaches based on different ontology-based techniques? Moreover, information about efficiency of the proposed (running times, scalability) method should be added in the experimental Section.

Finally, a deep linguistic review is necessary.

Review #3
Anonymous submitted on 01/Oct/2018
Suggestion:
Major Revision
Review Comment:

The paper presents HASO: an ontology modeling human affective states.
Topics of the proposed manuscript are inline with the scope of the Semantic Web journal.
However, the paper cannot be accepted in the current form because it lacks of clarity, novelty, and the effectiveness of the proposed ontology has not been fairly demonstrated.
First of all, I want to clarify that opinion mining and sentiment analysis are not synonyms.
Opinion mining is the task of detecting if a text contains opinion or not and if such opinions are subjective or objective.
Instead, sentiment analysis is the task of inferring the polarity of the opinion detected by the opinion mining task.
I strongly invite the authors to fix this error in the introduction.

Concerning the issues, I can summarize them in the following three points: (i) comparison with the state of the art; (ii) construction of the ontology; and (iii) usefulness.

1. COMPARISON WITH THE STATE OF THE ART
Section 2 discusses the literature about ontologies and other structured resources within the field of sentiment analysis and opinion mining.
This section has to be enriched by providing a more detailed discussion about the contribution of HASO with respect to the mentioned resources.
Which are the gap to fill?
How does HASO fill these gaps?
What does HASO enable that the mentioned resources do not?
The authors should answer to these questions into the paper in order to clarify the reader's mind about the importance of the work proposed in this paper how it advances the state of the art.
In Section 2 there are also other minor issues to fix:
- when SentiStrenghts is mentioned for the first time in Section 2.1, the reference has to be provided;
- concerning the Emotive Ontology, it is not necessary to report the reference[11] twice. One when the resource is mentioned for the first time is enough.
- reference to SenticNet is wrong: SenticNet 5 has been released and also its metamodel OntoSenticNet. The authors may find details here:
Mauro Dragoni, Soujanya Poria, Erik Cambria: OntoSenticNet: A Commonsense Ontology for Sentiment Analysis. IEEE Intelligent Systems 33(3): 77-85 (2018)
Erik Cambria, Soujanya Poria, Devamanyu Hazarika, Kenneth Kwok: SenticNet 5: Discovering Conceptual Primitives for Sentiment Analysis by Means of Context Embeddings. AAAI 2018
- reference [12] is not proper for a scientific journal. Archival non-peer-reviewed paper has not been used as references.

2. ONTOLOGY CONSTRUCTION
Section 3 presents the ontology process.
This section should be re-organized in the following way:
- a first part where the seven phases of METHONTOLOGY are clearly described and the modeling process detailed;
- a second part where the main concepts of the ontology are presented;
- a third part describing how the ontology enables a reasoning process that is original with respect to the state of the art.

3. USEFULNESS
The evaluation part is quite weak.
First, the effectiveness of the ontology should be compared with other state of the art resources (e.g. the ones mentioned in Section 2) and the impact of reasoning activities on the ontology should be demonstrated.
Second, further datasets should be used. The SemEval one is quite small.
I invite the authors to validate the ontology on the DRANZIERA dataset:
- Mauro Dragoni, Andrea Tettamanzi, Célia da Costa Pereira: DRANZIERA: An Evaluation Protocol For Multi-Domain Opinion Mining. LREC 2016
After the comparison with other resources and the validation on wider datasets, a comparison of the HASO ontology with machine learning approaches could be presented.
Finally, a discussion on the error analysis should be reported.