Help Privacy Policy Disclaimer
  Advanced SearchBrowse


  PETaLS: Perception of Emotions in Text: a Linguistic Simulation

Volkova, E. (2011). PETaLS: Perception of Emotions in Text: a Linguistic Simulation. Talk presented at 21. Tagung der Computerlinguistik Studierenden (TaCoS 2011). Giessen, Germany. 2011-06-17.

Item is


show Files




Volkova, EP1, 2, Author           
1Department Human Perception, Cognition and Action, Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_1497797              
2Max Planck Institute for Biological Cybernetics, Max Planck Society, Spemannstrasse 38, 72076 Tübingen, DE, ou_1497794              


Free keywords: -
 Abstract: The emotional aspect is an integral part of human-human interaction. There are few spheres where emotional communication is unnecessary or not desirable. Feelings, opinions and attitudes are often expressed in text, which makes senti- ment analysis (SA) very needed and valuable. Most existing sentiment analysis systems are implemented for and used in specific predefined areas. The applica- tion field could be anything from extracting appraisal expressions (Whitelaw et al., 2005) to opinion mining of customer feedback (Lee et al., 2008).
PETaLS, the SA system we have recently developed, has its intended applica- tion in emotion enhancement of human-computer interaction, especially in virtual or augmented reality. The project is based on supervised machine learning and is meant to build a bridge between unprocessed input text and visual and auditory information, coming from the virtual character, like generated speech, facial ex- pressions and body language. This would enable a virtual character to simulate emotional perception and production of text in story telling scenarios.
Thus the first step towards building the PETaLS system was to collect a reli- able corpus of texts that are annotated for emotional states, which could later be used for training and testing. We conducted two annotation experiments: a pilot study in which a few selected texts were analyzed by several participants each and a large scale experiment where more than 70 texts were annotated by two participants each. The main goal of the first experiment, which was presented at the previous TaCoS conference (Volkova et al., 2010), is to research the nature of the sentiment analysis performed by humans and to examine whether they can reliably perform the task. The second experiment aimed at collecting of a corpus of reliable data of adequate size for ML training and testing.
For both experiments, we chose Brother Grimms fairy tales as the main anno- tation material. One of the challenges was to analyze inter-annotator agreement, which could not be accurately measured with the help of classical methods (Art- stein, 2008) due to the complexity of the annotation task. However, our more flexible approach to annotations comparison showed consistent similarity between annotators as thus proved the acceptable quality of the collected corpora: on av- erage 53 in the first experiment and 66 for the second.
We used a chain of NLP tools (Schmid, 1994, 1999; Klein Manning, 2002; Kunze, 2004; Hinrichs et al., 2009) to analyze the initial texts and to retrieve a large number of linguistic features which then were employed by the TiMBL (Daelemans et al., 2004) for training and testing. A few of the used features, to the best of our knowledge, had never been mentioned in the relevant literature, e.g., finding nouns that occur several times in a story, thus referring to characters and objects that are important for the plot and have a higher chance to trigger emotional states.
Classification was performed on short phrases where each classification item was represented by feature values describing the item itself and its closest neigh- bors to the left and to the right. We developed a hierarchy of emotional classes, which assisted deeper understanding of automatic classification problems and in- dicated where the improvements should be made.
The accuracy of our classifiers ranges from 47 to 81, depending on the classification task. Each time the major baseline is outperformed and our results are comparable with those of other researches. The most successful approach is when we only distinguish between three emotional categories: positive, negative and neutral, the accuracy for this classification task is 68, which is 1.7 higher than the corresponding baseline of 40.
Since each story can be told in various acceptable ways, we conducted a human test were we applied PETaLS-generated emotion annotations of several texts to the Virtual Story Teller Framework (Alexandrova et al., 2010). They were evaluated by ten participants along with virtual story telling session driven by human annotations. At the current stage of VSTF development, human and computer generated behavior scripts trigger comparable human evaluation responses.
We believe that PETaLS, in combination with VSTF, can become a useful tool for automatic animation of virtual characters and thus provide valuable insights in such fields as neuroscience, cognitive psychology and psychophysics.


 Dates: 2011-06
 Publication Status: Published online
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: BibTex Citekey: Volkova2011
 Degree: -


Title: 21. Tagung der Computerlinguistik Studierenden (TaCoS 2011)
Place of Event: Giessen, Germany
Start-/End Date: 2011-06-17
Invited: Yes

Legal Case


Project information