Refine
Document Type
- Article (3)
- Conference Proceeding (2)
Keywords
- biodiversity (1)
- data science (1)
- earth observation (1)
- ecosystem integrity (1)
- emotion recognition (1)
- facial coding (1)
- functions (1)
- genesis (1)
- geodivesity (1)
- geomorphic traits (1)
Institute
- Fakultät für Angewandte Informatik (5) (remove)
Background
Emotions play a key role in psychotherapy. However, a problem with examining emotional states via self-report questionnaires is that the assessment usually takes place after the actual emotion has been experienced which might lead to biases and continuous human ratings are time and cost intensive. Using the AI-based software package Non-Verbal Behavior Analyzer (NOVA), video-based emotion recognition of arousal and valence can be applied in naturalistic psychotherapeutic settings. In this study, four emotion recognition models (ERM) each based on specific feature sets (facial: OpenFace, OpenFace-Aureg; body: OpenPose-Activation, OpenPose-Energy) were developed and compared in their ability to predict arousal and valence scores correlated to PANAS emotion scores and processes of change (interpersonal experience, coping experience, affective experience) as well as symptoms (depression and anxiety in HSCL-11).
Materials and methods
A total of 183 patient therapy videos were divided into a training sample (55 patients), a test sample (50 patients), and a holdout sample (78 patients). The best ERM was selected for further analyses. Then, ERM based arousal and valence scores were correlated with patient and therapist estimates of emotions and processes of change. Furthermore, using regression models arousal and valence were examined as predictors of symptom severity in depression and anxiety.
Results
The ERM based on OpenFace produced the best agreement to the human coder rating. Arousal and valence correlated significantly with therapists’ ratings of sadness, shame, anxiety, and relaxation, but not with the patient ratings of their own emotions. Furthermore, a significant negative correlation indicates that negative valence was associated with higher affective experience. Negative valence was found to significantly predict higher anxiety but not depression scores.
Conclusion
This study shows that emotion recognition with NOVA can be used to generate ERMs associated with patient emotions, affective experiences and symptoms. Nevertheless, limitations were obvious. It seems necessary to improve the ERMs using larger databases of sessions and the validity of ERMs needs to be further investigated in different samples and different applications. Furthermore, future research should take ERMs to identify emotional synchrony between patient and therapists into account.
Remote sensing (RS) enables a cost-effective, extensive, continuous and standardized monitoring of traits and trait variations of geomorphology and its processes, from the local to the continental scale. To implement and better understand RS techniques and the spectral indicators derived from them in the monitoring of geomorphology, this paper presents a new perspective for the definition and recording of five characteristics of geomorphodiversity with RS, namely: geomorphic genesis diversity, geomorphic trait diversity, geomorphic structural diversity, geomorphic taxonomic diversity, and geomorphic functional diversity. In this respect, geomorphic trait diversity is the cornerstone and is essential for recording the other four characteristics using RS technologies. All five characteristics are discussed in detail in this paper and reinforced with numerous examples from various RS technologies. Methods for classifying the five characteristics of geomorphodiversity using RS, as well as the constraints of monitoring the diversity of geomorphology using RS, are discussed. RS-aided techniques that can be used for monitoring geomorphodiversity in regimes with changing land-use intensity are presented. Further, new approaches of geomorphic traits that enable the monitoring of geomorphodiversity through the valorisation of RS data from multiple missions are discussed as well as the ecosystem integrity approach. Likewise, the approach of monitoring the five characteristics of geomorphodiversity recording with RS is discussed, as are existing approaches for recording spectral geomorhic traits/ trait variation approach and indicators, along with approaches for assessing geomorphodiversity. It is shown that there is no comparable approach with which to define and record the five characteristics of geomorphodiversity using only RS data in the literature. Finally, the importance of the digitization process and the use of data science for research in the field of geomorphology in the 21st century is elucidated and discussed.
NOVA: a tool for explanatory multimodal behavior analysis and its application to psychotherapy
(2020)
In emergency medicine, timely intervention for patients at risk of suicide is often hindered by delayed access to specialised psychiatric care. To bridge this gap, we introduce a speech-based approach for automatic suicide risk assessment. Our study involves a novel dataset comprising speech recordings of 20 patients who read neutral texts. We extract four speech representations encompassing interpretable and deep features. Further, we explore the impact of gender-based modelling and phrase-level normalisation. By applying gender-exclusive modelling, features extracted from an emotion fine-tuned wav2vec2.0 model can be utilised to discriminate high- from low suicide risk with a balanced accuracy of 81%. Finally, our analysis reveals a discrepancy in the relationship of speech characteristics and suicide risk between female and male subjects. For men in our dataset, suicide risk increases together with agitation while voice characteristics of female subjects point the other way.
