Using machine learning to generate engaging behaviours in immersive virtual environments
- Our work aims at implementing autonomous agents for Immersive Virtual Reality (IVR). With the advances in IVR environments, users can be more engaged and respond realistically to the events delivered in IVR, a state described in literature as presence. Agents with engaging verbal and nonverbal behaviour help preserve the sense of presence in IVR. For instance, gaze behaviour plays an important role, having monitoring and communicative functions. The initial step is to look at a machine learning model that generates flexible and contextual gaze behaviour and takes into account the rapport between the user and the agent. In this paper, we present our progress to date on the problem of creating realistic nonverbal behaviour. This includes analysing a multimodal dyad data, creating a data-processing pipeline, implementing a Hidden Markov Model and linking the Python scripts with the VR game engine (Unity3D). Future work consists of using richer data for more complex machine learningOur work aims at implementing autonomous agents for Immersive Virtual Reality (IVR). With the advances in IVR environments, users can be more engaged and respond realistically to the events delivered in IVR, a state described in literature as presence. Agents with engaging verbal and nonverbal behaviour help preserve the sense of presence in IVR. For instance, gaze behaviour plays an important role, having monitoring and communicative functions. The initial step is to look at a machine learning model that generates flexible and contextual gaze behaviour and takes into account the rapport between the user and the agent. In this paper, we present our progress to date on the problem of creating realistic nonverbal behaviour. This includes analysing a multimodal dyad data, creating a data-processing pipeline, implementing a Hidden Markov Model and linking the Python scripts with the VR game engine (Unity3D). Future work consists of using richer data for more complex machine learning models, with a final aim of integrating the gaze model (plus future nonverbal behaviour models) into an autonomous virtual character framework.…
Author: | Georgiana Cristina DobreGND |
---|---|
Frontdoor URL | https://opus.bibliothek.uni-augsburg.de/opus4/112657 |
ISBN: | 978-1-7281-3892-3OPAC |
Parent Title (English): | 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 3-6 September 2019, Cambridge, UK |
Publisher: | IEEE |
Place of publication: | Piscataway, NJ |
Type: | Conference Proceeding |
Language: | English |
Year of first Publication: | 2019 |
Release Date: | 2024/04/23 |
First Page: | 50 |
Last Page: | 54 |
DOI: | https://doi.org/10.1109/aciiw.2019.8925113 |
Institutes: | Fakultät für Angewandte Informatik |
Fakultät für Angewandte Informatik / Institut für Informatik | |
Fakultät für Angewandte Informatik / Institut für Informatik / Lehrstuhl für Menschzentrierte Künstliche Intelligenz | |
Dewey Decimal Classification: | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 000 Informatik, Informationswissenschaft, allgemeine Werke |