MultiMediate: multimodal behaviour analysis for artificial mediation

  • The MultiMediate challenge is a multi-year endeavour to lay the foundations for socially capable artificial mediators that can support human interactions. In the its first years, MultiMediate has focused on solving basic social behaviour sensing tasks including eye contact detection, backchannel detection, and bodily behaviour recognition. More recently, the challenge focused on the development of algorithms that can estimate the degree by which a human is engaged in a social interaction - a complex phenomenon that is strongly influenced by cultural norms and the nature of the social situation. By providing a diverse set of training and testing datasets, MultiMediate has facilitated the development of generalisable engagement estimation approaches. In MultiMediate ’26, the diversity of training and evaluation data is enriched further by including the PInSoRo dataset of child-child and child-robot interactions annotated with both social and task engagement. As such, MultiMediate ’26The MultiMediate challenge is a multi-year endeavour to lay the foundations for socially capable artificial mediators that can support human interactions. In the its first years, MultiMediate has focused on solving basic social behaviour sensing tasks including eye contact detection, backchannel detection, and bodily behaviour recognition. More recently, the challenge focused on the development of algorithms that can estimate the degree by which a human is engaged in a social interaction - a complex phenomenon that is strongly influenced by cultural norms and the nature of the social situation. By providing a diverse set of training and testing datasets, MultiMediate has facilitated the development of generalisable engagement estimation approaches. In MultiMediate ’26, the diversity of training and evaluation data is enriched further by including the PInSoRo dataset of child-child and child-robot interactions annotated with both social and task engagement. As such, MultiMediate ’26 poses the challenging task of creating engagement estimation approaches that are able to transfer between different social situations, languages, age groups, and notions of engagement.…show moreshow less

Download full text files

Export metadata

Statistics

Number of document requests

Additional Services

Share in Twitter Search Google Scholar
Metadaten
Author:Philipp Müller, Marius FunkORCiDGND, Daksitha Withanage DonGND, Elisabeth AndréORCiDGND, Victor Oei, Andreas Bulling, Michal Balazia, Francois Bremond
Frontdoor URLhttps://opus.bibliothek.uni-augsburg.de/opus4/130096
URL:https://openreview.net/forum?id=LuwqLXLAIb
Parent Title (English):ACM Multimedia 2026: Multimedia Grand Challenge Proposal (ACMMM2026-MGC-Proposal), Rio de Janeiro, Brazil, 9 November 2026
Publisher:OpenReview
Type:Conference Proceeding
Language:English
Date of Publication (online):2026/05/04
Year of first Publication:2026
Publishing Institution:Universität Augsburg
Release Date:2026/05/04
Institutes:Fakultät für Angewandte Informatik
Fakultät für Angewandte Informatik / Institut für Informatik
Fakultät für Angewandte Informatik / Institut für Informatik / Lehrstuhl für Menschzentrierte Künstliche Intelligenz
Dewey Decimal Classification:0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik
Latest Publications (not yet published in print):Aktuelle Publikationen (noch nicht gedruckt erschienen)
Licence (German):CC-BY 4.0: Creative Commons: Namensnennung