Giving robots a voice: human-in-the-loop voice creation and open-ended labeling

  • Speech is a natural interface for humans to interact with robots. Yet, aligning a robot’s voice to its appearance is challenging due to the rich vocabulary of both modalities. Previous research has explored a few labels to describe robots and tested them on a limited number of robots and existing voices. Here, we develop a robot-voice creation tool followed by large-scale behavioral human experiments (N=2,505). First, participants collectively tune robotic voices to match 175 robot images using an adaptive human-in-the-loop pipeline. Then, participants describe their impression of the robot or their matched voice using another human-in-the-loop paradigm for open-ended labeling. The elicited taxonomy is then used to rate robot attributes and to predict the best voice for an unseen robot. We offer a web interface to aid engineers in customizing robot voices, demonstrating the synergy between cognitive science and machine learning for engineering tools.

Download full text files

Export metadata


Number of document requests

Additional Services

Share in Twitter Search Google Scholar
Author:Pol van RijnORCiD, Silvan MertesORCiDGND, Kathrin JanowskiORCiDGND, Katharina WeitzORCiDGND, Nori JacobyORCiD, Elisabeth AndréORCiDGND
Frontdoor URL
Parent Title (English):CHI '24: proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, May 11-16, 2024
Publisher:Association for Computing Machinery (ACM)
Place of publication:New York, NY
Editor:Florian Floyd Mueller, Penny Kyburz, Julie R. Williamson, Corina Sas, Max L. Wilson, Phoebe Toups Dugas, Irina Shklovski
Type:Conference Proceeding
Year of first Publication:2024
Publishing Institution:Universität Augsburg
Release Date:2024/05/13
Tag:Roboter; Sprachsynthese; Gibbs-sampling; Personalisierung; Crowdsourcing
Crowdsourcing; Personalization; Robot; Text/Speech/Language
First Page:584
Institutes:Fakultät für Angewandte Informatik
Fakultät für Angewandte Informatik / Institut für Informatik
Fakultät für Angewandte Informatik / Institut für Informatik / Lehrstuhl für Menschzentrierte Künstliche Intelligenz
Dewey Decimal Classification:0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik
Licence (German):CC-BY 4.0: Creative Commons: Namensnennung (mit Print on Demand)