My Master’s thesis investigated how people embody sounds—using gestures and vocalizations—to communicate on them.
People often rely on gestures to communicate on sounds, often combining them with vocalizations to express feelings or features about them. Drawing from speech studies, we were interested in studying the potential links between gestures and vocalizations in sound communication.
We first led qualitative annotation of an audio-visual database of gestural and vocal imitations of sounds to establish a set of hypotheses. We then led a controlled experiment and performed quantitative analysis of motion and sound data harvested.
Our results suggests that people embody sounds using gestures that metaphorically express one salient feature of sound, along with vocalizations that attempt to reproduce all sound features as faithfully as humanly possible.
Year
2015
Credits
The project was developed with Guillaume Lemaitre, Frédéric Bevilacqua, Patrick Susini, and Jules Françoise in collaboration with the PDS and ISMM groups of IRCAM, in the context of the European research project SkAT-VG, and the Master in Engineering of Sorbonne Université.
Publications
Article in PLOS One (2017)
Presentation at ISGS (2016)
Poster at JASA (2015)
Master’s thesis (2015)