This experimental study investigated human perception of machine behaviour in reinforcement learning applied to sonic exploration.
Reinforcement learning defines a formal framework for the interaction between a learning agent and an environment in terms of states, actions, and rewards. Our interest situated in interactive approaches to reinforcement learning, where the agent directly receives the reward signal from human feedback. Such an expressive workflow could allow humans to teach agents specific behaviours based on subjective preferences. We were interested to apply interactive reinforcement learning to creative tasks related to sound.
We first wanted to study how humans may perceive agent behaviour through sound listening during feedback-based teaching. We thus led a controlled experiment where we asked participants to guide three types of agents using only feedback, in two sound synthesis environment. Participants successfully interacted with these agents to reach a sonic goal in two cases of different complexities. Subjective evaluations suggest that the exploration path taken by agents, rather than the fact of reaching a goal, may be critical to how agents are perceived as collaborative.
The project was developed with Frédéric Bevilacqua and Baptiste Caramiaux in collaboration with the ISMM group of IRCAM and the ex)situ group of LRI (INRIA), in the context of the Sorbonne Université Doctorate in Computer Science.
Paper at SMC (2018)