The Co-Explorer is a software agent that uses deep reinforcement learning to support synthesis exploration through human feedback.
It allows people to explore parametric design spaces by only communicating positive or negative preferences to the machine. A deep reinforcement learning algorithm is used to autonomously explore the space while interactively learning the user’s tastes, in an expressive workflow that we called co-exploration.
Co-Explorers result from a user-centered design process of deep reinforcement learning led in close collaboration with expert sound designers. We first led a pilot study with an early reinforcement learning model prototype, called Sarsa, to understand what features would be important for sound designers leading parameter exploration. We then injected their feedback in the design of our final model prototype, based on Deep TAMER, which we evaluated in a creative workshop. Co-Explorers were also practiced within a research and creation project, which resulted in the creation of the ægo artwork.
The current version of the software implements different user controls over agent’s exploration, along with a history that maps the space with user preferences. It is coded in Python; the OSC protocol supports connection between the deep reinforcement learning model (coded with the TensorFlow library) and parametric environments.
The project was developed with Bavo Van Kerrebroeck, Baptiste Caramiaux, and Frédéric Bevilacqua in collaboration with the ISMM group of IRCAM and the ex)situ team of LRI (INRIA), in the context of the Sorbonne Université Doctorate in Computer Science.
arXiv preprint (2019)
Available on GitHub