The Co-Explorer is a software agent that uses deep reinforcement learning to support synthesis exploration through human feedback.

It allows people to explore parametric design spaces by only communicating positive or negative preferences to the machine. A deep reinforcement learning algorithm is used to autonomously explore the space while interactively learning the user’s tastes, in an expressive workflow that we called co-exploration.

The Co-Explorer result from a user-centered design process of deep reinforcement learning led in close collaboration with expert sound designers. We first led a pilot study with an early reinforcement learning model prototype, called Sarsa, to understand what features would be important for sound designers leading parameter exploration. We then injected their feedback in the design of our final model prototype, based on Deep TAMER, which we evaluated in a creative workshop.

The current version of the software exists as a Python library; the OSC protocol supports connection between the deep reinforcement learning model (coded with the TensorFlow library) and parametric environments.

The Co-Explorer was also practiced within a research and creation project, which resulted in the ægo performance artwork.


The project was developed with Bavo Van Kerrebroeck, Baptiste Caramiaux, and Frédéric Bevilacqua in collaboration with the ISMM group of IRCAM and the ex)situ team of LRI (INRIA), in the context of the Sorbonne Université Doctorate in Computer Science.

Article at ACM TOCHI (2021)

Exhibition @ ISMIR 2018


Ce diaporama nécessite JavaScript.