My second collaboration with world-class beatboxer Reeps One, the Reeplicator, posed significant challenges. The stage was set at SXSW: with only one performance, there was no room for failure. The goal was to create an AI performer who could duet live with a human beatboxer. If it were any other instrument, one could digitize the perfomance, but a human voice required a neural network just to hear what Reeps was doing, segment and classify the sound. That would have been a difficult task, but was only half of it. A second AI had to understand musical concepts, learn from previous performances and different genres of music, and have several controllable behaviors: call and response, or duet, and finally battle mode, all of it piped through ableton to use banks made by the unstoppable Brooklyn audio studio, Antfood.
For this project, I had to devise the architecture to handle multiple layers of latency. I oversaw the creation of the recognition engine while training the response generation and test its improvisational capability, preparing to speak about the architecture and run the system live. In the end, the Reeplicator put up a good fight but didn’t stand a chance in live battle mode.
From the Mill website:
The Mill took its first foray into the world of Artificial Intelligence in 2018, debuting a newly developed AI with a live performance at SXSW. When AI Beatboxing and Mixed Reality Collide was a concert and talk by Mill ECD Rama Allen, Creative Developer Mike Manh and Champion Beatboxer Reeps One, that imagined the future of artistic collaboration with intelligent technologies.
In exploring how intelligent machines can produce new kinds of art, The Mill created The Reeplicator AI, the world's first live beatboxing artificial intelligence. On stage, the audience experienced a beatbox battle; man vs machine, Reeps One vs The Reeplicator.
Through this live experiment, The Mill demonstrated that artists who augment their work via intelligent technologies, have the power to create output that differs categorically from all other art created by humans through history.