From the Mill Website:
See Sound is a generative artwork experience that creates sound sculptures based on the human voice. Each user has access to a microphone and hardware interface, and can trigger different visual behaviors within the music visualizer based on experimentation, meaning each user leaves with their own bespoke sound sculpture.
This was the first collaboration I did with world-class beatboxer Reeps One. Initially I proved the concepts of taking audio and storing them so a shader could deform it. Then I collaborated with Houdini magician (see what I did there) Jimmy Gass to bring his visual mastery to its materiality and interaction. On top of that, I had to wrangle the latency of audio to allow a performer (or an attendee), to use it as both as a sound looper and a multistage sculpting tool.
Turning audio into a digital sculpture sitting on a GPU was quite the trick, but the original brief challenged us to make a real sculpture as a stretch goal. This involves getting the mesh off the GPU after it had been modified by a shader. We went through the whole process without knowing if this was possible but we ended up solving it in the end. Unfortunately our 3D printer wasn’t quite up to the task, as you can see.
If you’re hungry for more see sound sculptures, you should check out the gallery (click here).