Mixing messages: experimental tools for remixing human speech

A recurring personal hobby has been mining human speech for music. In 2016, I tried to convert spoken poetry into sheet music. In 2020, I created a tool for remixing sounds from the Library of Congress into hip hop music, including spoken word recordings from oral histories, interviews, and radio.

Over the past couple years, I leaned into this interest a bit more. I created an experimental tool called DJ Phonetic for making beats with the kicks, snares, and hi-hats that are hidden within historical recordings of human speech.

With this tool, I allude to beatboxing, the art mimicking drum sounds with your mouth, lips, tongue, and voice. If you break down human speech, you can easily find drum sounds: the /b/ in "ball", /k/ in "kite", /t/ in "tap" works well as a kick, snare, and hi-hat respectively. You can read a little more about this process here and as always, the tool is open source.

I also created Mixed Messages, an experimental tool for manipulating and playing with audio recordings of spoken phrases, mostly from speeches or poetry recordings. With this tool, users can simply cut, move, duplicate, shrink, or stretch spoken words in real-time. It's almost like MS Paint for recorded spoken word. The results may vary, but you can occasionally find some catchy loops, hidden melodies, or just create a dense wall of sound. The code is also open source.

There is an untapped ocean of spoken word recordings that are just waiting to be mined for musical elements. Spoken word recordings such as speeches, interviews, and oral histories have the added benefit of being less copyright restrictive than music recordings, mostly due to the complex nature of music copyright and sampling (PDF). There's a lot more creative research to be done here and this is a topic I'll definitely keep returning to.