I spent some months doing a residency in Rio at the Museum of Tomorrow. I carried out different experiments in the space of music and artificial intelligence. I also held a few workshops around artificial intelligence techniques and their intersection with art for artists and technologists of Rio.
Samba Music
Working in a museum in Rio, i focused on creating Samba music using AI. Within Samba, there are lots of variations, and i ended up focusing on creating Samba Marchinha music- a type of music popular in the Brazil Carnival.
Before getting into details- here’s a couple of clips of Samba Marchinhas created using AI:
[soundcloud url=”https://api.soundcloud.com/tracks/301572856″ params=”auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=true” width=”100%” height=”150″ iframe=”true” /]
I worked with a drummer- Carlos, who created various drum tracks with different variations of Marchinhas which i then used to train an RNN on following the scheme of accounting for different instruments within the drum track.
Interactive Humming Melody generation:
I am always interested in thinking of how AI can be a creative partner to explore new ideas, and a tool i worked on with this in mind was the Humm-RNN. You humm a melody you have in mind to a computer and the computer continues the melody, responding to your input.
For this, i wrote a small PD (pure-data) patch that takes mic input and converts the humm to a midi-note which is played through a virtual synth. I used Google Magenta’s interactive MIDI code to then let a computer respond to the humming melody.
Here’s a short clip with some experimental (test) humming:
[soundcloud url=”https://api.soundcloud.com/tracks/301570078″ params=”auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=true” width=”100%” height=”150″ iframe=”true” /]
Would be fun to whistle into the system as well, with someone who whistles well!
Another aspect i’ll probably explore with this is to chart out a real-time score with the human and computer outputs to aid in further editing of the melody.
Samba Marchinhas Lyrics:
I also scraped this website for various Samba lyrics and ran an RNN script on it to generate lyrics of a Samba Marchinha. Léo, a musician, performed the Lyrics on RNN generated drums at the end of the residency.
Here’s the lyrics that were performed:
E o meu coração de cantar
A luz do céu é o meu carnaval
Se a cara de cantar a vida da minha paixão
O passarela de coração
A mais de ser contar
A linda a realidade na fantasia
E a casa de amor
Eu vou esta estrela em faz a beleza
E o meu coração de amor
O coração de amor
No povo de paz de minha roda
Se sol e cantar
A vida de amor de carinhos
E a vida pra conta de fazer a canção
Eu sou o carnaval
Carnaval
TSNE City Sounds with Visuals:
One idea that i had that excited me quite a bit was to capture video (with audio) footage around the city and perform a TSNE on that (based on audio features) and then use that as a means to perform music from city sounds while also associating visuals to the sound- knowing where this sound is coming from. I believe different neighborhoods, major places of the city will have their own unique sounds and associating the city visual with that could be interesting. Doing this then across multiple cities could get very interesting! Didn’t get time to delve deep into this, but would keep it as an idea for further exploration!
Here’s a short clip of what i was getting at:
Apart from these, i did a TSNE of various Samba songs at the beginning after splitting them into 2 second clips. Audio TSNE gives you a great way to familiarize yourself with the different sounds of a particular kind of music, if you have little idea about it! here it is:
i did Audio TSNE work based off of Gene Kogan’s ML4A audio TSNE example.