Bot Dylan: Singer-songwriter robot called Shimon can write its own lyrics and has an album out
View(s):
Developed by researchers from the Georgia Tech Center for Music Technology, the robot collaborates with human musicians and even has an album out in the spring
Shimon the singing songwriting robot has been taught to write his own lyrics by studying tens of thousands of songs written by the musical greats.
Developed by researchers from the Georgia Tech Center for Music Technology, the robot collaborates with human musicians and even has an album out in the spring.
The robot was given a dataset of 50,000 lyrics covering all genres including rock, hip-hop, jazz and progressive as part of its song writing education.
As well as writing the lyrics the robot can sing them and dance while performing with ‘his band’ made up of Georgia Tech students and researchers.
Professor Gil Weinberg, creator of Schimon said he works with humans to create music, they are a mixture of songs made by human and robot together.
‘I suggest a theme, he writes lyrics and gives me a melody that I compose into a song – we then perform it together with other humans while he sings,’ he was quoted as saying.
The process starts when Weinberg gives Schimon a theme – such as space – and the robot will then go away and come up with a set of ‘coherent lyrics’ that flow together.
‘It learns the correlation between the thousands of lyrics it has studied and can then go into the dataset to generate new lyrics that fit melodies,’ said Weinberg.
‘I always wanted to write songs, but I just can’t write lyrics. I’m a jazz player,’ Weinberg said. ‘This is the first time that I actually wrote a song, because I had inspiration: I had Shimon writing lyrics for me.’
The lyrics created by Schimon include the lines ‘there may be music on a star into your mind’ written for the song Into Your Mind, the first release from the album.
PhD student Richard Savery who worked on the robot plays base in the band with Schimon, says it is an unusual but rewarding experience playing with it.
‘It is great interacting with him, I play base and it feels like another musician on stage, there is lots of dialogue between me and him,’ he said.
‘There are ups and downs in the relationship, sometimes it is ‘it’s really annoying with that guy’ and other days it is the greatest collaboration.’
As well as having him study the lyrical datasets they use semantic knowledge – that is where it can identify one word and link it to other matching words.
‘Shimon will knows the word storm is linked to rain as will generate lyrics linked to rain such as sunshine,’ said Savery. ‘The whole lyric system is finding words that are interconnected.’
When Shimon sings the songs he really does sing, said Weinberg.
He has a unique voice created by collaborators at Pompeu Fabra University in Barcelona and involved creating an AI voice trained on hundreds of other songs.
Along with his new musical skills Shimon has some new hardware that improves its movement and ability to play the marimba.
As well as having him study the lyrical datasets they use semantic knowledge – that is where it can identify one word and link it to other matching words.
‘Shimon will knows the word storm is linked to rain as will generate lyrics linked to rain such as sunshine,’ said Savery. ‘The whole lyric system is finding words that are interconnected.’
When Shimon sings the songs he really does sing, said Weinberg.
He has a unique voice created by collaborators at Pompeu Fabra University in Barcelona and involved creating an AI voice trained on hundreds of other songs.
Along with his new musical skills Shimon has some new hardware that improves its movement and ability to play the marimba.
He is still mostly stationary, but he has a mouth, new eyebrows, and new head movements designed to help convey emotion and interact with his bandmates.
‘Shimon plays much faster — about 25 to 30 hertz at the maximum — and also much more expressively, playing from a soft dynamic range to a strong dynamic range,’ said PhD. student Ning Yang, who designed the all-new motors and hardware.
‘That also allows him to do choreography during the music being played,’ he said.
Yang worked closely with fellow PhD. student Lisa Zahray, who created a new suite of gestures for the robot — including how he uses those new eyebrows.
We have to think about his role at each time during the song and what he should be doing,’ Zahray said.
‘We also want to make sure he’s interacting with the other musicians around him to give that feel that he’s performing with people.’
Shimon can count in at the beginning of songs to cue the band, and sometimes he’ll wave his mallets around in time to the music.
New brushless DC motors mean he has a much greater range of motion and control of that motion.
Yang accomplished that by bringing his engineering knowledge and musical background together to create human-inspired gestures.
A big part of what we are trying to do is to take the research and bring it to the people,’ Weinberg said.
‘I want the whole experience, it is just like a human, it is interesting but I think we want the audience to just sit back and say ‘I like the music’.
‘This is music a human themselves might not have written, it is unique to the robot.’
Th partnership with people is key for Weinberg, who said teaching Shimon new skills isn’t about replacing musicians but expanding music.
“We will need musicians, and there will be more musicians that will be able to do more and new music because robots will help them, will generate ideas, will help them broaden the way they think about music and play music,” Weinberg said.
Shimon, Weinberg, and the band are building a touring schedule now with the goal of taking their unique blend of robot and human-created music to more people.
Weinberg said he hopes those shows will prove to be more than a novelty act
© Daily Mail