22
edits
no edit summary
No edit summary |
No edit summary |
||
| Line 19: | Line 19: | ||
CONCLUSION: the software was trained by dog-persons. | CONCLUSION: the software was trained by dog-persons. | ||
3/10. | 3/10. | ||
==The Julian Mosbach String Quartet== | |||
I can't play the cello. But in this exercise I used computers to make it sound even worse. | |||
Ever since I got into classical music, I found Bach's "The Art Of The Fugue" to be just the most beguiling set of pieces out there. I always wanted to play it. However I only know how to play the piano and while there is a version for piano (which sadly is incredibly difficult to play and thus even the most skilled pianists play it way too slow - not for me) the piece just shines played by a string quartet. I, however can't play any string instruments, not do I own any. But my sister does. She used to play the cello when she was younger and showed me how to produce notes. So I sat down with that half-scale childrens cello and recorded every note in chromatical order for our algorithm to make me a cello quartet. | |||
And this is where the trouble begins. I did not record to a metronome, but instead thought I could just record long notes, which PD could then chop up. It does not. It finds the note that is closest in length and plays that in its entirety. (by the way: the video of me playing the cello is eleven minutes long, which means I sat through hours of just listening to it again and again then the sorted notes and that for every instrument to find that mistake). So I needed to give the algorithm perfect note lengths. I did that by putting the original audio in a DAW based on 115BPM (the pieces tempo) and cut each note in semi, quarter and eigth notes. I got very lucky here because the piece mostly consists of these. I ran the analysis again, to find that the detection of the notes themselves is very challenging, since the cello has so many overtones that it just confuses sigmund. At first I tried to EQ the first and second overtones (octave and fifth above), which again is 148 notes that each get their own treatment. But that again didn't work out. So in a last final-straw attempt I took a synth with a simple sine wave and manually reproduced the original audio with quantized notes. 148 midi notes placed by hand. Needless to say decisions were regretted at this point. BUT THEN. IT WORKED. | |||
Kind of. Each instrument sounded roughly how it was supposed to, but when I put them together nothing worked. This was because the piece contained linked and dotted notes. And the algorithm looked for absolute lengths. So instead of putting even more notes in the original audio, I edited the midi files to remove any links and dots. So a dotted quarter note became a quarter and then an eigth of the same pitch. Arguably at this point its not Bach anymore, its Mosbach-Bach (say it fast). Now, finally after all this editing it was ready to work. I was emotionally wrecked but excited to see what PD had cooked for me. And something happened that I didn't expect. For some reason, the algorithm had pitched each instrument 3 semitones down. Except for the cello-voice. This took me hours to find, because I checked the cellos pitch again and again and it was correct. | |||
But then finally, after pitching the cello down to match the other instruments, I was ready to press play... | |||
- just to find out, that our algorithm does not take into account pauses. So whenever there should be a pause, it simply played the next note without hesitation. At this point I am wondering whether I should have simply cut everything by hand. That might have saved me A LOT of time. | |||
But I kept on working and edited each pause by hand, using the other instruments as reference. | |||
{{#ev:youtube|QD62-dQRCHY|740|left}} | |||
Its sounds...accetable? Maybe? There is one very prominent issue still prevalent which I wasn't able to solve. The instruments alone work perfectly, but played together, they keep going out of sync. I think there is two reasons for that: Firstly any error in detection leads to a slightly changed note length which multiplies when more notes are played. The violin 1- voice for example has the most eigth notes and seems to always drag behind the others (constant error multiplied). And secondly I think that sigmund detects high notes faster than low notes, which makes sense physically, because it should need at least one full cycle of the wave before detecting a note. And that just takes twice as much time per octave you go down. | |||
To give that combined error a number: the pieces original tempo was 115 BPM but due to lag between the notes and inconsistent detection the PD-Version is just 108BPM. In this case, manually editing the pauses really saved a lot, because it meant that the instruments were always synced up again every 30 seconds or so. | |||
This might have been it, but I decided to take one last step. I am a firm believer in "the product", meaning that a "good" process in my eyes shouldn't make up for a bad product. Or in other words: nobody is going to applaud your for putting lots of effort in something if the end product is bad. | |||
So, as this is mostly a video editing class, I took the original sound files again and cut it up to build a Kontakt-Sample-Instrument. (If you don't know what that means: Sampling takes a bunch of notes and maps them to midi notes, which gives you the ability to play something using the notes you recorded in real life). | |||
I completed it with a legato-script, reverb and a binaural mix for complete immersion. | |||
After that I took the same video and just married them together: | |||
{{#ev:youtube|fafbMK1yUtU|740|left}} | |||