Virtual Conductor was established to control a musical arrangement in real time much the same way as a conductor controls an orchestra. A musical piece is heard and the conductor has control over overall tempo, overall dynamics, as well as individual dynamic enhancement of the four instrument sections (horns, woodwinds and strings, percussion and the solo violin. This was done using a Wii remote was programmed through max/msp.
All the members of our group were involved in all the processes in order to perfect this project. We decided that we should all be in charge of one major aspect of then assignment.
Stephen: Head programmer, Nick: Head of documentation and research, Michael: Head of Audio and pictures.
We collated all our parts together to create an immersive and enjoyable experience for the user of our system.
RELATED AND INFLUENTIAL WORKS
Our idea was conceived because all members of the team had
musical backgrounds. The main works that influenced this idea was
Guitar hero @ guitarhero.com/, and Wii Music @
http://en.wikipedia.org/wiki/Wii_Music and on
e3.nintendo.com/wii/wiimusic/index.html.
Our project turned out quite similar to the Wii Music work, and this
Is exactly what we hoped for.
DESIGN MOTIVATION
Our intent was to provide the user with an immersive but enjoyable experience, they could use over and over again. —The main motivation for this project is that we all have musical back grounds and wanted to control music using gestures in a similar way to which a conductor controls an orchestra.
I personally felt that we successfully achieved this, given the positive reaction of users on the presentation night.
TECHNICAL METHOD
We wanted the user to simulate what a conductor of an orchestra does instinctually. For this we decided that the use of the wiimote and Nun-chuk expansion would be a suitable interface for these purposes. It works by the wiimote controlling the playback speed of pre-recorded orchestral tracks and the Nun-chuk attachment controlling the volume.
The aka.wiimote patch was used for mapping of the wiiremote
To execute this we used the IRCAM software supervp.play~ patch to control the playback speed changes, with limiting the minimum and maximum speeds reachable, while pitch and envelope correction maintained the original pitch and envelope. The nun-chuk joystick is used to increase the dynamics of one of four musical sections. The C and Z button controls the master volume of playback, up and down.
The simplicity in the interface is designed that a person with no musical back ground can come wave their hand to get a varying tempo and use there other hand to adjust volume levels. This I believe is achieved efficiently for the user as there is not many button presses involved for the user.
The interface is also defined that the two parameters that the user will control are laid out one in each hand.
The music was composed with usability of the interface in mind. Knowing that tempo and dynamics were our main controls, I composed the piece so that it builds in intensity and power from beginning to end. This enables the operator to instinctually feel, when something is to be louder or softer, or speed up. The song goes for three minutes and has 4 major sections to it. The 16 or 32 bar sections enable the user to easily manipulate the song, as it is easy to perceive an oncoming change even if you are not familiar with the song.The music was composed in my studio in Pro Tools. 90% of the sounds came from an orchestral sampler program called EWQLSO GOLD. This was used because of its lifelike and “Hollywood movie” typed sound qualities. This was this taken to be mixed using Izotope Ozone 2.
I also recorded an electric violin in the final dramatic section. I felt like the piece was perfect for our Virtual Conductor example due to its intensity and room for dynamic and tempo articulation. I received very positive feedback on this from almost all the people who came to use our system on the night.
USER EXPERIENCE AND USABILITY CONSIDERATIONS
The user will be a conductor in charge of directing the flow of a piece of music using two Wiimotes as batons.
One hand will be triggering musical sections (strings, horns, woodwind, brass), and the other hand will be manipulating variables such as sustain, dynamics and expression.
It was easy for users to understand and use our system but like anything else it was difficult to master at first attempt. This would make it a great game ( Wii music).
In our earlier reports we made it clear that an instinctual and easy system of mapping must be implemented for ease of use and practicality, I believe we achieved this.
Every person enjoyed using this device especially us, who mastered conducting the song. It was quite addictive and was not too embarrassing waving your hands around.
The user really got the feel of interacting as a conductor in our system. It should not be too embarrassing for a user to move their arms around, as it will be enjoyable to hear the variations in the audio samples in sympathy to the users motions.
One of the main difficulties the user faced is co-ordinating both hands simultaneously for the best possible result. This was exactly what we predicted, but this is another challenge that would make this a good starting basis for a computer game.
Our goal would be for a user to have an immersive and fun experience through manipulating a musical arrangement.
FINAL THOUGHTS
After seeing our final project on show at the presentation night we were quite pleased with the final result. All but one of our major ideas were implemented.
We initially wanted to have additional controls over the 4 sections, especially sustain and reverb. This would have greatly enhanced the experience but was unable to be implemented due to the system crashing. It was something to do with supervp patch lagging the system. In fact I even had to bounce down the individual musical sections as mono, because having 4 stereo tracks was overloading the system too.
To make this a commercial computer style game some sort of graphical representation of what was going on would be seen on the screen, this was definitely out of our league within the time given.
One criticism of our project is that there was only one song. In a game there would be at least a dozen or so songs. Once people played and mastered the song some asked if there was another they could play. Even though this was the case, for this instance it would have been impossible to produce many quality songs in the time given.
Resources
Mac Book Pro- Running of applications
Max 5 demo version – programming
Wiimote – Conductors baton Nun-
chuk attachment – dynamic adjustment
Khalil Sound Production Studio – Various equipment for music production
References
www.cycling74.com
http://en.wikipedia.org/wiki/Wii_Music
www.e3.nintendo.com/wii/wiimusic/index.html.
www.iamas.ac.jp/~aka/maxwww.ircam.fr
1 comments:
Hi Michael.
I didn't know you could play the violin! I would be interested to hear your thoughts on the East West orchestral library...I should've listened closer on the night.Cheers. Nigel.
Post a Comment