Rubik's Studio

Piers Gilbertson & Dan Gallard

Brief overview & description

The aim of The Rubik’s Studio was to create an original audio interface from an existing product. The Rubik’s Cube seemed a perfect candidate, as there is a lot of scope for complexity within a relatively simple idea.

Each coloured face of the Cube represents an original audio composition in its entirety. Each tile of the cube represents a segment, or sample, from one of these audio files. As the cube is mixed up, these samples align to produce new and relatively limitless variations, (somewhere in the region of 48,000,000), of the original samples. These can be mixed and reordered ‘on-the –fly’ for live performance or to generate unique sounds/ combinations to be added to different instruments/media etc as a compositional element.

The real strength of the Rubik’s Studio comes from its versatility. It is very easy to make each colour represent different audio samples, giving it limitless applications in terms of composition, performance, sound effects, and the creation of completely original sounds based on the interaction of its tiles.

To hear the original samples the user must solve each face of the cube – giving the user an intellectual challenge and an element of mystery to further engage the user.

Design motivation

The idea behind the Rubik’s Studio – to make a Rubik’s Cube into an interactive sample generator with the help of reacTIVision fiducial markers and software – remained relatively unchanged throughout the project. The methodology and intended approach to the project was modified however – based on advice from Sam and Kirsty - in terms of simplification and specification, following our initial presentation.

Due to time considerations, we decided to focus on only one cube as opposed to the two in our original brief. The second cube was to be an effect cube, adjusting the parameters of the first cube in terms of reverberation, EQ, delay etc.

The second modification was again due to time constraints. Our original idea was to work collaboratively on each part of the process together. Upon beginning the project we decided that Dan (due to his stellar marks in Digital Audio last semester) would be in charge of programming the necessary code, whilst Piers (having seen Home Improvement once), would be in responsible for designing/building the interface and composing/recording the audio material.

Technical method

The Cube:

There were several considerations and goals in making the Rubik’s Studio Cube. We wanted each face to be coloured as per the original Rubik’s Cube for user familiarity; to maintain the original idea behind the Rubik’s Cube; and to trigger the audio sample each colour represents in its original form.

Furthermore, we then assigned a ReacTIVision fiducial marker to each individual tile of the Rubik’s cube. These tiles represent a 1/8^th section (sample) of the audio file that colour represents.

These individual tiles are read and passed to the ReacTIVision software via a live video feed. The software turns the incoming video signal into monochrome picture and from there identifies and tracks each marker.

Because of this, we had to determine, by trial and error, what colours we could use so that the software would still recognise the fiducials. We created a template in Adobe Illustrator based on the tile parameters and tested numerous color combinations in terms of aesthetics, opacity and contrast to black and white. Figure 1 shows the final colours we used. We then printed the tiles onto stickers, which were cut out and used to replace the originals.

Figure 1: Fiducial marker stickers for the Rubik’s cube.

Picture 1: Modified Rubik's cube with Reactivision fiducial markers. [6]

Picture 2: Fiducial markers being read by thr ReactiVision software. [6]

The Podium:

There were numerous factors to be considered in designing and building the podium:

Structural integrity: As there was expensive equipment to be contained within the podium it had to be sturdy. It also reduces your credibility if your interface collapses mid-exhibition. To this end the base of the podium was constructed out of very dense, heavy, though rather unattractive wood.

The Tardis: The podium had to be big enough to house a camera and tripod, 3 lights, an audio interface and potentially a laptop whilst still being small enough to be easily portable.

Picture 3: The Podium open showing DV camera and lighting. [6]

Internal accessibility and ventilation: We had to be able to easily access the lighting and camera equipment within the podium to fine-tune them. To this end the sides were not boarded up. This feature also solved the problem of ventilation. The lighting created quite a lot of heat, and hence a potential fire hazard, so we needed to have good consistent airflow.

Aesthetically pleasing: We wanted the interface to appear very simple, concealing the complexity of its internal workings.

The table-top frame: was made from Australian Cedar Pine, a very attractive and sturdy Native tree. We wanted it to look great in order to attract users.

Picture 4: The Podium table-top from above with the cube. [7]

The table-top: was made from 2 layers of plastic. The first layer was clear plastic so that the camera could read the fiducials through it. We tested with frosted plastic but settled on clear plastic to avoid any technical problems. On top of the clear pane was a translucent blue pane with a rectangle laser cut into the centre to define the cameras viewing parameter. This blue caused the whole interface to glow when lit from beneath making it look ‘futuristic’ and concealing the camera beneath. We used tracing paper over between the panes to further conceal the inner workings of the podium.

The curtain: Matt Storey kindly donated some dense studio curtaining which we used to surround the podium, hiding its inner workings whilst still maintaining accessibility and ventilation.

Inexpensive: the whole interface (excluding the equipment below) cost less than $60.

We would like to thank the wonderful USYD Architecture workshop team for their generosity with their time and expertise, and Matt Storey for donating the curtain.

Equipment:

The following is a list of the technical elements used to create this interface:

• MacBook laptop


• Digital video camera (640x480, 25fps) and tripod (connected via firewire)


• 3 lamps (a 12" mini-fluroescent, a 10W compact fluroescent and a 75W incandescent)


• USB audio / midi interface – in this case an Edirol UA-25


• 2 x Yamaha MSP active studio monitors

Audio:

Having decided that the Rubik’s Studio would function initially as a sequencer – i.e. that it would play portions of audio samples in a given sequence – we set our compositional parameters to 120bpm 4/4 timing. This was merely to improve the chances of creating audio that would ‘fit’ into a sequencer and still sound good.

We layered 6 different audio tracks (1 for each colour face), which combined to create a complex rhythm or melody. These tracks are only heard in their original form when a whole colour side is presented.

We ended up with 6 different cube ‘versions’ (made up of 6 audio tracks each – 36 in total), which we could switch between with the click of a button:

• Drunk Percussionist – Mainly syncopated percussion elements


• Hippy bells – as above


• Jungle Drums – as above


• “Thank you for coming and have a pleasant evening” – vocoder messages that can only be heard when a colour is deciphered


• “The Amazing Rubik’s Studio, insert advertisement here”


• SFX cube – 6 different sound effects

Audio was produced in Logic Pro using Stylus RMX and Sampletank 2.2.

Max/MSP Programming:

Figure 2 shows several audio sample presets we had which loaded a different 4-second loop into a buffer for each side of the cube. This was done through the loadSamples subpatch shown in Figure 3.

Figure 2: Various Sample Presets.

Figure 3: loadSamples subpatch.

Figure 4 shows the patch used to take incoming messages from Reactivision. The TuioClient object is supplied with the Reactivision software for integration into Max/MSP. This object outputs the following TUIO messages:

• addObject session_id fiducial_id


• updateObject session_id fiducial_id xpos ypos angle xspeed yspeed rspeed maccel raccel


• removeObject session_id fiducial_id


• addCursor session_id


• updateCursor session_id xpos ypos xspeed yspeed maccel


• removeCursor session_id

Figure 4: Reactivision TuioClient.

Figure 5: Rubik's cube mapping

The only TUIO messages we required were the addObject fiducial_id and the removeObject fiducial_id. The route object was used to split the addObject and removeObject messages, and then the zl slice object was used to remove the session_id message, as this was not required. The remove message was then fed through a pipe object to delay it by 500ms so that the cube could be rotated and the audio would overlap. The add messages were then funneled into the Rubik’s cube mapping patch as shown in Figure 5. The select object then chose according to fiducial_id messages and sent a bang to a toggle, then a 1 to the associated send object. When the fiducial markers were removed, another bang would be sent to the relevant toggle (following the 500ms delay) and a 0 to the associated send. The send’s are patched via identically named receives located in the playBuffer patch (Figure 9).

Figure 6: Sequencer.

The sequencer uses the metro object to regulate the playback speed and then uses a counter object to step through. This information is sent into the play_samples subpatch (Figure 7) and then to the six playFace objects.

Figure 7: play_samples subpatch. [5]

The playFace objects control which part of the four second buffer is played and when. If all red squares are placed down on the podium, the whole four second loop stored in the red buffer would be played back. When the cube has been mixed up, the corresponding parts of the audio in the different buffers gets played allowing a mixing of the six audio samples. The play~ object plays the buffer contents according to the information received from the playFace patch. The minimum object automatically mixes the audio signals to prevent any digital distortion and clipping.

Figure 8: playFace patch. [5]

Figure 9: playBuffer patch. [5]

The playFace patch (Figure 8) takes one input argument to set which face of the cube it is associated to.

The playBuffer patch (Figure 9) has four input arguments (seen in Figure 8). The first two arguments set the receive object to the correct corresponding part of the Rubik’s cube which opens the gate. The second argument also sets what sequencer message that the patch responds to. Argument three sets the start time (in ms) of the buffer playback and argument four sets how long it is to be played. PlayBuffer also sets the adsr~ object which fades the volume at the beginning and end of the playback to eliminate any clicks in the audio.

User experience, usability considerations and possible future development:

Picture 5: The Exhibition. [6]

Having not been able to extensively test the interface before its unveiling, it was very interesting to observe how users reacted to the Rubik's Studio.

Everyone seemed initially fascinated and intrigued by the interface, and quickly began playing with the Cube and modifying the sounds. 

However initially users weren’t spending long playing with the Cube, though all were fascinated to hear about its design. We began to realize that the audio tracks, once mixed, were sounding fragmented and ‘ bitsy’, so we sped up the playback rate and added a smoothing patch to the programme. Suddenly the Cube began to sound far more sonically interesting. It became possible to modify the Cube in real time making it possible for users to ‘jam’ with the Cube with far more satisfying results.

Users after this point became far more engaged with the interface. This was very encouraging, and with the addition of further audio we believe this could become a powerful and entertaining user interface.

 

 

Video 1: User interaction. [7]

Picture 6: User interaction. [7]

Picture 7: User interaction (trying to solve the Rubik's cube). [7]

Picture 8: Rubik's performance. [7]

Picture 9: Psuedo DJ Battle. [7]

Picture 10: Already a virtuoso [6]

 

Video 2: Virtuoso. [7]

It was interesting to observe also that after experimenting with the cubes sounds, they wanted to hear the original samples so began trying to solve the Cube, adding another level of engagement and satisfaction for the user.

This Cube represents a prototype of an idea. Future expansion on the interface could include:

• Effects cubes (reverb etc)


• Sampler cubes


• Grain synth cubes


• Audio sample banks for cube ‘personalities’.

Related and influential works

1. The Orchestra Explorer


2. BYORC (Bring Your Own Rubik's Cube) Music

 

Resources and references

1. http://reactable.iua.upf.edu/?software/


2. http://www.abstractmachine.net/blog/3/


3. http://cxycling74.com/twiki/bin/view/ResourceGuide/InterestingWork


4. Antonio Camurri, Corrado Canepa, Gualtiero Volpe, Active listening to a virtual orchestra through an expressive gestural interface: the orchestra explorer . New Interfaces For Musical Expression: Proceedings of the 7th international conference on New interfaces for musical expression, New York, New York. SESSION: Controllers and physical models. Pages: 56 – 61. ACM Publishing.(2007)


5. Max/MSP programming assistance by Sam Ferguson.


6. Photo's by Kirsty Komuso (Kirsty Beilharz).


7. Photo's and video's by Piers Gilbertson.