What remains of a performer, if he or she may may only give the impulses for auditive and visual processes of a performance, but when the actual performance is done by audiovisual electronics?
This is the central question Falk Hubner asked himself in the Shadows-project.
To try out the possibilities of live interactive recording and manipulation of sound and video Falk came to do a LAB. He would bring the viola player Örse Ádám and dancer Gabriela Tarcha with him as performers. In his briefing he formulated the following aim for the LAB:
At the end I want to know what I precisely want with the electronic system, what it should be able to to. At the end of the lab I want to take the patches home, so that I can continue working on them on my own. I want to know what the system will be doing.
Also he wrote a list of experiments he wanted to try out:
- Delay for audio and video, and combining the electronic material with the performance-fragments of the performers
- Processing and manipulation of audio and video – filters, saturation, modulation
- Relationships and possibilities of manipulation between audio and video (video-signal controlling audio, vice versa)
- Recorded (e.g. buffer~) vs. live-processing
- Experimenting with projections on the screens standing in front of the performers, and how the live and projected pictures could merge (or not)
- Experimenting with the microphones: What can they record besides music and voices (movement noises)?
At that stage the Performance Engine and especially the Sound Engine was not yet capable to do real-time recording and processing of sound. Also the 2D engine was not yet capable of using live-video input. So we had to negotiate what we could do in the short time we had in preparation to the LAB. We focused mostly on the Sound Engine and prioritized real-time recording and playback of audio as the main feature that needed to be developed.
The first day we tried out several features. Using the WII-mote, motion tracker and region patch we could let the dancer decide to record audio with the WII-mote that the Viola player was playing and playback the last sample by moving into several area’s on the floor. At some stage additional pre-recorded audio fragments where added to the playback capabilities. These prerecorded tracks where manipulated by Falk in advance, thus compensating for the lack of real-time sound manipulation in the Audio Engine.
The dancer had now most of the control of what was being recorded and played back. To balance this out we gave the Viola player also a separate WII-mote. Now she was in control of what would be recorded and which sample would be played back when the dancer would move into the playback region. The dancer however could still ‘respond’ with her pre-recorded sounds. An interesting ‘dialogue’ of sound and movement was starting to unfold.
At the end of day one video was introduced. Some video was shot on-stage and some was recorded earlier. Video close-ups of both players where put at separate places on the back wall. Motion tracking was now used to fade in and out both faces in opposite direction. A third full wall video was put over both close-ups and connected to a marker of the Reactivision system. The dancer could manipulate both the playrate and fading of this third movie. Both performers needed now to move to control the video’s.
The second day Falk introduced a little patch he made in MaxMSP himself. This piece of software was counting down a random amount of seconds. When hitting zero it would display a number on either the left or right side. These numbers corresponded with actions the dancer or viola player needed to execute. This way they would be ‘controled’ by the machine. The patch was actually connected to the Statemachine, enabling it to give cues to the Sound Engine.
In the last part of the day all media where combined in an effort to make a small presentation workflow. A day later the experimental scenes where presented to an audience. While Falk was very happy with the opportunity to be able to test his ideas he also realized that his approach in the LAB didn’t deliver material he wanted to use for his final piece. The LAB helped him, through quick-prototyping, to realize that a new approach to his research and performance ideas was necessary. Which saved him a lot of time in developing technological systems that would in the end not result in the experience he was looking for. Both the viola player and dancer where very enthusiastic about the LAB and the technological possibilities.