Yuri's Night Bay Area 2008 (Mindchill, Introscope)
April 2008, myself and two YNBA veterans (Gautam Agarwal, Jonathan Toomim) are participating in Yuri's Night at NASA Ames Moffett Field Research Center with some biofeedback-related art-tech projects we hope will be both intellectually as well as aesthetically engaging. See below for more information on this amazing worldwide expo/art-show/party as well as information on our two projects, Mindchill and Introscope.
[from http://ynba.org/2008/overview.php:]
Yuri's Night Bay Area is a massive celebration of space, science, art, music, and technology . Once a year in over a hundred places all over the world, Yuri's Night commemorates the anniversary of the launch of the first man in space , Yuri Gagarin, and the launch of the first Space Shuttle exactly twenty years later. Starting in 2007, Space Generation teamed up with the NASA Ames Research Center for the first time, along with a team of amazing volunteers, to host the largest Yuri's Night celebration ever held in a massive hangar on Moffett Field in Mountain View, CA.
Yuri's Night Bay Area taps into the San Francisco area's unique energy to bring together scientists, artists, technologists, musicians, and space enthusiasts in a fusion of celebration and education that is unlike anything else you've ever seen.In 2008 the event is growing to twice the size, bringing in more hot musicians , more brilliant scientists , more amazing artists , and the all-new Festival of Ideas .
Learn more about Yuri's Night Bay Area from NASA!
Mindchill and Introscope
by Infodelic Ectomorphs (i.e.)
(Tim Mullen, Gautam Agarwal, Jonathan Toomim)
This project addresses the YNBA '08 theme of "Radical Technology for a Sustainable Future"
Philosophy:
Changes in the world are tightly coupled to changes in our being. While the rapid transformation of the earth becomes more evident, the concomitant acceleration of our thought processes remains invisible. The sensorium created by our technology tends toward finer spatiotemporal scales, making the potentially dire trajectory of the slow and vast processes that sustain life as we know it more an intellectual concern and less a salient experience. We hope to engage Yuri's Night attendees to broaden their scope of awareness through the use of biofeedback. The body is a microcosm of the planet, a complex web of interactions where one ebb leads to another flow. Through biofeedback, users will experience such a coupling between external events and internal states. Along with techniques such as meditation, biofeedback may be useful in exploring the intricate relation between the mental and physical world. We believe that developing an ecology of mind may be a crucial step in understanding and restructuring the immense energetic transaction we are engaged in with our planet today.
Description:
Subjects' galvanic skin response (GSR) and brainwaves (EEG) will control an audiovisual environment that informs them of moment-to-moment fluctuations in their level of arousal (GSR) and general cognitive state (EEG).
Mindchill: Changes in sympathetic activity have been shown to correlate with changes in emotional arousal and stress levels. We will measure this change using a galvanic skin response (GSR) sensor. The user's skin conductance will be measured continuously and his/her arousal will be represented as water in its solid, liquid, or gas states. As a user becomes more aroused, she will cause water to thaw or boil; as she relaxes, she will cause it to condense or freeze. This will be attempted in real-time using a thermoelectric junction, with close-ups of ice-crystal/gas-bubble formation streamed through a webcam and projected in real-time on a screen in front of the user. Alternate feedback modes will include coupling the GSR to time-lapse films of plant growth, ice crystal formation, polar ice melt or other engaging natural processes. Users have the option of being presented with a series of verbal or visual stimuli to trigger emotional responses.
Introscope: Electroencephalograhy (EEG) is a method for measuring the electrical activity of the brain using electrodes placed on the scalp. This project will focus on coupling a user's brain dynamics to an audiovisual environment, affording the user visualization and enhanced control of the ongoing electrical activity of the brain. The EEG signal will have independent and simultaneous auditory and visual representations. Two separate auditory representations will be selectable. In the first, the EEG be transformed in the frequency domain to a range audible to humans and then played through speakers and/or headphones. A number of filters may optionally be applied to increase the saliency of EEG features that correlate well with the quality of subjective experience. The second will analyze the EEG for certain discrete electrophysiological events which will be used to guide a software synthesizer-based music generator. Visual representations will be produced with an array of bright multicolored flashing LEDs. After nightfall, the high contrast between the LEDs' brightness and average ambient brightness will reduce the saliency and visibility of the subject's physical surroundings, facilitating the fusion of the internal and external worlds and carrying her through a journey of her own creation. Additional LED pods will be placed in the area in a dance light-like manner for spectators' enjoyment; the algorithm used to control the audience LEDs will be adjustable independently of that of the subject's dedicated LED pod, and can be turned off selectively.
Below are some photos of Mindchill 1.0 and its debut at NASA Ames Research Center for YNBA '08.
|
Version 1.0 of our home-grown GSR-thermocouple system
|
The Infodelic Ectomorphs testing Mindchill
(left: Gautam Agarwal, right: Tim Mullen)
|
|
Setup: Hanger A wherein Mindchill was housed
|
Gautam discussing matters of the Mind with a crowd gathered around Mindchill
|
|
Mindchill (reverse of projection screen)
|
Another Mindchill mode (chill out and you keep the polar ice caps from melting!)
|
Fluxpoint (2008)
(Mindchill 2.0)
Fluxpoint occurred on Friday June 20th to a sold-out audience at the Chez Poulet Gallery in San Francisco. The event, principally organized by British electroacoustic/new classical composer Richard Warp, was a terrific mixture of experimental music and video, with several live performances, some pre-recorded compositions (combined with video), and a group improvisation session. Throughout the performance, audience members and performers were hooked up to Mindchill, which visualized their arousal level (as measured by galvanic skin response) in the form dynamic changes in time-lapse photography of plant growth, crystal formation, and other natural processes, as well as some fractal image evolution. This was projected on a large display behind the main stage. Additionally, during a special session, audience members were able to manipulate both the visual display as well as a 5-instrument synth ensemble based on their arousal levels and breathing patterns (Rich Warp helped develop the MAX/MSP interface for this).
Below are a couple photos from the event. For more photos go here.
A string quartet performs Claire Singer's 4:8:1
A group improvisation with Mindchill (projected on screen) controlled by Rich Warp (on the Theramin).
Group improvisation with Mindchill (plants); Jacob Wolkenhauer (guitar, center), Claire Singer (cello, right), Richard Warp (theramin/mindchill, far right)
Below is the full program for the event
|
You are invited to FLUXPOINT an eclectic evening of experimental music and film in San Francisco...
HEATHER FRASCH
Heather Frasch is a composer of acoustic and electro-acoustic music, improviser, sound installation artist, and experimental flutist whose music has been performed in the US, Europe and Asia.
MINDCHILL
Mindchill, created by UC Berkeley brain hackers Gautam Agarwal and Tim Mullen, attempts to engage individuals in broadening their scope of awareness through the use of biofeedback. Fluctuations in the user's arousal and affect are measured continuously via GSR and represented as real-time changes in plant growth, ice crystal formation and other engaging and artistic natural processes.
CLAIRE M SINGER
Claire is an electroacoustic composer and performer from Scotland. Her compositional work includes fixed media (stereo and multi-channel), site-specific, multi-media, live electronics and collaborative work. Claire is also involved in 'cello and electronic improvisational work and has performed with various experimental music groups in London and throughout the rest of the UK.
In 2007, Claire was awarded the PRS Atom Award for New Music, which has funded her trip to San Francisco to develop her work with Max/MSP.
DAMON WAITKUS
Damon Waitkus was born in Boston, Massachusetts, in 1977, and earned an MA in composition from Mills College in 2006. Much of his recent work has been for recorded media, combining field recordings collected from various natural, domestic, and urban environments with passages for traditional instruments.
RICHARD WARP
British electroacoustic/new classical composer based in Berkeley, California. An MMus graduate of Goldsmiths College, University of London, his work attempts to explore the musical mind/body schism between "cerebral" sonic architecture and instinctual, emotionally driven impulses.
JACOB WOLKENHAUER
A guitarist who plays unorthodox music to serve two completely different ends - to make the familiar become foreign, and to make the abstract become universal. Through the de-humanization of sampled text, he expands awareness of our most basic form of communication: speech. In combination with instrumental music, he takes abstract forms with no literal reference and gives them voice.
Please do not park in the parking lot next door, it does not belong to us and the neighbors are rabid.
The Chez Poulet is 2 blocks from the 24th street BART train. Plenty of bike parking.
Refreshments available.
RSVP MANDATORY, as space is limited and the event will defiantly sell out.
Our sliding scale is designed to accommodate students, artists in residence or on stipend, interns and monks. Please be generous.
Cell phones ringing during the performance will be confiscated and blended to a puree.
|
In Tones (2010)
In Tones is an upcoming quartet of concerts/installations (16/01/2010) produced by Adam Jansch and Richard Glover at Phipps Hall and St. Paul's Hall at the University of Huddersfield, United Kingdom. The performances will focus predominantly on three fundamental mediums of communication that defined the 20th century: Radio, Television, and the Internet. SF-based British electroacoustic/new classical composer Richard Warp and I will be creating the Internet installation. Our work is, in part, inspired by Alvin Lucier's 1965 Music for Solo Performer, the first work in history to use brain waves to generate sound. In Solo, alpha (8-12 Hz) "brainwaves," recorded from Lucier's brain using EEG, are transmitted to amplified loudspeakers that are used to resonate percussion instruments placed around the hall. By modulating his alpha rhythm, Lucier can effect changes in the musical structure of the performance.
Our performance, Music for Online Performer, will take this idea a step further, exploring the interaction and cooperative control between a "brain musician," attempting to manipulate a quartet of (acoustic) robotic instruments by modulating four fundamental brain rhythms, and a human "composer/conductor" (Richard Warp) who has created a composition that will isolate certain combinations of instruments at different stages in the performance, and is furthermore directing the musician, in real-time, to increase/decrease the power of specific neural rhythms and thereby evolve the musical composition. Furthermore, the musician, composer, and quartet/audience are physically located thousands of miles apart (San Diego, USA; San Francisco, USA; Huddersfield, United Kingdom, respectively) connected only via the Internet. The entire performance will be streamed live online with cameras in all three geographic locations allowing people to connect from anywhere in world and be part of the virtual audience. Online participants are encouraged to interact with the composer/conductor in real-time via a chat room and suggest changes in the ongoing composition (e.g., "increase the cello pitch!").
Details:
Electrical signals recorded from the brain of a participant in San Diego, USA will be used to manipulate acoustic instruments in front of a live audience at Phipps Hall at the University of Huddersfield, UK. Neural activity from the "musician" will be measured continuously using electroencephalography (EEG). These signals will be separated into quasi-independent components using a spatial filter previously learned by Independent Component Analysis (ICA). The activations of 4 informative components are selected and reduced to 4 variables representing changing aspects of neuronal activity in four fundamental frequency bands: theta (4-8 Hz, over midline frontal cortex), alpha (8-12.5 Hz, over visual cortex), mu (10-12.5 Hz, over left hand sensorimotor cortex), beta (12.5-30 Hz, over right hand motor cortex)). These signals will be streamed continuously to Phipps Hall where they are converted to variations in pitch and percussive frequency through the mechanical manipulation of a four-piece instrument ensemble (cello, tympani, cymbal, chimes/bells) using an Arduino board interface. Using Skype, the music is streamed back to the conductor (Warp) in San Francisco and the musician (Mullen) in San Diego, who uses this feedback (along with local visual feedback), combined with compositional instructions delivered by the conductor, to manipulate his brain rhythms and thereby inform the ongoing composition.
Below is a short documentary created by Adam Jansch in which the artists discuss the motivations and challenges behind the four pieces which comprised In Tones (organ/radio/television/internet). Music for Online Performer is discussed towards the end of the film.
in tones: organ/radio/television/internet from Adam Jansch on Vimeo.
Performance Credits:
Thanks to Yijun Wang, Ph.D for his invaluable assistance in setting up the EEG system. Thanks to Swartz Center for Computational Neuroscience for providing the EEG hardware. Software for interfacing with the EEG hardware was adapted from the DataRiver/Matriver package developed by Andrey Vankov and Nima Bigdely Shamlo. Finally, a big thanks to Adam Jansch for his incredible assistance with (among many other things) setting up the robotic instruments in the UK and for inviting our participation in the In Tones series.
| |
A paper describing Online Performer was published in the Proceedings of the 2011 International Conference on New Interfaces for Musical Expression (NIME) in Oslo, Norway and can be accessed by clicking the icon on the left. An extended version of the paper can be accessed here
|
Just: A Suite for Violin, Cello, Flute and Brain (2010)
Just was composed by Scott Makeig and first performed by Grace Leslie, Alex Khalil, Scott Makeig and Tim Mullen on June 2nd, 2010 at the Fourth International Brain-Computer Interface meeting at the Asilomar Conference Center in Monterrey, California. Mental state classification was done with Christian Kothe's BCILab software.
|
|
The full programme notes for Just is available here (click on the PDF icon).
This contains a description of the installation and bios of performers.
|
Final rehearsal prior to performance of Just at the Fourth International BCI meeting at Asilomar (Monterrey, California, USA). Left to right are Grace Leslie (flute, percussion, Max/MSP), Alex Khalil ('cello), Scott Makeig (violin, composer), Tim Mullen ("brainist", neural interface), Christian Kothe (BCI), Dev Sarma (Tech support).
Performance of Just at the Sonic Diasporas Music Festival at UC San Diego.
In this video clip (starting around minute 23:00), Just (and the underlying BCI technology) is featured in UCSDTV's UCSD@50 series honoring UC San Diego's 50th anniversary.
The above highlight article discusses Just as well as other Brain-Machine Interface work being carried out at our lab at the Swartz Center for Computational Neuroscience.
EEG Ocean
EEG Ocean (Nima Bigdely-Shamlo and Tim Mullen @ SCCN/UCSD) is a method for low-dimensional (2D) visualization of the spatial topographies, temporal activations, and correlations between multiple quasi-independent components (sources) extracted from high-density EEG. For more information visit the EEG Ocean page.
|
