EEG Ocean

EEG Ocean (Nima Bigdely-Shamlo and Tim Mullen @ SCCN,INC/UCSD) is a method for low-dimensional (2D) visualization of the spatial topographies, temporal activations, and correlations between multiple quasi-independent components (sources) extracted from high-density EEG. 

Background

The brain is a complex environment with many spatially-removed subsystems interacting over time. Visualizing patterns of brain activity over long periods of time is generally impossible by simply looking at long time-series of EEG. As such, researchers usually apply reductive techniques which seek to extract meaningful patterns of activity from the time-series. For example, we might apply spectral decomposition techniques to identify frequency-specific structure within activity recorded from different parts of the brain. However, with these reductive techniques it can be difficult to get an intuitive sense of the overall pattern of ongoing brain activity across time and space.

EEG Ocean is a novel approach to visualizing complex spatiotemporal patterns of dynamic brain activity over long periods of time in an intuitive and aesthetically-pleasing manner. EEG is thought to be the summed activity of multiple cortical generators or "sources." To approximately recover the activations of these sources, Independent Component Analysis (ICA), a blind source separation technique which separates a multi-channel EEG signal into maximally independent components (ICs), can be applied to EEG data collected over a long period of time. Sources that are interacting (non-independent) may preserve some residual mutual information after applying ICA. As such, ICs are clustered using Multidimensional Scaling (MDS) such that those with high residual mutual information are spatially proximal while those with little shared information are pushed apart. The activity of these ICs are then displayed as "ripples" propagating out through time from their respective ICs (neural "sources"). With a high-resolution display this allows us to visualize the collective activity of many neural components (independent or interacting across time) over long periods of time (e.g., many minutes, rather than seconds). Event Related Potentials (ERPs) and oscillatory bursts appear as prominent ripple sequences which can be traced back to their respective sources. Relative timing of neural events between ICs can be easily inferred by comparing the respective radii of ripples emanating from different ICs. A single frame is a "snap-shot" of all significant neural events over some period of time up to the present, where the length of time visualizable depends on the step size between frames and the resolution of the display. A more detailed description of the procedure is outlined below.

Algorithm

  1. N-channel EEG is separated into N (maximally) independent components using Independent Component Analysis (ICA)
  2. A subset of informative Independent Components (ICs) are selected for display (this selection may vary depending on the goals of the researcher)
  3. Selected ICs are projected onto a 2D image using metric multidimensional scaling on a relevant measure (such as mutual info or latency between two ICs in which mutual info is maximum).
  4. ICA weights are rendered as topographic plots ("islands") each depicting the spatial distribution of activation of a given IC across the scalp (nose points up)
  5. Select a reference time point (current time). For each pixel in the image, distance from the pixel to each IC location is calculated and mapped into latency w.r.t. the reference point (longer distance equals longer latency before current time point in EEG). A measure of IC activity (e.g., activation, abs. activity, power at certain band, or mutual information with class) is assigned to that pixel based on calculated latency and these values for different ICs are summed together.
  6. Apply a preferred color mapping to pixel values.
  7. Repeat 5-6 for a range of current time points over a segment of EEG (of arbitrary length) with a specified step size (e.g., 50 ms).

Example

In the following demo, EEGOcean is applied to 64-channel EEG of a representative subject performing a Rapid Serial Visual Presentation task (RSVP; many image chips displayed in rapid sequence (12 images per/sec) with a target chip randomly embedded in the sequence). EEG is collected at 256 Hz. ICA is applied to the data and selected IC power (abs. value of IC activations after bandpassing with 3-30 Hz zero-phase FIR filter) is visualized in approximately real-time (with a rate of 48ms of EEG per frame, each second of video corresponds to 1.1209 seconds of EEG). For this demo, we have a display width of 3200 pixels, so any given frame contains between 6.25 and 12.5 seconds of activity for a given IC (depending on the location of the IC). Note that with a resolution of 35,000 x 8000 (e.g., maximum HiPerSpace resolution) and a step size of 100ms of EEG per frame, a single EEGOcean snapshot would visualize ~30-60 minutes of activity for all selected ICs. This provides a unique ability to gain an intutive sense of ongoing brain dynamics and identify salient neural events occurring over very long periods of time, such as during sleep, resting state, etc.


The movie below is a reduced-resolution video (for web streaming) of EEG Ocean applied to 12.5 seconds of continuous EEG.

EEG Ocean Movie Clip



High-resolution screenshots of EEG Ocean from the above movie (click for larger image). An interactive multi-scale version is available here.
 
 
 
 


In November, 2008 EEG Ocean was displayed on the HiPerSpace wall at Calit2. Until 2009, HiPerSpace was the world's largest LCD display wall with a maximum resolution of 35,840 x 8,000 pixels for a total of 286,720,000 pixels. I was assisted in this demo by Ramsin Khoshabeh and members of Falko Kuester's lab at Calit2. Below is video and images of EEG Ocean on HiPerSpace. Note that for this demo we used a display resolution of 3200 x 1600. As such, EEG Ocean is making use of only a fraction of the total HiPerSpace real estate. 

Screenshots from EEG Ocean on HiPerSpace at Calit2
 
 
 
 


Future Work

EEGOcean is still in its infancy. Within certain resolution and framerate limitations, is currently possible to use EEG Ocean in real-time using SCCN's open-source Datariver/Matriver software. We plan to add interactive control allowing the user to call up alternate views (e.g., 3D dipole displays and raw time-series) for selected components and obtain more detailed information about a given component or display section. Multi-scale imaging technology can be used to allow the user to change the temporal scale (zoom in) while preserving resolution. See this link (thanks to Nima Bigdely) for a simple example of an interactive multi-scale implementation using Microsoft Seadragon Ajax.

Software

A beta (older) version of EEG Ocean capable of real-time usage is included in the open-source Matriver package (Matlab) accessible here.  Contact me for details on newer versions. 

Copyright (c) 2009 Tim Mullen