Originally published in SciArt Magazine: Special Topics August 2017: STEAM
By Jose Luis Contreras-Vidal (Distinguished Professor of electrical and computer engineering at the University of Houston) & Dario Robleto (Artist-in-Residence in Neuroaesthetics at the University of Houston's Cullen College of Engineering)
Creativity and the experience of aesthetic reflection are two of our most profound mysteries − approaches to problem solving and emotional expression that help define what it means to be human. The burgeoning field of neuroaesthetics offers a unique possibility involving three of our greatest disciplines – art, science and engineering – to work in a truly interdisciplinary way, moving beyond the pitfalls of simply referencing across fields as we probe these essential expressions of our humanity. To ask, in turn, an artist and a neural engineer about the interplay between perception, cognition, memory, and emotion that gives rise to the individualized experience of creativity and aesthetics will undoubtedly provoke different methods of inquiry and possible answers. But it is precisely this difference in perspective, perhaps even surprise at our commonalities, and the challenge of communicating it across fields that is needed for innovative approaches to these profound questions.
Through our collaboration at the University of Houston, in museums and other public spaces, we have begun to investigate these mysteries through capturing and analyzing the brain activity of people “in action and in context,” as they engage in observing, interpreting, and experiencing art or performing creative acts in real-world settings, outside the constraints of the laboratory.
We are finding a common neural grammar, as well as individual differences in the neural structure and language of creativity and aesthetic experiences. And we are talking with people across scientific and artistic communities, seeking not only to explore fundamental questions about the link between the brain and artistic expression but also to challenge the clichéd ways in which both artists and scientists may think when they envision how the other side is formulating questions. These clichés can become harmful when, for example, science views “creative liberties” as a lack of rigor or does not understand artistic inquiry and methods as valid forms of knowledge production. There can be a tendency, and it is a legitimate concern, to consider the artist a “lab rat,” as if the range of creative behavior can be tidily recorded on cue. Perhaps worst of all, there is an assumption that the artist’s role is to render or illustrate data sets supplied by the scientist.
Artists, on the other hand, can misunderstand the methods of science through approaches such as “quantification,” “localization,” or “reductionism” as blunt tools that completely miss the nuance of aesthetic and creative experience. This is exacerbated by the misunderstanding that neuroscience’s aim is to “explain” the meaning of art, rather than to elucidate the underlying mechanisms that allow for the experience, such as perception and cognition, and are no less crucial to creating the meaningful experience we yearn for.
Bluntly, the artist’s misunderstanding usually resides in a fear that science is going to “do” something to the inherent “mystery” of art, namely destroy it by looking under that sacred rock.
These positions must be vigorously challenged through collaboration.
Does studying what happens in the brain during the act of creativity or aesthetic experience lose the forest for the trees? A main criticsm of this young field of neuoraesthetics is that the methods of science undermine the personal, emotional and experiential qualities of making and observing art, which are undoubtedly based in hard-to-quantify variables such as the construction of meaning at the level of the individual, context, time and cultural factors. In other words, does this investigation destroy the very state of mind it seeks to understand?
We believe it does not. To look under the rock is not to strip away the mystery. It is to find more mystery, new questions, and a better understanding of the neural grammar of the mind.
In our own transdisciplinary collaboration, we have involved not just visual artists but also dancers, musicians, writers and chefs, all using variations on the Exquisite Corpse parlor game made famous by the Surrealists in the 1920s. It has become fashionable to make the claim that artists were our first neuroscientists. Studying painters of the past, for example, offers insight into how artists illuminated brain structure and the mechanisms of perception through inventive techniques of luminosity, rendering of shadows, and an understanding of the visual illusions our brain plays on perception. Less explored is an analogous argument, that the rich tradition of artist’s inventive performances, games, “actions,” or “prompts” holds similar insights for the brain sciences today. By adapting these games as experiment design, we hope to tap into this tradition of the arts exploring the inner workings of the mind.
Other projects have involved museum visitors, children and people using technology to create virtual worlds, interact with the environment, or augment their sensorimotor capabilities. We have strived to collect data in real-world settings, often museums, which have proven to be a rich laboratory for the study of creativity and to facilitate discussions about innovation in science, technology, engineering, arts, and math (STEAM). Our work has been aided by a new generation of mobile brain-body imaging (MoBI) technology, which can capture signaling information from different parts of the brain and the moving body, and a growing awareness that the answers matter, both for art and for the development of neurodevices to address a myriad of health problems.
In this regard, knowledge about the neural grammar of creative and aesthetic experiences can lead to personalized art therapy, new forms of precision medicine and neurotechnologies to promote learning, creativity, and insight. Thus, devices that interface with the brain for diagnostics and therapeutics, or modulate real-time brain activity and behavior, will be a major locus of innovation worldwide over the next decade.
In Action and In Context
Both of us have long been fascinated with what neuroscience can tell us about the creative process, an interest solidified with a 2015 grant from the National Science Foundation to study what happens in the brain as people create and contemplate art and beauty. In this project, we are quantifying the extracranial electroencephalogram (EEG) brain signal with increasingly sophisticated algorithms (“quantitative” or “qEEG”). The accurate analysis of these qEEG measurements is critical to understand the role of naturally occurring variability in brain activity – a necessary step to understand the neural underpinnings of creativity.
One ongoing project began as a collaboration between the University of Houston’s Cullen College of Engineering and Blaffer Art Museum, the university’s contemporary art museum. A number of other museums and institutions have since participated, including the Museo de Arte Contemporáneo de Monterrey in a collaboration with the School of Engineering and Sciences at the Tecnologico de Monterrey, Mexico.
Three visual artists, Lily Cox-Richard, Jo Ann Fleischhauer, and Dario Robleto were named artists-in-residence at the Cullen College, signaling the importance of artists to the work conducted in engineering professor Jose Luis Contreras-Vidal’s Noninvasive Brain-Machine Interface Systems Laboratory.
Hoping to prod at and complicate the divides between art and data acquisition through experimentation, the first in a series of “Your Brain on Art” public collaborations began in late 2015.
That first session featured Cox-Richard, Fleischhauer, and Robleto in a game of Exquisite Corpse, in which each artist – wearing a skullcap embedded with 64 electrodes to track brain activity – began a piece of art, trading places to continue the work begun by one of their colleagues at set times.
The artists’ brain activity flowed across a screen, allowing both researchers and the audience to see the changes as the artists considered what to do, picked up pastels, paints, and other materials and began to work.
Another protocol followed Fleischhauer for 14 months as she planned a sculptural installation, tracking brain activity during both active research and planning activities and times of more casual brainstorming, including while she exercised at a neighborhood gym.
We have worked to involve artists not only in designing these experiments but also in helping to analyze the resulting data. Each participating artist is asked to provide information about their thought and creative processes at specific points in the experiment, ranging from insights about a project to concerns as mundane as diet and medication. Those annotations are critical for task analyses that can elucidate ‘top-down’ mechanisms, which are added to data-driven ‘bottom-up’ analyses performed by the scientists with the help of computers to gain a fuller understanding (‘closing the loop’) of what is happening in the brain.
For the researchers, the ability to capture raw data “in action and in context” has been a rare opportunity, as most previous work in this field has relied upon data gathered in a laboratory setting, mostly involving tracking brain activity as people viewed reproductions of art and other visual stimuli on a computer monitor. The Exquisite Corpse series and related projects have been conducted in public settings, often museums, and involve actual pieces of art.
A point of great interest has been to have artists engage in the experiments as test subjects so they can better understand the mechanisms and objective of the experiment, so as to offer refinements to the questions being asked and the experiment design itself. Of particular interest for us has been the insight gained around the question of what constitutes “authentic” or “ecologically valid” data and if that can meet the standards of both artist and scientist.
In other words, can understanding and taking steps to accommodate the artist’s desire for authenticity — so crucial to the production of meaning in art — make for better science? This is an important question because much of the argument for MoBI technology resides in the assumption that recording real-time data from a diverse group of freely behaving individuals outside the artificial constraints of the laboratory will produce a more “accurate” reading. With these concerns in mind, we are working on criteria that we hope will refine future experiment design and inch us closer to a shared understanding of the expectations, concepts and language used in our respective fields.
No definitional givens of key concepts like “art,” creativity,” “aesthetics,” or “authenticity” should be assumed.
A New Kind of Collaboration
Our own collaboration began in 2014, as Robleto worked on an installation that would be exhibited at the Menil Collection in Houston later that year.
“The Boundary of Life is Quietly Crossed” was a wide-ranging exhibit, combining deep research, historical artifacts, and objects created by the artist, telling stories as disparate as the history of the artificial heart and the effort to send men into space. The show’s inspiration was drawn from the famous “Golden Record” — an “interstellar message” placed onboard NASA’s Voyager probes in the late 1970s to explore the outer planets. The record, containing greetings, sounds, music, images, and languages from Earth’s history also contained the EEG and EKG recordings of the record’s creative director, Ann Druyan. The recordings were made just after she and astronomer Carl Sagan had become engaged with the hope that the memory of their love was somehow recordable and decipherable in the future through the electricity of her brain and heart. Thirty-seven years after Voyager’s launch, Robleto wanted to investigate the underlying premise of that gesture — is our individuality contained in these electrical signatures, and can they be “played” back like our favorite record?
He reached out to Contreras-Vidal, aware that the engineer was working to decode brain activity related to intention and movement. At the time of the Voyager’s launch in 1977, no one working in the field of EEG would have suggested these recordings carried the possibility for “playback” as it related to individual intention, whether as mechanical movement of our limbs or emotional experience. And yet in 2017, Contreras-Vidal’s pioneering work with EEG recording and decoding, along with others, is allowing stroke victims, amputees, and paraplegics some freedom of movement by simply thinking about it, as intention is translated to prosthetic legs, arms and hands. Although the leap to deciphering individualized emotional experience through these technologies is far off, if it can ever be achieved, Robleto sought Contreras-Vidal’s expertise on this frontier of art, science, and philosophy.
That conversation led to discussions about the still young field of neuroaesthetics and plans for an ambitious experiment.
Researchers from Contreras-Vidal’s lab outfitted more than 400 people who visited Robleto’s exhibition with non-invasive EEG skullcaps to record their brainwaves as they observed his art. The ensuing data collection already has resulted in one publication, with others underway, looking at both technical information about the usability and reliability of the MoBI headsets and at the early findings based on qEEG.
The analysis continues, involving data gathered at the Menil and elsewhere. We are uncovering common and unique patterns, based on qEEG, in the participants’ brainwaves to isolate the effects of the aesthetic experience on neural activity or the neural correlates of intentionality, emotion, and expressive movement as artists or children create art. Our qEEG measurements are therefore valuable as biomarkers, as objective endpoints for measuring the efficacy of medical products and art interventions, and as a metric to investigate cognition, emotion, and action.
Yet despite the growing importance of qEEG, little is known about the constancy and variability of qEEG measurements in health and disease across the life span, or even across mental states in real-life situations. In the same way that individual variation in gene sequences makes certain drugs more or less effective for certain people, giving rise to the need for pharmacogenomics, individual variation in qEEG parameters should be understood in relation to emotion, cognition, and action during creative or aesthetic behaviors.
An exciting development has been the full embrace of the museum as a legitimate site for scientific experimentation and data collection. This acknowledges the importance of context in making, observing, and interpreting art that a museum can afford. Thinking through the implications of this new kind of space — the museum as laboratory — required multiple departments at an organizational level to negotiate this novel experience between a work of art and its audience. From the museum’s curatorial department, visitor services, security, legal, and education, as well as the participation of human subjects in experimentation and the University’s engineering team and publicity staff — this effort has been another example of the rare opportunity neuroaesthetics affords to meaningfully collaborate across fields beyond symbolic gestures.
So science clearly has much to gain from learning more about the neural basis of creative action and reflection. But art stands to benefit, too. And while the emerging field of neuroaesthetics has struggled to balance the “neuro” – the science – with the “aesthetics” or artistic side of the endeavor, we are making meaningful strides.
This project is in many ways an extension of that, an intertwining of disciplines that can produce not only their individual products – art and data – but also something more substantive and unique, as we seek to reveal something larger about our humanity.