What Does Music Look Like?

Music is an aural art form.  Unlike language, which can be accurately communicated through the visual medium (text), looking at music notation doesn’t carry the same meaning as the actual sound of that piece.  Where an author’s work of art is in the text of the book, a musician’s art isn’t in the score, but the performance.

Human beings are visually oriented, though.  We take in most of our information about the world around us through our eyes.  We also have visual associations with sound (e.g., a dark timbre, a colorful chord).  Writing for Psychology Today’s blog, evolutionary neurologist Mark Changizi considers What Does Music Look Like To Our Brain?

Visual and auditory information interact in the brain, and the brain utilizes both to guess the single scene to render a perception of. For example, the research of Ladan Shams, Yukiyasu Kamitani and Shinsuke Shimojo at Caltech have shown that we perceive a single flash as a double flash if it is paired with a double beep. And Robert Sekuler and others from Brandeis University have shown that if a sound occurs at the time when two balls pass through each other on screen, the balls are instead perceived to have collided and reversed direction.

Changizi offers some other examples of research about visual and auditory processes in the brain, but I think he makes a couple erroneous assumptions.

One approach is to simply ask which visuals are, in fact, associated with music? For example, when people create imagery of musical notes, what does it look like? One cheap way to look into this is simply to do a Google (or any search engine) image search on the term “musical notes.” You might think such a search would merely return images of simple notes on the page.

However, that is not what one finds. To my surprise, actually, most of the images are like the one in the nearby figure, with notes drawn in such a way that they appear to be moving through space.

He’s right about the image search.  Click here to see Google’s image results.  But what if we use the same search engine to search for the terms “musical notation?”  It’s a much different result.  Changizi’s on this musings are interesting to read, but I think he’s using a false analogy.

Could it be that the musical notes are depicted as moving through space because sound waves move through space? The difficulty with this hypothesis is that all sound moves through space. All sound would, if this were the case, be visually rendered as moving through space, but that’s not the case. For example, speech is not usually visually rendered as moving through space.

Really?

The music images Changizi is looking at aren’t meant to communicate any specific piece, even to musicians who can read it (and I imagine that most of those music images are for and by non-musicians).  They are like the comic book style font above, meant to evoke an association with the movement of music through time and space.  If I tried to use that font for this whole post, however, it would distract from the specific ideas I’m trying to convey.

But those points are really just food for thought that Changizi speculates on to get to the main point of his article, that humans visualize music as humans moving.  Using dance as an example, Changizi argues that choreography interacts in time with the music.  We expect the dancers’ movements to correspond with the sounds we hear.

These strong opinions about what music looks like make perfect sense if music mimics human movement sounds. In real life, when people carry out complex behaviors, their visual movements are tightly choreographed with the sounds – because the sight and sound are due to the same event. When you hear movement, you expect to see that same movement. Music sounds to your brain like human movement, which is why when your brain hears music, it expects that any visual of it should be consistent with it.

Does music really sound like human movement, or do we just associate the sound of music with humans that move to it?  Music is performed by humans moving, and they need to move in time to the music.  That’s what we’re used to seeing when we hear music live (we see the drummer hit the cymbal at the same time we hear the sound).  I’ve always found it annoying to watch a video with the sound a bit off, whether or not it’s music or speech.  Anything that makes sound, whether or not it’s music, gets a visual association with the action that produces it.

Changizi writes a little more about this on his personal blog, where he talks about his upcoming book, Harnessed: How Language and Music Mimicked Nature and Transformed Ape to Man.  It looks like it’s due to be released in 2011 and might be a good read.

Thanks to Lyle Sanford and his Music Therapy blog for linking to Changizi’s article.