Music notation

Western music notation is a venerable method of visualizing music. It’s a very neat and compact system, unambiguous and digital, and not too difficult to learn. Programs like Sibelius can effortlessly translate notation to and from MIDI data, too.

But western notation has some limitations, especially for contemporary music. It doesn’t handle microtones well. It has limited ability to convey performative nuance — after a hundred years of jazz, there’s no good way to notate swing other than to just write the word “swing” at the top of the score. The key signature system works fine for major keys, but is less helpful for minor keys and modal music and is pretty much worthless for the blues.

Here’s a suggestion for how notation could improve in the future. It’s a visualization by Jon Snydal of John Coltrane’s solo in Miles Davis’ “All Blues”  (I edited it a little to be easier on the eyes.)

Snydal’s visualization is more analog than digital — it shows the exact nuances of Coltrane’s performance, with subtle shadings of pitch, timing and dynamics.

MIDI sequencers suggest further improvements over standard notation. Here’s a simplified electronic music sequencer called iNudge. Play, it’s fun:

Here’s Thelonious Monk’s tune “Four In One” as shown in standard MIDI “piano roll” view. The rectangles show not only which notes are being played and when, but exactly how long they’re held. Darker red means louder, paler pink means quieter. You can also read volume off the bars along the bottom.

MIDI is a versatile and user-friendly system. It can capture your keyboard performances, you can import scores, and you can even just draw notes onto the screen directly (my preferred method.)

The Music Animation Machine has a wonderful series of videos matching MIDI piano rolls of various classical pieces with recordings of them. Here’s Bach’s infamous Toccata and Fugue in D minor.

As software gets more sophisticated in its ability to extract pitch data from actual audio recordings, you can start manipulating them with the same ease as MIDI. Here’s a screencap of the pitch-correction program Melodyne, a close cousin of Auto-tune.

The lines show the actual sung pitches, and the orange blobs show the notes the program thinks the singer meant to hit. The blobs’ thickness shows volume. You can drag and drop the blobs and redraw the lines at will to alter the melody to your heart’s content. Melodyne even transcribes the performance to standard notation and MIDI for you.

High and low

We’ve made up our collective mind that faster frequencies should be spatially represented as being “higher,” and that slower ones should be spatially “lower.” It seems so reasonable, but really it’s totally arbitrary, and doesn’t even line up with physical experience. On the piano, the high notes are on the right and the low ones on the left. On the guitar, the “low” E string is physically located above the “high” one. The fingerings for higher and lower notes on wind instruments don’t correspond to a simple higher-lower axis either.

Absolute pitch is a straight line ladder, but pitch class is circular. The truest representation of pitch space is a helix.

Other ways to conceptualize pitch space

High and low aren’t the only metaphors we use for faster and slower vibrations. Like I said, pitch class is circular.

But the circle is really just replacing up/down with clockwise/counterclockwise. There are other ways to conceptualize pitch. We intuitively experience changing pitches as moving closer and further, or inwards and outwards. We also think of higher pitches as brighter and lower pitches as darker. Players of stringed instruments sometimes tune their upper strings a little bit too high on purpose, producing an effect known as brilliance.

Time

It’s a universal convention that notation shows time moving from left to right. But that’s not the only possible axis to use. How about forwards and backwards instead? That’s the paradigm in rhythm games like Dance Dance Revolution and Guitar Hero. The purest realization of this concept is in a game called FreQuency.

The game even allows you to construct your own remixes.

I like this tunnel metaphor and would like to see it extended into a full-blown production environment.

Waves

Pitches are sine-wave vibrations, and you can visualize them as such.

Sine waves wouldn’t make for very a helpful music notation, but they do help you understand what’s going on scientifically when you physically hear something. They’re even better animated:

See all of Wikipedia’s animated drum heads.

Waveforms

Audio editors show music as amplitude waveforms, blobs that get wider where the sound is louder. Here’s the Funky Drummer break in Recycle. The blue blobs show drum hits. These amplitude blobs don’t tell you much about the musical content except for timing and volume. But Recycle was meant for drum loops, where timing and volume are the only information you really need.

Here’s a graphic I made showing how you hear the Funky Drummer as it’s looping:

In a post on Design Observer, Rob Walker discusses the waveform as the new icon for music, replacing the stylized eighth notes or records that have done the job in the past. The SoundCloud player uses an attractive waveform graphic that helps the listener track where they are in the song by following the volume peaks. There’s even a SoundCloud group called Pretty Waveforms.

Tomorrow Never Knows by ethanhein

The waveform has the potential to move from purely functional settings to more decorative ones. Here’s a waveform-based labeling concept by Joshua Distler, showing the tracks on Post by Björk.

Music theory and networks

I’ve always thought it would be cool to use networks to conceptualize music theory, and have made a few attempts at doing so. Here’s a comparison between the circle of half-steps and the circle of fifths, which are involutes of each other:

Here’s a map of the chord progressions in “Giant Steps” by John Coltrane.

And here’s a flowchart showing how you can figure out what scale or mode you’re hearing.

It would be way cooler to have more abstract three-dimensional interactive visualizations showing how chords, scales and melodies function. Leonhard Euler showed how you can represent tonal harmony as a lattice with the topology of a torus, as shown in this animation. Red lines show major thirds, green lines show minor thirds, and blue lines show fifths:

I have ambitions of my own in this area, but so far, I lack the programming skills to realize them. Others are taking some exciting strides, though. Dmitri Tymoczko made waves for getting the first music-related article published in Science about his topological visualization methods for tonal harmony. I can’t quite wrap my head around his ideas, but they’re intriguing.

Here’s an illustration by Aniruddh Patel from his paper, “Language, Music, Syntax And The Brain.” Again, I’m not totally clear what it all means, but I plan to investigate further.

Other theorists have attempted to use color to show harmonic function. Scriabin invented a “keyboard of lights” for that purpose, though it didn’t really catch on.

Visualizing musical form and structure

I like to use simple color-coding to keep track of which section is which while working on a song. Yellow is for intros and outtros, blue is for verses, green is for choruses and orange is for instrumentals and breakdowns.

Edward Tufte shows some more sophisticated song structure visualizations on his forum:

The Shape of Song project by Martin Wattenburg shows repetition within a piece of music. Here’s his visualization of “Like A Prayer” by Madonna.

Here’s Wattenburg’s visualization of Beethoven’s “Für Elise.”

Speculation

Here’s an entertaining video showing how you can create a happening drum machine sequence using counting in binary by Niklas Roy.

Wouldn’t this graph coloring system make a cool music notation or interface?

Visual Complexity has many more ideas like this one.

I feel like we’ve barely scratched the surface of useful and attractive schemes. Are there other cool visualization methods I should know about? Hit the comments.

Updates

John Clover hipped me to this post, which overlaps heavily: Amazing Music Visualizations and Teaching

I just had the chance to play with some of Björk‘s Biophilia song/apps. Some of them are groundbreaking interactive visualizations; some are just entertaining and groovy; some are baffling but deserve points for creativity. All the way around, it’s a remarkable experiment, one that I think is going to be influential.

See this post on Quora

DJ Kool Herc was one of the first DJs to remix tracks on the fly using turntables and a mixer. Rather than playing songs from beginning to end, Herc isolated and repeated the hooks and breaks. The break in a dance song is the section where all of the instruments drop out except the drums, percussion and maybe the bass. In disco and funk songs, the break is the energetic peak moment, when the dancers really get down. Herc discovered that by isolating and looping, say, the break from “Apache” by the Incredible Bongo Band, he could bring the dance floor to ecstatic new heights.

To loop a break, you need two copies of the same record, one on each turntable. While the break plays from the left turntable, you cue up the beginning of it on the right one. At the end of the break, you quickly crossfade to the right turntable. While the break plays from there, you cue up the first copy to the beginning of the break on the left turntable. In theory, you could loop a break like that indefinitely.

Along with Herc, the first generation of hip-hop DJs also included Afrika Bambaataa and Grandmaster Flash, whose dense vinyl collages prefigured the mashup movement. Hear the prehistory of the mashup in “The Adventures Of Grandmaster Flash On The Wheels Of Steel.”

It’s relatively easy to put a collage like this together on the computer, but it takes delicate timing and tons of practice to do it using vinyl.

Turntablism broke into mass consciousness when Grand Mixer DST appeared with Herbie Hancock on “Rockit” in 1983. The first turntablist I had a relationship with was Jam Master Jay, from his work with Run-DMC in tracks like “Peter Piper.” More recently I got hip to DJ Premier, who produced my favorite Nas track, “Nas Is Like.” Hear Primo on Gang Starr’s “DJ Premier in Deep Concentration.”

As jazz gave birth to technical and esoteric styles like bebop, so turntablism has its own highbrow virtuosos. DJ Rob Swift is a leadng member of the artsier school of turntablism. He’s more of an abstract improviser than a sequencer of recognizable hooks and grooves. Hear Rob Swift scratching a series of records, starting with the Funky Drummer bonus beat reprise.

Here, Rob Swift demonstrates a turntablist notation system (thanks Nick Seaver):

See more scratch notation, courtesy of Wayne Marshall.

Speaking of the Funky Drummer, see an amazing freestyle by Mos Def, Black Thought and Eminem over a DJ scratching the Bonus Beat reprise.

DJs have done most of their collaboration with emcees and electronica producers. They’ve been slower to work with traditional instrumentalists — there’s a lot of new musical vocabulary that has to be learned on both sides. It’s beginning to happen, though. In my hippie-ish youth I enjoyed going to see DJ Logic scratching it up with jam bands. And Youtube loves this video of a violinist getting down over a few different records, including the instrumental of “Check The Rhime” by A Tribe Called Quest.

Vinyl and the gear to play it are heavy, bulky and expensive. If I ever start DJing seriously, I’ll probably opt for vinyl emulation software. It doesn’t necessarily have the same touch as vinyl, but it would be pretty awesome to be able to scratch anything in my iTunes library.

Using a turntable, you can simulate unearthly speech sounds by rapidly scratching the record to and fro. The circle becomes complete when beatboxers emulate the sound of DJ scratching with their mouths. Enjoy Rahzel beatboxing a set of Wu-Tang songs.

Any more videos, links or other turntable-related goodness I’m missing? Hit me up in the comments.

Update: great blog post on classical musicians’ experiments with turntablism pre-1950.

The simplest sheet music reads as a straightforward top-to-bottom list of instructions. You start on measure one and read through to the end sequentially. That’s fine unless the music is very repetitive, which most popular music is. The loop is the basic compositional unit of nearly every song you could dance to. The problem is that writing loops out sequentially is very tedious.

Rather than writing the same passage over and over, you can save yourself a lot of laborious writing by using repeat markers. They’re like the GOTO instruction in BASIC. Here are the first four bars of “Chameleon” by Herbie Hancock. This four-bar phrase repeats hundreds of times over the course of the song. You wouldn’t want to write them all out. With repeat markers, you don’t have to.The repeat markers are the things on the ends that look like brackets with two little dots inside them. When you get to the closing repeat marker at the end of the passage, you jump to the opening repeat marker.

By default, repeat markers tell you to just play the repeated section once. You can also specify any number of multiple repeats. For “Chameleon”, that wouldn’t help much, because if you said repeat 128 or 256 times, no musicians could count that high. For open-ended music, you’re better off writing “repeat until cue.” This leaves it to the performer to decide how many repeats there should be.

Repeat markers give sheet music the topology of a clock face. Here’s how you hear the Chameleon loop. Read clockwise:

Western music notation also includes more complex repeats, with different conditional instructions waiting for you depending on whether it’s your first, second or third pass. Repeats get combined with special jump instructions like DC, short for Da Capo, “to the head.” DC tells the performer to jump to the beginning of the piece. Coda, “tail”, means “jump to the ending.”

Through clever use of repeat markers and jumps, you can fit elaborately complex musical scores onto a sheet or two of paper, which is very convenient if you’re playing an instrument that makes it hard to turn pages in mid-flight.

Even with its repeats and jumps, European classical music is mostly linear. The scores can be expanded into single rows of measures, like a chain of paper clips. Open-ended loops, the ones marked “repeat until cue”, are another story. A linear expansion of a piece containing open-ended loops will be different for every performance. Who knows how many paper clips you’ll wind up with in your chain?

European classical music can get boring because it isn’t repetitive enough. Outside of western Europe, open-ended repetition is the central organizational element of music. We in America have been wise to pay attention to this lesson. Jazz, country, rock, funk, hip-hop and dance music of all description use some form of “repeat until cue” in almost every song.

James Brown in particular was a master of open-ended loops. Usually bandleaders take the musicians in and out of loops through hand signals or eye contact. James Brown preferred to shout instructions to his band out loud in mid-song, and his shouts became a key element of his music. The most famous example is in “(Get Up I Feel Like Being A) Sex Machine.”

As the band loops on the main E thirteenth chord riff, James Brown and Bobby Byrd debate whether it’s time to take it to the bridge:

James Brown: Bobby! Should I take ‘em to the bridge?

Bobby Byrd: Go ahead!

James Brown: Take ‘em on to the bridge!

Bobby Byrd: Take ‘em to the bridge!

James Brown: Should I take ‘em to the bridge?

Bobby Byrd: Yeah!

James Brown: Take ‘em to the bridge?

Bobby Byrd: Go ahead!

James Brown: Hit me now!

The band transitions into a loop on A ninth. They play that loop until James Brown orders them back to the main part. At the end of the song comes another debate:

JB: Can we hit it like we did one more time, from the top? Can we hit like that one more time?

BB: One more time!

JB: One more time, let’s hit it and quit!

BB: Go ahead!

JB: Can we hit it and quit?

BB: Yeah!

JB: Can we hit it and quit?

BB: Yeah!

JB: Can we hit it and quit?

BB: Yeah!

JB: Hit it!

Loops are easy to memorize, but not so easy to perform. It takes deep concentration to play James Brown songs. Musicians trained in the Western classical tradition often find reading through a complex linear score to be easier than sustaining an open-ended groove.

Electronic instruments, on the other hand, make looping a breeze. Once you have a drum machine pattern, sample or MIDI sequence set up, you can loop it effortlessly and endlessly. The effort comes in breaking the symmetry, deciding how long the loop should play, when parts should enter or exit. My friend Josh Koppel likes to use the phrase “a digital thing in analog form.” In the hands of James Brown, his band functioned as an analog loop and sample player, one that could improvise to boot.

The computer science equivalent of “repeat until cue” is the for loop, also known as the if-then loop. Instead of explicitly telling the computer how many times to repeat a given instruction, you can have it keep repeating until some condition is met. Let’s say you want the computer to list every number from one to ten thousand. Here’s one way to do it in pseudocode — the second to last statements is the if-then loop.

Input starts at zero.

Input plus one equals output.

Print output to screen.

If output is less than ten thousand, then take output as the new input and repeat.

If output equals ten thousand, then end.

Using a few if-then loops, you can generate huge or even infinite amounts of complexity from very short programs. Fractals like the Mandelbrot set are generated using just a few simple loops.

Computers need to be told explicitly when and how to terminate their loops. Otherwise, they go around and around forever, a condition you experience as a crash. Humans are smarter in this regard: when told to repeat forever, we eventually get bored and stop.

The color-coding system started as a simple information-management technique, but it ended up improving my ears. Spending so much time looking at these colorfully abstracted representations of so many songs, I couldn’t help but notice some patterns. I’ve done enough tracks now that I can lay something out in the sequencer and know that it’ll basically work without having to listen to it first. Classical and jazz musicians get to the point where by glancing over a score, they can hear it quite clearly in the mind’s ear. The Reason sequencer has a much shorter path into the brain’s deep sense-data processing centers because it’s dynamic, animated, and responsive to my thoughts in real time.

So here are three representative tunes. The rows are instruments, mostly sample players along with the odd drum machine or synth. The columns are groups of eight bars, sixty-four beats according to the dance music convention of a bar comprising eight eighth notes. You can see that every phrase in these tunes is two, four, eight or sixteen bars long. This is no accident. Powers of two sound good. Each colored brick is a phrase worth of sequencer data. My system is to color intros and outtros pale yellow, the verses blue, choruses green, instrumentals and breakdowns orange, and bridges purple. The colors are chosen soley on the basis of what looks good together on the screen.

The Sign – Revival Revival vs Ace Of Base vs Super Mario Bros vs Scientist

Even though this track is a freakshow sonically and memetically, its underlying structure is total pop boilerplate. Every phrase is eight bars long. The intro is mostly identical to the verses, each of which is followed by the chorus. The breakdown is modeled on the verse, but twice as long, and is followed by a triple chorus to end the song. The outtro is the last chorus spaced out and with no drums, plus a little tag on the very end.

Tomorrow Never Knows – Revival Revival vs the Beatles vs M.I.A. vs Missy Elliot

Tomorrow Never Knows by ethanhein

Since this is a fake Ravi Shankar tune, it has a more open-ended, less narrative structure. The single-chord jam fits electronic music like a glove, and fake Middle Eastern and Asian music usually translates better to computers than Western linear tunes with definite beginnings, middles and ends. Tomorrow Never Knows is a single mixolydian scale to infinity, verses interspersed with open-ended passages of swirling modal chaos. I have a few different versions of the basic loop with different densities, but functionally they’re all interchangeable. The colors are more general landmarks for me.

Love Her Madly – Revival Revival vs the Doors vs James Brown

We like this tune so much that we extend it like crazy in performances. The recording only goes to the bridge once, but live we do a breakdown and then go back into the bridge, sometimes a few times. The outtro is when I rap.

MIDI sequencers like Reason have done for music notation and composition what word processing and the internet did for the written word. Especially intriguing is the way you can move the loop markers around during playback. My one sadness with Reason’s sequencer is that while there are many operations you can perform during playback and live recording, you can’t copy and paste in the sequencer window without stopping first. Maybe there’s some unavoidable software constraint here, or maybe it’s just lazy coding, I’ll give the Reason guys the benefit of the doubt and assume the former. I’m finding so much inspiration in their software, it feels ungrateful to criticize.

Reblogged on Hysysk’s Delicious