Standard music notation draws a simple and direct connection between pitch height and staff position. However, no such direct visual mapping exists between a note’s duration and its length on the page. The illustration below shows two measures of 4/4 time. One might naively expect the measure on the left to be much longer than the one on the right, but they occupy precisely the same amount of musical time.
Asking beginner music students to simultaneously learn rhythmic concepts and a rather abstract system for representing them is asking a great deal. The MIDI piano roll is a better aid to comprehension for beginners, since it shows longer notes as being visually longer. The relationship between frequency of onsets and tempo is reinforced by the combination of the sight of the playback head sweeping across the piano and the resulting sounds.
Toussaint (2013) compares eight different visual representations of the Cuban rhythm son clave, known to American rock audiences as the Bo Diddley beat. The top four representations are variations on standard musical notation. The bottom four are simplified and abstracted visualizations, less culturally specific.
The fifth of these representations is called the Time-Unit Box System, and will already be familiar to producers of electronic music, who have encountered variations on it in their hardware and software. It is the Time-Unit Box System that forms the basis for the Drum Loop.
We frame all abstract thought using metaphors relating back to states of our bodies. Indeed, body states are the only basis for abstract thought that we possess (Lakoff & Johnson, 2003). In music, we “climb up” a scale, we “ratchet up” tension and “release” it, we “land” on the tonic chord after a cadence. The closer a metaphor is to a state of the body, the shorter the chain of abstraction we must parse out, and the easier it is for us to understand. Metaphors that are several layers of abstraction removed from bodily states will be more difficult to learn and remember.
Musicians take bodily metaphors for granted, but it is worth pausing to reflect on the strangeness of finding visual expression for purely auditory phenomena. Music does not “look like” anything; it is remarkable that we have nevertheless evolved so many strongly felt ideas for how it looks. Brower (2008) lists the six most commonly used visual metaphors for music:
Wilkie, Holland and Mulholland (2010) demonstrate that the most effective bodily metaphors for aiding in musical understanding include containers, cycles, verticality, balance, the notion of center-periphery, and a narrative of source-path-goal. The music software should use these metaphors to create intuitive mappings between sound and image.
Time-Unit Box Systems
The Time-Unit Box System (TUBS) is a simple method for transcribing rhythm. Each time unit, usually an instance of the underlying beat or tactus, is represented by a box. If a box contains an onset, it is filled; otherwise, it is empty. The TUBS representation is popular among ethnomusicologists, especially when dealing with percussionists who are unfamiliar with Western notation. Toussaint (2013) shows the six fundamental 4/4 time clave and bell patterns in TUBS notation:
The hip-hop transcriptions created by Charlie Hely turn standard western notation into a TUBS by quantizing it spatially on graph paper.
Drum machine interfaces are almost always predicated on TUBS. For example, the image below shows a drum pattern in Ableton Live’s MIDI clip editor. Time goes from left to right, and then circles back to the leftmost edge. The vertical axis lists different drum sounds: kick, rim shot, snare, clap, hi-hat and so on. The MIDI sequencer adds another axis to the basic TUBS, using both a color scheme and vertical lines below to show each hit’s loudness (velocity).
The MIDI piano roll can be thought of as a kind of “super TUBS” with an infinite number of boxes that can represent arbitrarily large or small units of time. The image below shows part of “Four In One” by Thelonious Monk, as represented in Propellerhead Reason’s MIDI piano roll. This time, the vertical axis represents pitch, as indexed by the piano keyboard on the left. The vertical line shows the playback position, which the user can manipulate at will.
MIDI sequencers are a remarkable hybrid between music notation, recording and performance. There is no distinction between recording and notating a MIDI performance. Reading the “score” and hearing the playback are almost coextensive as well.
Appealingly intuitive though the MIDI piano roll may be, it still has some shortcomings as a music visualization system. It gives little indication as to the function of the musical events. While it is easy to see the short repeated whole-tone scale figure in the Thelonious Monk composition above, it is not so easy to grasp the broader metrical scheme without doing a great deal of meticulous counting. How might MIDI and other TUBS systems help us see the structure of the music beyond just a series of sequential events?
Meter relates non-adjacent musical events
Most music is organized into repeated rhythmic cycles, and cycles of cycles, and very often cycles of cycles of cycles. We make sense of these cycles using meter, “the grouping of perceived beats or pulses into equivalence classes” (Forth, Wiggin & McLean, 2010). Linear musical concepts like small-scale melodies depend mostly on relationships between adjacent events, or at least closely spaced events. But periodicity and meter depend on relationships between nonadjacent events. Linear representations of music show meter only indirectly. We must count grid lines (or implicit grid lines in standard western notation) in order to understand where in the cycle a particular event lies.
If we wrap the musical timeline into a circle, meter becomes much easier to parse. Metrically related events can be placed adjacently, and their position on the circle can represent their position within the meter. This system works especially well for repetitive, loop-oriented music. This graphic by Forth, Wiggin and McLean (2010) uses circular notation to show different subdivisions of triple meter:
There is some precedent for circular representation of other cyclical music concepts. Pitch class is commonly represented as circular, organized either by semitones or fifths.
Representing cyclical music with a cyclical graph
Benadon (2007) observes that a linear left-to-right orientation tends to conceal the recursive nature of beat-based patterns. He uses circular representations to describe the nuances of a performance’s rhythm and pitch. For example, the graphic below represents Bubber Miley’s trumpet solo on “Creole Love Call” by Duke Ellington:
There is a long historical precedent for radial depictions of rhythm. The Book of Cycles, an Arabic book about rhythm written by Safi al-Din al-Urmawi in 1252, depicts rhythms as circles divided into wedges. Shaded wedges show beat onsets, while unshaded wedges are rests (Toussaint 2004). This notation is remarkably similar to the system I arrived at independently for the Drum Loop.
The fundamental unit of hip-hop and electronic dance music is the loop. This is nowhere more true than in the case of drums. Rhythm patterns in electronic music repeat with little to no variation throughout long passages of the music. In this idiom, a traditional linear visualization of the patterns is not the clearest representation; a circular visualization describes the music more intuitively. The Drum Loop’s radial grid was inspired by circular visualizations of samples and breakbeats that I created using the Polar Coordinates filter in Adobe Photoshop. The image below shows the opening keyboard figure from Herbie Hancock’s “Chameleon.” The figure is repeated identically for a considerable length of time, and the circular representation feels appropriate.
The next image below is a screen capture of the loop from “The Funky Drummer (Bonus Beat Reprise)” by James Brown, loaded into Propellerhead Recycle, and given the same polar transform in Photoshop. As with the Hancock piece, the beat goes through an enormous number of identical repetitions over the course of the track, and again, the circular graphic feels like the most appropriate visualization.
While circular representation is rarely used at the level of entire songs or pieces, a few intriguing examples do exist. One such is the Infinite Jukebox by Paul Lamere (2013). The software uses the Echo Nest API to search for repeated musical elements within a song. Repeated segments are connected by colored arcs. The image below shows “Billie Jean” by Michael Jackson as visualized by the Infinite Jukebox. The software plays the song clockwise around the circle, sometimes jumping across arcs when it encounters them (as seen with the green arc below.) By seamlessly connecting repeated segments of a song, the software algorithmically creates an extended dance remix that, in theory, could go on forever.
Swing is a subtle concept, not easily grasped by beginners, but it is an essential component in dance music drum programming. Drum machines typically represent swing with a knob ranging from 0% (straight eighth notes) to 100% (quarter note/eighth note triplet feel.) Some drum machines, like the Roland TR-808 and its various software emulators, refer to swing as “shuffle.”
Jazz rhythms create a continual feeling of anacrusis (anticipation). Devices like the backbeat, syncopation and swing create metrical tension at multiple levels: the beat, the bar, the phrase, and the section. The continual flow of information provided by swung eighth notes draws focus to the quarter notes by perceptually grouping the shortened upbeat eighth both with its predecessor and the following on-beat. This device strengthens the tactus, making it easy to follow against the disruptive effects of the backbeat and other syncopations. Jazz musicians use rhythmic tension and release to motivate active and participatory listening (Butterfield, 2011).
Swing is rarely visualized in any explicit way. Sequencing software will sometimes show swing by displacing alternate eighth notes on the MIDI piano roll. Most drum machine interfaces do not show swing at all, except by reading a numerical value from the Shuffle knob. Swing is sometimes shown in classical music notation in exaggerated triplet form, but this is not an accurate representation. Jazz and country use more swing than any other idioms, but their practitioners do not notate swing at all; at most, they will make a terse verbal notation on the top of the lead sheet. By and large, swing is implicitly understood more than it is explicitly specified.
The Drum Loop uses a novel (and to my knowledge unprecedented) literal graphical representation of swing. The wedges alternately expand and contract in width according to the amount of swing specified. At 0% swing, the wedges are all of uniform width. At 100% swing, the first eighth note in each pair is twice as long as the second, so the first wedge is twice as wide as the second. As the user adjusts the swing slider, the wedges dynamically change their width accordingly.
Users who have difficulty understanding swing by ear can reinforce their learning visually.