{"id":14982,"date":"2016-10-06T16:58:19","date_gmt":"2016-10-06T20:58:19","guid":{"rendered":"http:\/\/www.ethanhein.com\/wp\/?p=14982"},"modified":"2018-09-08T14:11:44","modified_gmt":"2018-09-08T18:11:44","slug":"qwertybeats-research","status":"publish","type":"post","link":"https:\/\/www.ethanhein.com\/wp\/2016\/qwertybeats-research\/","title":{"rendered":"QWERTYBeats research"},"content":{"rendered":"

Writing assignment for\u00a0Design For The Real World<\/a> with Claire Kearney-Volpe<\/a> and Diana Castro<\/a>\u00a0– research about a new rhythm interface for blind and low-vision novice musicians<\/em><\/p>\n

Definition<\/h2>\n

I propose a new web-based accessible rhythm instrument called QWERTYBeats.<\/p>\n

\"QWERTYBeats<\/a><\/p>\n

Traditional\u00a0instruments are highly accessible to blind and low-vision musicians. Electronic music production tools\u00a0are not. I look at the history of accessible instruments and software interfaces, give an overview of current electronic music hardware and software, and discuss the design considerations underlying my project.\u00a0<\/p>\n

Historical overview<\/h2>\n

Acoustic instruments give rich auditory and haptic feedback, and pose little obstacle to blind musicians. We need look no further for proof than the long history of iconic blind musicians like Ray Charles and Stevie Wonder. Even sighted instrumentalists\u00a0rarely look at their instruments\u00a0once they have attained a sufficient level of proficiency. Music notation is not accessible, but Braille notation has existed since the language’s inception. Also, a great many musicians both blind and sighted play entirely by ear anyway.<\/p>\n

Most of the academic literature around accessibility issues in music education focuses on wider adoption of and support for Braille\u00a0notation. See, for example, Rush, T. W. (2015). Incorporating Assistive Technology for Students with Visual Impairments into the Music Classroom<\/a>. Music Educators Journal<\/i>, 102<\/i>(2), 78\u201383. For electronic music, notation is rarely if ever a factor.<\/p>\n

Electronic instruments pose some new accessibility challenges. They may use graphical interfaces with nested menus, complex banks of knobs and patch cables, and other visual\u00a0control surfaces. Feedback may be given entirely with LED lights and small text labels. Nevertheless, blind users can master these devices with sufficient practice, memorization and assistance. For example, Stevie Wonder has incorporated synthesizers and drum machines in most of his best-known recordings.<\/p>\n

Most electronic music creation is currently done\u00a0not with instruments, but rather using\u00a0specialized software applications called digital audio workstations (DAWs). Keyboards and other controllers are mostly used to access features of the software, rather than as standalone instruments. The most commonly-used DAWs include Avid Pro Tools, Apple Logic, Ableton Live, and Steinberg Cubase. Mobile DAWs are more limited than their desktop counterparts, but are nevertheless becoming robust music creation tools in their own right. Examples include Apple GarageBand and Steinberg Cubasis.\u00a0Notated music is commonly composed using score editing software like Sibelius and Finale, whose functionality increasingly overlaps with DAWs, especially in regard to MIDI sequencing.<\/p>\n

DAWs and notation editors pose steep\u00a0accessibility challenges due to their graphical and spatial interfaces, not to mention their sheer complexity.\u00a0In class, we were given a presentation by Leona Godin<\/a>, a blind musician who\u00a0records and edits audio using Pro Tools by means of VoiceOver. While it must have taken\u00a0a heroic effort on her part to learn the program, Leona\u00a0demonstrates that it is possible. However, some DAWs pose insurmountable problems even to very determined blind users because they do not use standard operating system elements, making them inaccessible via\u00a0screen readers.<\/p>\n

Technological interventions<\/h2>\n

There are no mass-market electronic interfaces specifically geared toward blind or low-vision users. In this section, I discuss one product frequently hailed for its “accessibility” in the colloquial rather than blindess-specific sense, along with some more experimental and academic designs.<\/p>\n

Ableton Push<\/a><\/strong><\/h3>\n

\"Push<\/a><\/p>\n

Ableton Live has become the DAW of choice for electronic music producers. Low-vision users can zoom in to the interface and modify the color scheme. However, Live is inaccessible via screen readers.<\/p>\n

In recent years, Ableton\u00a0has introduced a hardware controller, the Push, which is designed to make the software experience more tactile and instrument-like. The Push combines an eight by eight grid of LED-lit touch pads with banks of knobs, buttons and touch strips.\u00a0It makes it possible to create, perform and record a piece of music from scratch without looking at the computer screen. In addition to drum programming and sampler performance, the Push also has an innovative melodic mode which maps scales onto the grid in such a way that users\u00a0can not play a wrong note.\u00a0Other comparable products exist; see, for example, the Native Instruments Maschine<\/a>.<\/p>\n

There are many pad-based drum machines and samplers. Live’s main differentiator is its Session view, where the pads launch clips:\u00a0segments of audio or MIDI that can vary in length from a single drum hit to the length of an entire song. Clip launching is tempo-synced, so when you trigger a clip, playback is delayed until the start of the next measure (or whatever the quantization interval is.) Clip launching is a forgiving and beginner-friendly performance method, because it removes the possibility of playing something out of rhythm. Like other DAWs,\u00a0Live also gives rhythmic scaffolding in its software instruments by means of arpeggiators, delay and other tempo-synced features.<\/p>\n

The Push is a remarkable interface, but it has some shortcomings for blind users. First of all, it is expensive, $800 for the entry-level version and $1400 for the full-featured software suite. Much of its feedback is visual, in the form of LED screens and color-coded lighting on the pads. It\u00a0switches between multiple modes which\u00a0can be challenging to distinguish even for sighted users. And, like the software it accompanies, the Push is highly complex, with a steep learning curve unsuited to novice users, blind or sighted.<\/p>\n

The aQWERTYon<\/a><\/strong><\/h3>\n

Most DAWs enable users to perform MIDI instruments on the QWERTY keyboard. The most familiar example is the Musical Typing feature in Apple GarageBand.<\/p>\n

\"GarageBand<\/a><\/p>\n

Musical Typing\u00a0makes it possible to play software instruments without an external MIDI controller, which is convenient and useful. However, its layout counterintuively follows the piano keyboard, which is an awkward fit for the computer keyboard. There is no easy way to distinguish the black and white keys, and even expert users find themselves inadvertantly hitting the keyboard shortcut for recording while hunting for F-sharp.<\/p>\n

The aQWERTYon is a web interface developed by the NYU Music Experience Design Lab\u00a0specifically intended to address the shortcomings of Musical Typing.<\/p>\n

\"aQWERTYon<\/a><\/p>\n

Rather than emulating the piano keyboard, the aQWERTYon draws its inspiration from the chord buttons of an accordion. It fills the entire keyboard with harmonically related notes in a way that supports discovery by naive users. Specifically, it maps scales across the rows of keys, staggered by intervals such that each column forms a chord within the scale. Root notes and scales can be set from pulldown menus within the interface, or preset using URL parameters. It\u00a0can be played\u00a0as a standalone instrument, or as a MIDI controller in conjunction with a DAW.<\/p>\n

The aQWERTYon is a completely tactile experience. Sighted users can carefully match keys to note names using the screen, but more typically approach the instrument by feel, seeking out patterns on the keyboard by ear. A blind user would need assistance loading the aQWERTYon initially and setting the scale and root note parameters, but otherwise, it is perfectly accessible. The present project was motivated in large part by a desire to make exploration of rhythm as playful and intuitive as the aQWERTYon makes exploring chords and scales.<\/p>\n

Soundplant<\/strong><\/h3>\n

The QWERTY keyboard can be turned into a\u00a0simple drum machine quite easily using a free program called Soundplant<\/a>. The user simply drags audio files onto a graphical key to have it triggered by that physical key. I was able to create a TR-808 kit in a matter of minutes:<\/p>\n

\"Soundplant<\/a><\/p>\n

After it is set up and configured, Soundplant can be as\u00a0effortlessly accessible as the aQWERTYon. However, it does not give the user any rhythmic assistance.\u00a0Drumming in perfect time is an advanced musical skill, and playing drum machine samples out of time is not much more satisfying than banging on a metal bowl with a spoon out of time. An ideal drum\u00a0interface would offer beginners some of the rhythmic scaffolding and support that Ableton provides via\u00a0Session view, arpeggiators, and the like.<\/p>\n

The Groove Pizza<\/strong><\/a><\/p>\n

Drum machines and their software counterparts offer an alternative form of rhythmic scaffolding. The user sequences patterns in a time-unit box system<\/a> or piano roll<\/a>, and the computer performs those patterns flawlessly. The MusEDLab<\/a>‘s\u00a0Groove Pizza<\/a>\u00a0app is a web-based drum sequencer that wraps the time-unit box system into a circle.<\/p>\n

\"Groove<\/a><\/p>\n

The Groove Pizza was designed to make drum programming more intuitive by visualizing\u00a0the symmetries and patterns inherent in musical-sounding rhythms. However, it is totally unsuitable for blind or low-vision users. Interaction is only possible through the mouse pointer or touch, and there are no standard user interface elements that can be parsed by screen readers.<\/p>\n

Before ever considering designing for the blind, the MusEDLab\u00a0had already considered the Groove Pizza’s limitations for younger children and users with special needs: there is no “live performance” mode, and there is always some delay in feedback between making a change in the drum pattern and hearing the result. We have been considering ways to make a rhythm interface that is more immediate, performance-oriented and tactile. One possible direction would be to create a hardware version of the Groove Pizza; indeed, one of the\u00a0earliest prototypes was a hardware version built by Adam November out of a pizza box. However, hardware design is vastly more complex and difficult than software, so for the time being, software promises more immediate results.<\/p>\n

Haenselmann-Lemelson-Effelsberg MIDI sequencer<\/h3>\n

This experimental interface is described in Haenselmann, T., Lemelson, H., & Effelsberg, W. (2011). A zero-vision music recording paradigm for visually impaired people<\/a>. Multimedia Tools and Applications<\/i>, 5<\/i>, 1\u201319.<\/p>\n

\"Haenselmann-Lemelson-Effelsberg<\/a><\/p>\n

The authors create a new mode for a standard MIDI keyboard that maps piano keys to DAW functions like playback, quantization, track selection, and so on. They also add “earcons” (auditory icons) to give sonic feedback when particular functions have been activated that normally only give graphical feedback. For example, one earcon sounds when recording is enabled; another sounds for regular playback. This interface\u00a0sounds promising, but there are significant obstacles to its adoption. While the authors have released the source code as a free download, that requires a would-be user to be able to compile and run it. This is presuming that they could access the code in the first place; the download link given in the paper is inactive. It is an all-too-common fate of academic projects to never get widespread usage. By posting our projects on the web, the MusEDLab hopes to avoid this outcome.<\/p>\n

Statement<\/h2>\n

Music\u00a0education philosophy<\/strong><\/p>\n

My project is animated by a\u00a0constructivist philosophy of music education, which operates by the following axiomatic assumptions:<\/p>\n