{"id":16091,"date":"2017-05-03T21:45:53","date_gmt":"2017-05-04T01:45:53","guid":{"rendered":"http:\/\/www.ethanhein.com\/wp\/?p=16091"},"modified":"2019-09-08T19:55:42","modified_gmt":"2019-09-08T23:55:42","slug":"designing-a-more-welcoming-aqwertyon-experience","status":"publish","type":"post","link":"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/","title":{"rendered":"Designing a more welcoming aQWERTYon experience"},"content":{"rendered":"<p><em>This post documents my final project\u00a0for User Experience Design with <a href=\"http:\/\/ahnjune.com\/\">June Ahn<\/a><\/em><\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"16105\" data-permalink=\"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/best-aqw-screencap\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/best-aqw-screencap.png?fit=994%2C680&amp;ssl=1\" data-orig-size=\"994,680\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"The best aQWERTYon screencap\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/best-aqw-screencap.png?fit=640%2C438&amp;ssl=1\" class=\"alignnone wp-image-16105 size-full\" src=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/best-aqw-screencap.png?resize=640%2C438&#038;ssl=1\" alt=\"The best aQWERTYon screencap\" width=\"640\" height=\"438\" srcset=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/best-aqw-screencap.png?w=994&amp;ssl=1 994w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/best-aqw-screencap.png?resize=300%2C205&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/best-aqw-screencap.png?resize=768%2C525&amp;ssl=1 768w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/p>\n<h2>Overview of the problem<\/h2>\n<p><a href=\"https:\/\/musedlab.org\/aqwertyon\/\">The aQWERTYon<\/a> is a web-based music performance and theory learning interface designed by the <a href=\"http:\/\/musedlab.org\">NYU Music Experience Design Lab<\/a>. The name is a play on &#8220;<a href=\"https:\/\/en.wikipedia.org\/wiki\/QWERTY\">QWERTY<\/a> <a href=\"https:\/\/en.wikipedia.org\/wiki\/Accordion\">accordion<\/a>.&#8221; The aQWERTYon invites novices to improvise and compose using a variety of scales and chords normally available only to advanced musicians. Notes map onto the computer keyboard such that the rows play scales and the columns play chords. The user can not play any wrong notes, which encourages free and playful exploration. The aQWERTYon has a variety of instrument sounds to choose from, and it can also act as a standard MIDI controller for digital audio workstations (DAWs) like GarageBand, Logic, and Ableton Live. As of this writing, there have been over 32,000 aQWERTYon sessions.<\/p>\n<p><!--more-->One of our core design principles is to work within our users\u2019 real-world technological limitations. We build tools in the browser so they will be platform-independent and accessible anywhere where there is internet access. Our aim with the aQWERTYon was to find the musical possibilities in a typical computer with no additional software or hardware. That question led us to investigate ways of turning the standard QWERTY keyboard into a beginner-friendly instrument.<\/p>\n<p>While the aQWERTYon has been an effective tool in classrooms and online, it has some design deficiencies as well. It is difficult for\u00a0unassisted users to figure out what the app is for. While its functionality is easily discovered through trial and error, its musical applications are less self-explanatory. Some of this is due to the intrinsic complexity of music theory and all the daunting terminology that comes with it. But some of it is the lack of context and guidance we provide to new users.<\/p>\n<h2>The conjecture<\/h2>\n<p>This assignment coincided with discussions already taking place in the lab around redesigning the aQW. Many of those focused on a particular element of the user interface, the scale picker.<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aQWERTYon-scale-picker.png?ssl=1\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"16115\" data-permalink=\"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/aqwertyon-scale-picker\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aQWERTYon-scale-picker.png?fit=361%2C217&amp;ssl=1\" data-orig-size=\"361,217\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"aQWERTYon scale picker\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aQWERTYon-scale-picker.png?fit=361%2C217&amp;ssl=1\" class=\"alignnone size-full wp-image-16115\" src=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aQWERTYon-scale-picker.png?resize=361%2C217&#038;ssl=1\" alt=\"aQWERTYon scale picker\" width=\"361\" height=\"217\" srcset=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aQWERTYon-scale-picker.png?w=361&amp;ssl=1 361w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aQWERTYon-scale-picker.png?resize=300%2C180&amp;ssl=1 300w\" sizes=\"auto, (max-width: 361px) 100vw, 361px\" \/><\/a><\/p>\n<p>The user has a variety of scales to choose from, ranging from the familiar to the exotic. However, these scales all have impenetrable names. How are\u00a0music theory novices supposed to make sense of names like\u00a0harmonic minor or Lydian\u00a0mode?\u00a0How would they know to choose one scale or another? We debated the least off-putting way of presenting these choices: should we represent them graphically? Associate each one with a well-known piece of music? Or just list them alphabetically? I proposed a system of graphical icons showing the notes comprising each scale. While novices will find them no more intelligible than the names, the hope is that they would be sufficiently visually appealing to invite users to explore them by ear. You can read\u00a0<a href=\"https:\/\/ethanhein.com\/wp\/2017\/the-aqwertyon-pitch-wheels-and-the-future-of-music-theory-visualization\/\">a detailed explanation of the scale icons here<\/a>.<\/p>\n<p><a href=\"http:\/\/ethanhein.com\/scale_picker_test\/scale_picker.html\" target=\"_blank\" rel=\"noopener noreferrer\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"16124\" data-permalink=\"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/scale_picker\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/scale_picker.jpg?fit=659%2C741&amp;ssl=1\" data-orig-size=\"659,741\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;1&quot;}\" data-image-title=\"aQW scale picker interactive wireframe\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/scale_picker.jpg?fit=640%2C720&amp;ssl=1\" class=\"alignnone wp-image-16124 size-full\" src=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/scale_picker.jpg?resize=640%2C720&#038;ssl=1\" alt=\"aQW scale picker interactive wireframe\" width=\"640\" height=\"720\" srcset=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/scale_picker.jpg?w=659&amp;ssl=1 659w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/scale_picker.jpg?resize=267%2C300&amp;ssl=1 267w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/a><\/p>\n<p>Conversations with June helped me understand that there are some broader and more profound user experience\u00a0problems to solve before users ever get to the scale picker.\u00a0What is the experience of simply landing on the app for the first time? How do people know what to do? From this conversation came the germ of a new idea, a landing page offering a tutorial or introduction. We want users to have a feeling of discovery, a musical\u00a0\u201caha moment\u201d,\u00a0the chance to\u00a0be\u00a0a musical insider. The best way to do that seemed to be to\u00a0give\u00a0users a playlist of preset songs to jam with.<\/p>\n<h2>User characteristics and personas<\/h2>\n<p>There are three major user groups for the aQWERTYon, who I will describe as <em>students,<\/em> <em>teachers,<\/em> and <em>explorers.<\/em><\/p>\n<h4>Students and teachers<\/h4>\n<p>Students use the aQW in a guided and structured setting: a classroom, a private lesson, or an online tutorial. There are several\u00a0distinct user personas:\u00a0elementary, middle and high school students, both mainstream and with special needs;\u00a0college students; and <a href=\"https:\/\/soundfly.com\/series\/theory-for-producers\">online learners<\/a>, mostly adults. Each student persona has its corresponding teacher persona. For example, I use the aQW\u00a0with my music technology students at Montclair State University and NYU, and with some private students.<\/p>\n<p>The aQW&#8217;s biggest fan is MusEDLab partner Matt McLean, who teaches at the <a href=\"https:\/\/www.lrei.org\/\">Little Red Schoolhouse<\/a>\u00a0and runs a nonprofit organization called the <a href=\"http:\/\/www.yciw.net\/1\/\">Young Composers and Improvisors Workshop<\/a>. Matt\u00a0uses the aQW to teach composition in both settings,\u00a0in person and online. He has documented his students\u2019 use of the aQW\u00a0extensively. Some examples:<\/p>\n<ul>\n<li><a href=\"https:\/\/vimeo.com\/205386136\">Improvisation over beats<\/a> generated in the <a href=\"https:\/\/musedlab.org\/groovepizza\">Groove Pizza<\/a><\/li>\n<li><a href=\"https:\/\/vimeo.com\/191100496\">Performance<\/a> with <a href=\"http:\/\/noteflight.com\">Noteflight<\/a><\/li>\n<li><a href=\"https:\/\/vimeo.com\/187478427\">Chord melody activity<\/a><\/li>\n<li><a href=\"https:\/\/vimeo.com\/184071917\">Playing chord inversions<\/a><\/li>\n<li><a href=\"https:\/\/vimeo.com\/170222545\">Sixth graders\u2019 compositions<\/a> created in a single class period<\/li>\n<\/ul>\n<h4>Explorers<\/h4>\n<p>I use the term explorers to describe\u00a0people\u00a0who use the aQW without any outside guidance. Explorers do not fit into specific demographic groups, but they center around two broad, overlapping personas:\u00a0bedroom producers and\u00a0music theory autodidacts. Explorers may find the aQW\u00a0via a link, a social media\u00a0posting, or a Google search. We know little about these users beyond what is captured by Google Analytics. However, we can make some assumptions based on our known referral sources. For example, this\u00a0blog is a significant driver of traffic to the aQW. I have numerous posts on music theory and composition that link to the aQW so that readers can explore the concepts for themselves. My blog readership includes other music educators and some professional musicians, but the majority are amateur musicians and very enthusiastic listeners. These are exactly the users we are trying to serve: people who want to learn about music independently, either for creative purposes or to simply satisfy curiosity.<\/p>\n<p>While I am a music educator, I have spent most of my life as a self-taught bedroom producer, so I identify naturally with the explorers. I have created several original pieces of music with the aQW, both for user testing purposes and to show its creative potential. While I have an extensive music theory background, I am a rudimentary keyboard player at best. This has limited my electronic music creation to drawing in the MIDI piano roll with the mouse pointer, since I can not perform my ideas on a piano-style controller. The aQW suits my needs perfectly, since I can set it to any scale I want and shred fearlessly.\u00a0My hope is that more would-be explorers feel invited to use the aQW for similar creative purposes in their own performance and composition.<\/p>\n<h2>Tasks and Scenarios<\/h2>\n<p>It is possible to configure the aQWERTYon\u00a0via URL parameters to set the key and scale, and to hide components of the user interface. When teachers create exercises or assignments, they can link or embed the aQW\u00a0with its settings locked to keep students from getting lost or confused. However, this does not necessarily invite the user to explore or experiment. <a href=\"https:\/\/apps.musedlab.org\/aqwertyon\/?source=pub&amp;session=none&amp;clone=true&amp;controls=false&amp;layout=1-3-5-7&amp;scale=harmonic-major&amp;root=64&amp;video=4m1EFMoRFvY&amp;sound=lead_1_square&amp;override=true\">Here is an example of an aQW preset to accompany\u00a0a Beyonc\u00e9 song<\/a>. This preset\u00a0might be used for a variety of pedagogical tasks, including learning some or all of the melody, creating a new countermelody, or improvising a solo. The <a href=\"https:\/\/en.wikipedia.org\/wiki\/Harmonic_major_scale\">harmonic major scale<\/a> is not one that is\u00a0usually taught, but it a useful way to blend major and minor tonalities. Students might try using more standard scales like major or harmonic minor, and listen for ways that they clash with Beyonc\u00e9&#8217;s song.<\/p>\n<p>Tasks and scenarios for explorers might include creating a melody, bassline or chords for an original piece of music. For example, a self-taught dance music producer might feel limited by the scales that are easiest to play on a piano-style keyboard (major, natural minor, pentatonics), and might be in search of richer and more exotic sounds. This producer might play their\u00a0track in progress and improvise on top using different scale settings.<\/p>\n<p>One of the users I tested with suggested an alternative explorer use case. He is an enthusiastic amateur composer and arranger, who is trying to arrange choral versions of pop and rock songs. He is a guitarist who has little formal music theory knowledge. He might use the aQW to try out harmonic ideas by ear, write down note names that form pleasing combinations, and then transfer them to\u00a0the guitar or piano-based MIDI controller.<\/p>\n<h2>Understanding the problem<\/h2>\n<p>In the age of the computer and the internet, many aspects of music performance, composition and production are easy to self-teach. However, music theory remains an obstacle for bedroom producers and pop musicians (not to mention schooled musicians!)\u00a0There are so many chords and scales and rules and technical vocabulary, all of which have to be applied in all twelve keys.\u00a0To make matters worse, terminology hangs around long after its\u00a0historical context has disappeared:<\/p>\n<ul>\n<li>We no longer know what the Greek modes sound like, but we use their names to describe modern scales.<\/li>\n<li>C-sharp and D-flat\u00a0were different pitches in historical tuning systems, but now both names describe the same pitch.<\/li>\n<li>The harmonic and melodic minor scales are named after a stylistic rule for writing\u00a0melodies that\u00a0was abandoned hundreds of years ago.<\/li>\n<\/ul>\n<p>And so on.<\/p>\n<p>Most existing theory resources draw on the Western classical tradition, using examples and conventions from a repertoire most contemporary musicians and listeners find unfamiliar. Furthermore, these resources presume the ability to read standard music notation. Web resources that do address popular music are usually confusing and riddled with errors. I have worked with <a href=\"https:\/\/soundfly.com\">Soundfly<\/a> to fill this vacuum by creating <a href=\"https:\/\/soundfly.com\/series\/theory-for-producers\">high-quality online courses<\/a> <a href=\"https:\/\/soundfly.com\/courses\/how-to-create-emotional-impact-with-chords\">aimed at popular musicians<\/a>. Even with the best teaching resources, though, theory remains daunting. Exploring different chords and scales on an instrument requires significant\u00a0technical mastery, and many musicians give up before ever reaching that point.<\/p>\n<p>The aQW is intended to ease music theory learning by making scales and chords easy to discover, even by complete novices. Our expectation is that after explorers\u00a0are\u00a0able to\u00a0try theory ideas out in a low-pressure and creative setting, they will be motivated to put these ideas to work while playing instruments, composing, or producing. Alternatively, users can simply perform and compose directly with the aQW itself.<\/p>\n<h2>Social and technical\u00a0context<\/h2>\n<p>Most computer-based melody input systems are modeled on the piano. This is most obvious for hardware, since nearly all MIDI controllers take the form of literal piano keyboards. It is also true for software, which takes the piano keyboard as the primary visualization scheme for pitch. For example,\u00a0the MIDI editor in every DAW displays pitches on a &#8220;piano roll&#8221;.<\/p>\n<p>Some DAWs\u00a0include a \u201cmusical typing\u201d feature that maps the piano layout to the QWERTY keyboard, as an expediency for users who either lack MIDI hardware controllers, or who do not have them on hand. Apple\u2019s GarageBand uses the ASDFG row of the keyboard for the white keys and the QWERTY row for the black keys. GarageBand uses the other rows for such useful controls as pitch bend, modulation, sustain, octave shifting and simple velocity control.<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"13994\" data-permalink=\"https:\/\/www.ethanhein.com\/wp\/2016\/inside-the-aqwertyon\/garageband-musical-typing\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2016\/02\/GarageBand-musical-typing.png?fit=761%2C369&amp;ssl=1\" data-orig-size=\"761,369\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"GarageBand musical typing\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2016\/02\/GarageBand-musical-typing.png?fit=640%2C310&amp;ssl=1\" class=\"alignnone size-full wp-image-13994\" src=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2016\/02\/GarageBand-musical-typing.png?resize=640%2C310&#038;ssl=1\" alt=\"GarageBand musical typing\" width=\"640\" height=\"310\" srcset=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2016\/02\/GarageBand-musical-typing.png?w=761&amp;ssl=1 761w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2016\/02\/GarageBand-musical-typing.png?resize=300%2C145&amp;ssl=1 300w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/p>\n<p>Useful and expedient though it is, Musical Typing has some grave shortcomings as a user interface. It presumes familiarity with the piano keyboard, but it is not very playable\u00a0for\u00a0users do who possess that familiarity. The piano layout also makes a poor fit for the grid of computer keys. For example, there is no black key on the piano between the notes E and F, but the QWERTY keyboard gives no visual reminder of that fact, so it is necessary to just remember it. Unfortunately, the \u201cmissing\u201d black key between E and F happens to be the letter R, which is GarageBand\u2019s keyboard shortcut for recording. While hunting for E-flat or F-sharp, users are prone to accidentally start recording over their work. I have been using GarageBand for seven years and still do this routinely.<\/p>\n<p>Ableton\u2019s Push controller represents an interesting break with MIDI controller orthodoxy. It is a grid of 64 touch pads surrounded by various buttons, knobs and sliders.<\/p>\n<p><a href=\"https:\/\/www.ableton.com\/en\/push\/\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"16116\" data-permalink=\"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/push\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/push.jpg?fit=1440%2C805&amp;ssl=1\" data-orig-size=\"1440,805\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Ableton Push\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/push.jpg?fit=640%2C358&amp;ssl=1\" class=\"alignnone size-large wp-image-16116\" src=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/push-1024x572.jpg?resize=640%2C358&#038;ssl=1\" alt=\"Ableton Push\" width=\"640\" height=\"358\" srcset=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/push.jpg?resize=1024%2C572&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/push.jpg?resize=300%2C168&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/push.jpg?resize=768%2C429&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/push.jpg?w=1440&amp;ssl=1 1440w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/push.jpg?w=1280&amp;ssl=1 1280w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/a><\/p>\n<p>The pads were designed to trigger samples and loops like a typical drum machine, but Ableton also includes a melody mode for the Push. By default, it maps notes to the grid in rows staggered by fourths, which makes the layout identical to the bottom four strings of the guitar. This is quite a gift for guitarists like me, since I can use my familiar chord and scale fingerings, rather than hunting and pecking for them on the piano. Furthermore, the Push can be set so that the pads play only the notes within a particular scale, giving a \u201cno wrong notes\u201d experience similar to the aQWERTYon. Delightful though this mode is, however, it is imperfect. Root notes of the scale are colored blue, and other notes are colored white. While this makes the roots easy to distinguish, it is not so easy to visually differentiate the other pitches.<\/p>\n<p>Touchscreen devices like the iPhone and iPad open up additional new possibilities for melodic interfaces. Many mobile apps continue to use the piano keyboard for note input, but some take advantage of the touchscreen\u2019s unique affordances. One such is <a href=\"http:\/\/thumbjam.com\/\">Thumbjam<\/a>, which enables the user to divide the screen into slices of arbitrary thickness that can map to any arbitrary combination of notes.<\/p>\n<p><a href=\"http:\/\/thumbjam.com\/\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"16117\" data-permalink=\"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/thumbjam-split-screen\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/thumbjam-split-screen.jpg?fit=2048%2C1536&amp;ssl=1\" data-orig-size=\"2048,1536\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;1&quot;}\" data-image-title=\"Thumbjam\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/thumbjam-split-screen.jpg?fit=640%2C480&amp;ssl=1\" class=\"alignnone size-large wp-image-16117\" src=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/thumbjam-split-screen-1024x768.jpg?resize=640%2C480&#038;ssl=1\" alt=\"Thumbjam\" width=\"640\" height=\"480\" srcset=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/thumbjam-split-screen.jpg?resize=1024%2C768&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/thumbjam-split-screen.jpg?resize=300%2C225&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/thumbjam-split-screen.jpg?resize=768%2C576&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/thumbjam-split-screen.jpg?w=2048&amp;ssl=1 2048w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/thumbjam-split-screen.jpg?w=1280&amp;ssl=1 1280w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/thumbjam-split-screen.jpg?w=1920&amp;ssl=1 1920w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/a><\/p>\n<p>The app offers hundreds of preset scales to choose from. The user may have a small range of notes, each of which is large and easy to distinguish, or a huge range of notes, each of which occupies a narrow strip of screen area. Furthermore, the screen can be split to hold four different scales, played from four different instruments. While all of this configurability is liberating, it is also overwhelming. Also, the scales are arranged along one-dimensional lines; there is no easy way to play chords and arpeggios.<\/p>\n<h2>Evaluation criteria<\/h2>\n<p>Is the aQW\u2019s potential obvious enough to draw in explorers and educators? Will it be adopted as a tool for self-teaching? Does it invite playful exploration and experimentation? Is it satisfying for real-world musical usage? Is the UI self-explanatory, or at least discoverable? Is the music theory content discoverable? Have we identified the right user persona(s)? Is the aQW really a tool for beginners? Or is it an intermediate music theory learning tool? Or an advanced composition tool? Is\u00a0the approach of a &#8220;playlist&#8221;\u00a0of\u00a0example songs the right one? Which\u00a0songs, artists and genres should we include on the landing page? How many presets should we include? Should we limit it to a few, or should we offer a large, searchable database? And how\u00a0do we deal with the fact that many songs require multiple scales to play?<\/p>\n<h2>Proposed solution<\/h2>\n<p>I tested several interactive wireframes of this landing page\u00a0concept. Click the image to try it yourself:<\/p>\n<p><a href=\"http:\/\/ethanhein.com\/aqw_test\/aqw_landing.html\" target=\"_blank\" rel=\"noopener noreferrer\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"16103\" data-permalink=\"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/aqw_landing\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aqw_landing.jpg?fit=1366%2C774&amp;ssl=1\" data-orig-size=\"1366,774\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;1&quot;}\" data-image-title=\"aQWERTYon landing page interactive wireframe\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aqw_landing.jpg?fit=640%2C363&amp;ssl=1\" class=\"alignnone wp-image-16103 size-large\" src=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aqw_landing-1024x580.jpg?resize=640%2C362&#038;ssl=1\" alt=\"aQWERTYon landing page interactive wireframe\" width=\"640\" height=\"362\" srcset=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aqw_landing.jpg?resize=1024%2C580&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aqw_landing.jpg?resize=300%2C170&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aqw_landing.jpg?resize=768%2C435&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aqw_landing.jpg?w=1366&amp;ssl=1 1366w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/aqw_landing.jpg?w=1280&amp;ssl=1 1280w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/a><\/p>\n<p>The first wireframe had\u00a0nine\u00a0preset songs. I wanted to\u00a0offer reasonable musical diversity without overwhelming the user. The tenth\u00a0slot linked to the &#8220;classic&#8221; aQW, where users\u00a0are free to select their own video, scale, root, and so on. I chose songs that appealed to me (and presumably other adult explorers), along with some current pop songs familiar to younger users. I wanted to balance the choices by race, gender, era, and genre. I was also bound by a musical constraint: all songs need to be playable using a single scale in a single key. The initial preset list was:<\/p>\n<ul>\n<li>Adele &#8211; \u201cSend My Love (To Your New Lover)\u201d<\/li>\n<li>Mary J Blige &#8211; \u201cFamily Affair\u201d<\/li>\n<li>Miles Davis &#8211; \u201cSssh\/Peaceful\u201d<\/li>\n<li>Missy Elliott &#8211; \u201cGet Ur Freak On\u201d<\/li>\n<li>Bj\u00f6rk &#8211; \u201cAll Is Full Of Love\u201d<\/li>\n<li>Michael Jackson &#8211; \u201cDon&#8217;t Stop \u2019Til You Get Enough\u201d<\/li>\n<li>Katy Perry &#8211; \u201cTeenage Dream\u201d<\/li>\n<li>AC\/DC &#8211; \u201cBack In Black\u201d<\/li>\n<li>Daft Punk &#8211; \u201cGet Lucky\u201d<\/li>\n<\/ul>\n<p>After a few test sessions, it became apparent that no one was\u00a0clicking Mary J Blige. Also, the list did not include any current hip-hop. I therefore replaced Mary with Chance The Rapper. I initially offered a few sentences of instruction, but\u00a0feedback from my MusEDLab colleagues encouraged me to reduce the prompt down to\u00a0just a few words: &#8220;Pick a song, type, jam.&#8221;<\/p>\n<p>Further testing showed that while adults are willing to try out any song, familiar or not, children and teens are much choosier. Therefore, I added two more presets,\u00a0\u201cHotline Bling\u201d by Drake and\u00a0\u201cFormation\u201d by Beyonc\u00e9. The latter song proved problematic, however, because its instrumental backing is so sparse and minimal that it is difficult to hear how other notes might fit into it. I ultimately swapped it for &#8220;Single Ladies.&#8221; I had rejected this song initially, because it uses the idiosyncratic harmonic major scale. However, I came to see this quirk as a positive bonus: since one of our goals is to encourage users to explore new sounds and concepts, a well-known and well-loved song using an unusual scale is a rare gift.<\/p>\n<h2>User testing protocol<\/h2>\n<p>I used a think-aloud protocol, asking testers to narrate their thought processes as they explored the app. I recorded the one-on-one sessions using Screenflow. When testing with groups of kids, this was impractical, so instead I\u00a0took\u00a0notes during and after each session. For each user, I opened the\u00a0interactive wireframe, and told them, \u201cThis is a web based application for playing music with your computer keyboard. I\u2019m going to ask you to tell me what you see on the screen, what you think it does, and what you think will happen when you click things.\u201d I did not\u00a0offer any other explanation or context, because I wanted to see whether the landing page was self-explanatory and discoverable. I conducted informal interviews with users during and after the sessions as well.<\/p>\n<h2>User testing results<\/h2>\n<p>I tested with ten adults and approximately forty children. The adults ranged in age from early twenties to fifties. All were musicians, at varying levels of ability and training,\u00a0mostly enthusiastic amateurs. Sessions lasted for twenty or thirty minutes.\u00a0There were two groups of children: a small group of eighth graders at the Little Red Schoolhouse, and a large group of fourth graders from PS 3 who were visiting NYU. These testing sessions were shorter, ten to fifteen minutes each.<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"16099\" data-permalink=\"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/ps3-user-testing\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/PS3-user-testing.jpg?fit=3215%2C1874&amp;ssl=1\" data-orig-size=\"3215,1874\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;2.4&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;iPhone 5&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;1493028260&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;4.12&quot;,&quot;iso&quot;:&quot;50&quot;,&quot;shutter_speed&quot;:&quot;0.025&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;1&quot;}\" data-image-title=\"User testing the aQWERTYon with fourth graders\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/PS3-user-testing.jpg?fit=640%2C373&amp;ssl=1\" class=\"alignnone size-full wp-image-16099\" src=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/PS3-user-testing.jpg?resize=640%2C373&#038;ssl=1\" alt=\"User testing the aQWERTYon with fourth graders\" width=\"640\" height=\"373\" srcset=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/PS3-user-testing.jpg?w=3215&amp;ssl=1 3215w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/PS3-user-testing.jpg?resize=300%2C175&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/PS3-user-testing.jpg?resize=768%2C448&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/PS3-user-testing.jpg?resize=1024%2C597&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/PS3-user-testing.jpg?w=1280&amp;ssl=1 1280w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/PS3-user-testing.jpg?w=1920&amp;ssl=1 1920w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/p>\n<h4>Discovering melodies<\/h4>\n<p>It is possible to play the aQW by clicking the notes onscreen using the mouse, though this method is slow and difficult. Nevertheless, a number of the younger testers did this, even after I suggested that it would be easier on the keyboard.<\/p>\n<p>An adult tester with some keyboard and guitar experience told me, \u201cThis is great, it\u2019s making me play patterns that I normally don\u2019t play.\u201d He was playing on top of the Miles Davis track, and he was quickly able to figure\u00a0out a few riffs from Miles&#8217; trumpet solo.<\/p>\n<h4>Discovering chords<\/h4>\n<p>Several testers systematically identified chords by playing alternating notes within a row, while others discovered them by holding down random groups of keys. None of the testers discovered that they could easily play chords using columns of keys until I prompted them to do so. One even asked, \u201cIs there a relationship between keys if I play them vertically? I don\u2019t know enough about music to know that.\u201d After I suggested he try the columns, he said, \u201cIf I didn\u2019t know [by ear] how chords worked, I\u2019d miss the beauty of this.\u201d He compared the aQW to GarageBand&#8217;s musical typing: \u201cThis is not that. This is a whole new thing. This is chord oriented. As a guitarist, I appreciate that.\u201d The message is clear: we need to make the chords more obvious, or more actively assist users in finding them.<\/p>\n<h4>Other theory issues<\/h4>\n<p>For the most part, testers were content to play the scales they were given, though some of the more expert musicians changed the scales before even listening to the presets. However, not everyone realized that the presets were set to match the song.\u00a0A few asked me: \u201cHow do I know what key this song is in?\u201d\u00a0We could probably state explicitly that the presets line up automatically.<\/p>\n<p class=\"p1\">In general, adult testers found the\u00a0value of the aQW as a theory learning tool to be\u00a0immediately apparent. One told me: \u201cIf I had this when I was a kid, I would have studied music a lot. I used to hate music theory. I learned a lot of stuff, but the learning process was awful\u2026 Your kids\u2019 generation will learn music like this (snaps fingers).&#8221;<\/p>\n<h4>Sounds<\/h4>\n<p>The aQW comes with a large collection of <a href=\"https:\/\/en.wikipedia.org\/wiki\/SoundFont\">SoundFonts<\/a>, and users of all ages enjoyed auditioning them, sometimes for long periods of time.\u00a0Sometimes they apologized for how fascinating they found the sounds to be. But it is remarkable to have access to so many instrument timbres so effortlessly. Computers\u00a0turn us all into potential\u00a0orchestrators, arrangers, and sound designers.<\/p>\n<h4>Screen layout<\/h4>\n<p>The more design-oriented testers appreciated the sparseness and minimalism of the graphics, finding them calming and easy to understand.<\/p>\n<p>Several testers complained that the video window takes up too much screen real estate, and is placed too prominently. Two commented that videos showing live performers, like &#8220;Back In Back,&#8221; were valuable because that helped with timekeeping and inspiration. Otherwise, however, testers found the videos to either be of little value or actively distracting. One suggested having the videos hidden or minimized by default, with the option to click to expand them. Others requested that the video be below the keyboard and other crucial controls. Also, the eighth graders reported that some of the video content was distracting because of its content, for example the partying shown in &#8220;Teenage Dream.&#8221; Unsuitable content\u00a0will be an ongoing issue using many of the pop songs that kids like.<\/p>\n<h4>Technical browser issues<\/h4>\n<p>Having the aQWERTYon run in the browser has significant benefits, but a few limitations as well. Because the URL updates every time the parameters change, clicking the browser&#8217;s Back button does not produce the expected behavior&#8211;it might take ten or fifteen clicks to actually return to a previous page. I changed the links in later versions so each one opens the aQW in a new tab so the landing page would always be available. However, web audio is very memory-intensive, and the aQW will function slowly or not at all if it is open in more than one tab simultaneously.<\/p>\n<h4>Song choices<\/h4>\n<p>The best mix of presets is always going to depend on the specific demographics of any given group of users. However, the assortment I arrived at was satisfying enough for the groups I tested with. Miles Davis and Bj\u00f6rk do not have the wide appeal of Daft Punk or Michael Jackson, but their presence was very gratifying for the more hipster-ish testers.\u00a0I was extremely impressed that\u00a0an eighth grader selected the Miles song, though this kid turns out to be the\u00a0son of a Very Famous Musician and is not typical.<\/p>\n<h4>Recording functionality<\/h4>\n<p>Testers repeatedly requested\u00a0the ability to record their playing. The aQW did start out with a very primitive recording feature, but it will require some development to make it usable. The question is always, how much functionality is enough? Should users be able to overdub? If so, how many tracks? Is simple recording enough, or would users need to able to mix, edit, and select takes?<\/p>\n<p>One reason that recording has been a low development priority is that users can easily record their performances via MIDI into any DAW or notation program. The aQW behaves as if it were a standard MIDI controller plugged into the computer. With so many excellent DAWs in the world, it seems less urgent for us to replicate their functionality. However, there is one major limitation of recording this way: it captures the notes being played, but not the sounds. Instead, the DAW plays back the MIDI using whatever software instruments it has available. Users who\u00a0are attached to a specific SoundFont cannot record them unless they use a workaround like <a href=\"https:\/\/soundflower.en.softonic.com\/mac\">Soundflower<\/a>. This issue will require more discussion and design work.<\/p>\n<h2>New conjectures and future work<\/h2>\n<p>One of my most significant user testers for the landing page wireframe was Kevin Irlen, the MusEDLab&#8217;s chief software architect and main developer of the aQW itself. He found the landing page concept\u00a0sufficiently inspiring that he created a more sophisticated version of it, the\u00a0app sequencer:<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"16107\" data-permalink=\"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/app-sequencer-v1\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/app-sequencer-v1.png?fit=1161%2C682&amp;ssl=1\" data-orig-size=\"1161,682\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"aQWERTYon app sequencer v1\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/app-sequencer-v1.png?fit=640%2C376&amp;ssl=1\" class=\"alignnone wp-image-16107 size-large\" src=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/app-sequencer-v1-1024x602.png?resize=640%2C376&#038;ssl=1\" alt=\"aQWERTYon app sequencer v1\" width=\"640\" height=\"376\" srcset=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/app-sequencer-v1.png?resize=1024%2C602&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/app-sequencer-v1.png?resize=300%2C176&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/app-sequencer-v1.png?resize=768%2C451&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/app-sequencer-v1.png?w=1161&amp;ssl=1 1161w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/p>\n<p>We can add presets to the app sequencer using a simple web form, which is a significant improvement over the tedious process of creating my\u00a0wireframes by hand. The sequencer pulls images automatically from YouTube, another major labor-saver. Kevin also added\u00a0a comment field, which gives additional opportunity to give prompts and instructions.\u00a0Each sequencer preset generates a unique URL, making it possible to generate any number of different landing pages. We will be able to\u00a0create custom landing pages focusing on different artists, genres or themes.<\/p>\n<h4>Songs beyond the presets<\/h4>\n<p>Testing with the fourth graders showed that we will need to design a better system for users who want to play over songs that we do not include among the presets. That tutorial needs to instruct users how to locate YouTube URLs, and more dauntingly, how to identify keys and scales. I propose an overlay or popup:<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"16102\" data-permalink=\"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/any-video-tutorial\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/any-video-tutorial.png?fit=732%2C527&amp;ssl=1\" data-orig-size=\"732,527\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"aQWERTYon &amp;#8220;play over any video&amp;#8221; tutorial\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/any-video-tutorial.png?fit=640%2C461&amp;ssl=1\" class=\"alignnone wp-image-16102 size-full\" src=\"https:\/\/i0.wp.com\/ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/any-video-tutorial.png?resize=640%2C461&#038;ssl=1\" alt=\"aQWERTYon &quot;play over any video&quot; tutorial\" width=\"640\" height=\"461\" srcset=\"https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/any-video-tutorial.png?w=732&amp;ssl=1 732w, https:\/\/i0.wp.com\/www.ethanhein.com\/wp\/wp-content\/uploads\/2017\/05\/any-video-tutorial.png?resize=300%2C216&amp;ssl=1 300w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/p>\n<h4>Keyfinding<\/h4>\n<p>Testing with fourth graders also showed that helping novice users with\u00a0keyfinding may not be as challenging as I had feared. The aQW defaults to the D minor pentatonic scale, and that scale turns out to fit fairly well over most current pop songs. If it doesn&#8217;t, some other minor pentatonic scale is very likely to work. This is due to a music-theoretical quirk of the pentatonic scale: it happens to share pitches with many other commonly-used scales and chords. As long as the root is somewhere within the key, the minor pentatonic will sound fine. For example, in C major:<\/p>\n<ul>\n<li>C minor pentatonic sounds like C\u00a0blues<\/li>\n<li>D minor pentatonic sounds like Csus4<\/li>\n<li>E minor pentatonic sounds like Cmaj7<\/li>\n<li>F minor pentatonic sounds like C natural minor<\/li>\n<li>G minor pentatonic sounds like C7sus4<\/li>\n<li>A minor pentatonic is the same as\u00a0C major pentatonic<\/li>\n<li>B minor pentatonic sounds like C Lydian mode<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p>We are planning to revamp the root picker to show both a larger piano keyboard and a pitch wheel. We also plan to add more dynamic visualization options for notes as they are played, including a staff notation view, the chromatic circle, and the circle of fifths. The aQW leaves several keys on the keyboard unused, and we could use them for additional controls. For example, we might use the\u00a0Control key to make note velocities\u00a0louder, and\u00a0Option to make them quieter. The arrow keys might be used to cycle through the scale menu and to shift the root.<\/p>\n<h4 class=\"p1\">Built-in theory pedagogy<\/h4>\n<p class=\"p1\">There is a great deal of opportunity to build more theory pedagogy on top of the aQW, and to include more of it within the app itself. We might encourage chord playing by automatically showing\u00a0chord labels at the top of each column.\u00a0We might include popups or links next to each scale giving some explanation of\u00a0why they sound the way they do, and\u00a0to give some suggested musical uses. One user proposes a\u00a0game mode for more advanced users, where the scale is set to chromatic and players must identify the &#8220;wrong&#8221; or outside notes. Another proposes a mode similar to <a href=\"https:\/\/www.hooktheory.com\">Hooktheory<\/a>, where users could sequence\u00a0chord progressions to play on top of.<\/p>\n<h4 class=\"p1\">Rhythmic assistance<\/h4>\n<p class=\"p1\">A few testers requested some kind of\u00a0help or guidance with timekeeping. One suggested a graphical score in the style of Guitar Hero, or a &#8220;follow the bouncing ball&#8221; rhythm visualization.\u00a0Another pointed out that an obvious solution would be to incorporate the Groove Pizza, perhaps in miniature form in a corner of the screen. Synchronizing\u00a0all of this to YouTube videos would need to be done by hand, so far as I know, but perhaps an automated solution exists. Beat detection is certainly\u00a0an easier MIR challenge than key or chord detection. If we were able to automatically sync to the tempo of a song, we could\u00a0add the DJ functionality requested by one tester, letting users add cue points, loop certain sections, and slow them down.<\/p>\n<h4 class=\"p1\">Odds and ends<\/h4>\n<p class=\"p1\">One eighth grader suggested that we make aQW accounts with \u201cmusical passwords.\u201d<\/p>\n<p class=\"p1\">An adult tester referred to the landing page as the \u201cChoose Your Own Adventure screen.\u201d The idea of musical adventure is exactly the\u00a0feeling I was hoping for.<\/p>\n<p class=\"p1\">In addition to notes on the staff, one tester requested a spectrum visualizer. This is perhaps an esoteric request, but real-time spectrograms are quite intuitive and might be useful.<\/p>\n<p class=\"p1\">Finally, one tester made a comment that was striking in its broader implications for music education: \u201cI\u2019m not very musical, I don\u2019t really play an instrument, so these kinds of tricks are helpful for me. It didn\u2019t take me long to figure out how the notes are arranged.\u201d This person is a highly expert producer, beatmaker and live performer using\u00a0Ableton Live. I asked how he came to this expertise, and he said he felt compelled to learn it to compensate for his lack of \u201cmusicianship\u201d. It makes me sad that such a sophisticated musician does not realize that his skills &#8220;count&#8221;. In empowering\u00a0music learners with the aQW, I also hope we are able to help computer musicians value themselves.<\/p>\n<p class=\"p1\">\n","protected":false},"excerpt":{"rendered":"<p>This post documents my final project\u00a0for User Experience Design with June Ahn Overview of the problem The aQWERTYon is a web-based music performance and theory learning interface designed by the NYU Music Experience Design Lab. The name is a play on &#8220;QWERTY accordion.&#8221; The aQWERTYon invites novices to improvise and compose using a variety of &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.ethanhein.com\/wp\/2017\/designing-a-more-welcoming-aqwertyon-experience\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Designing a more welcoming aQWERTYon experience&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[928,9,927,498,12],"tags":[1625,185,1013,1840,1338,1160,1559,534,1653,1602,1921,773],"class_list":["post-16091","post","type-post","status-publish","format-standard","hentry","category-interfaces","category-music","category-music-teaching","category-music-theory","category-software","tag-aqwertyon","tag-design","tag-interfaces-2","tag-june-ahn","tag-musedlab","tag-music-teaching-2","tag-music-theory","tag-nyu","tag-phd","tag-soundfly","tag-user-experience-design","tag-visualization","entry"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pAPdE-4bx","jetpack_sharing_enabled":true,"jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/www.ethanhein.com\/wp\/wp-json\/wp\/v2\/posts\/16091","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ethanhein.com\/wp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ethanhein.com\/wp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ethanhein.com\/wp\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ethanhein.com\/wp\/wp-json\/wp\/v2\/comments?post=16091"}],"version-history":[{"count":6,"href":"https:\/\/www.ethanhein.com\/wp\/wp-json\/wp\/v2\/posts\/16091\/revisions"}],"predecessor-version":[{"id":18958,"href":"https:\/\/www.ethanhein.com\/wp\/wp-json\/wp\/v2\/posts\/16091\/revisions\/18958"}],"wp:attachment":[{"href":"https:\/\/www.ethanhein.com\/wp\/wp-json\/wp\/v2\/media?parent=16091"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ethanhein.com\/wp\/wp-json\/wp\/v2\/categories?post=16091"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ethanhein.com\/wp\/wp-json\/wp\/v2\/tags?post=16091"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}