Digital audio is just long lists of numbers

How do you get sound in and out of a computer? There are two steps. You have to turn the sound into electricity, and then you have to turn the electricity into numbers.

Turning sound into electricity

At the physical level, a sound is a rhythmic vibration of air molecules. Your ears can detect subtle changes in the air pressure, and can reconstruct good guesses about what might be agitating the air to produce those changes. When the air pressure fluctuates in a steady sine-wave pattern, you hear a musical pitch. The faster the fluctuation, the higher the pitch. This video by Vi Hart explains:

Microphones work a lot like your ears. They contain pieces of metal that vibrate in response to the vibrations of the air, generating a fluctuating electromagnetic disturbance.

Analog recording media store the fluctuating electric current as fluctuations in something else. In a vinyl record, the fluctuations are encoded in the squiggly physical shape of the sides of the groove. Magnetic tape stores the fluctuations in the alignment of tiny magnetic particles embedded in the plastic.

Turning electricity into numbers

The computer takes in fluctuating electric current and turns it into numbers. Long, long, lists of numbers. It does this with a specialized device called an analog-to-digital converter.

The analog-to-digital converter in the computer’s sound card has a clock, like the one synchronizing the activities of the computer generally. At each clock pulse, the converter takes a reading of the current on the input wire and finds the closest numerical value out of a finite set of choices. The more choices the converter has, the more accurately it can represent the input signal. A two-bit converter would only be able to tell if the input wire was on or off. A four-bit converter could sixteen different possible readings. An eight-bit converter could store 256 different readings. The more bits you have, the more accurately your digital readings can represent the original input signal. The standard for CD-quality audio is 16 bits, which means that each reading can take one of 65,536 different values. Higher bit depths are also possible — very fancy equipment can record 24-bit, 48-bit or even 96-bit audio.

Bit depth is only one factor in determining the quality of your digital audio. The other is sampling rate, the frequency with which the converter takes its readings.  If the converter takes its readings more often, it gets more accurate results. Standard CD-quality audio is recorded at 44,100 samples per second. This sounds like an incredible speed, but CPU clocks routinely operate thousands of times faster than that.

The image below shows a four-bit analog-to-digital converter in action.

The red line shows the amplitude of the input wire’s voltage over time. The sixteen horizontal grey lines are the different voltage levels the converter can detect. It takes four bits of data to specify the sixteen different possible values. The tick marks on the horizontal axis are clock pulses. Each reading gets stored as a binary number. The list of numbers makes up the digital audio file.

To play digital sounds, the computer sends its list of numbers to a digital-to-analog converter. To produce sound on speakers or headphones, the converter sends pulses at the appropriate voltages out on the wires, which your ear averages out into a pretty good reconstruction of the original sine wave.

Digital audio editing

Once you have your current stored as numbers, you can do a lot of cool stuff. Any sound in any digital medium is basically a spreadsheet with two extremely long columns, one for each stereo channel. In 16-bit audio, the numbers in the columns range from zero to 65,535 (2^16 – 1.) One second of stereo CD-quality audio is two lists of 44,100 numbers each. If the values of the numbers range smoothly along a sine wave that cycles four hundred forty times per second, you hear a computery beep playing concert A. If the numbers fluctuate along the pattern you get from superimposing the sine wave with another one that cycles six hundred sixty times per second, you hear two computery beeps a perfect fifth apart. Add in another sine wave doing eight hundred eighty cycles per second and you get the I-V-I power chord beloved by rock and roll.

All of the audio editing and processing that happens in Pro Tools and programs like it boils down to systematic mathematical operations on your lists of numbers. Auto-Tune looks for sine wave patterns and alters them so they snap to the closest piano-key frequency. At the transistor level, Auto-Tune is no different from Microsoft Excel, except that it acts a lot faster on bigger lists of numbers. Copying and pasting repeated sounds is the same procedure for the computer as copying and pasting a list of numbers or a string of text.

You need a fast computer with a capacious storage capacity to do serious audio work, but we’re lucky enough to live in an era when even a garden-variety laptop can handle stupendously complex audio tasks.

The great miracle of music for me is not any particular technique or piece or performer, but just the fact that it exists at all. A single linear wave can encode all the rich complexity of all the sounds we hear. This wave is as easily translated into numbers as dollars can be translated into pizzas. The complete works of Bach or Coltrane can be encoded as simple two-dimensional waveforms. All that music is two-dimensional curves, voltage vs time, or air pressure or guitar body flexion vs time.

Our brains are stupendously adept at detecting patterns of patterns of patterns in the linear waveform of air pressure, deconstructing and comparing the component sounds that went into it. If there are multiple frequencies present simultaneously in the pattern of vibrations, we can distinguish them and, with a little training, detect the ratios between them. I feel like we’ve barely begun to scratch the surface of the artistic possibilities of mathematical operations on numerical audio data.