Sound as Color

2017

What Does Music Look Like?

Sound As Color converts audible frequencies to the visible spectrum, using color, brightness, and size to represent frequency and amplitude. The result allows listeners to enhance their sensory experience of music by "seeing" a visually appealing range of sound.

Honors

Shortlist, Kantar Information Is Beautiful Awards, 2017

Tasks

Concept, Platform / Web Development

Tech

Web Audio API, Javascript, HTML5 Canvas

Sound as Color
iamamiwhoami - y
Roisin Murphy - Ten Miles High
Rilo Kiley - hail to whatever you found in the sunlight that surrounds you
    • Technical Process
    • Frequencies are extracted from the input audio and then mapped to the full visible spectrum, where the lowest audible frequency is equal to 390THz and the highest audible frequency is equal to 750THz.
    • Samples are extracted from the input audio signal at a rate of ~2.61 per 1 second.
    • Each sample contains 16,384 indices, each representing a 1.3458251953125Hz increase over the last, where the value is equal to the amplitude of the sampled location.
    • Colors are generated in HSB, where hue and saturation are extracted from the loudest audible frequency in each sample, and brightness is measured by the amplitude of that particular frequency.
    • HSB is converted to RGB.
    • Samples are represented by circles and are mapped across the output image in an L-T-R writing system.
    • The size of each circle is determined by the average amplitude of every frequency in the sample.