Spectral
From light, to sound (and back again).

Spectral is an Android application exploring the mapping between images and sound, by allowing you to take your pictures and turn them into spectrograms. Spectrograms are a succession of sound pulses, each of which have a frequency spectrum corresponding to a column in your original image. In receiver mode, the application uses the Fourier transform to reconstruct images from spectrograms.

Spectral is now available in the Android market. Alternatively, you can download the package from here. Bear in mind you'll have to turn on installing from unknown sources in Settings > Applications.

Using Spectral

Spectral also includes image manipulation tools, like brightness and contrast adjustment and special effects. Try using these on your images, and see how the resulting spectrogram changes. To save memory, Spectral is very fastidious in cleaning up after itself so make sure you save what you want to save.

The sounds it spits out can also be a bit grating, but this is a necessary tradeoff for good vertical resolution - adding a large number of cosine waves together, all at different frequencies, causes the "clicking" you can hear. Try using Find Edges to turn most of the pixels in your image black, and hear what a difference it makes.

Known issues

• The speaker and microphone of any device will of course impose their own characteristic peaks and dips on the frequency spectrum of sound. As such the images that you transmit and receive will not look quite as nice as they do on your screen. I'll be adding a "test pattern" to the beginning of transmitted spectrograms so that the receiver can calibrate its response to various frequencies.
• Applying convolution effects (i.e. blur, find edges...) takes a long time at the moment - this is because I'm using a 2D convolution matrix instead of two 1D matrices.
• If you keep the "Listen from microphone" option on, or decode an exceptionally long spectrogram, Spectral will eventually run out of memory and crash.
  
• The images decoded from spectrograms are automatically resized to fit the screen as of now.

Last updated: 7th August 2009, by Rhodri Karim.