Clinton Montague

Developer, learner of things, functional programming enthusiast, hacker, and all round inquisitor.

Analysing the visual data from the HTML5 audio API

March 7, 2011

I wasn’t expecting to get very much out of the HTML5 <canvas> and <audio> API demo apart from some cool looking animations. But I’ve surprised myself and have gotten interested in what the data can say as well as working out clever ways to make it look interesting.

A bit of history

So yesterday, I started playing with getting the data from an audio tag and visualising it with canvas. I wrote two simple programs to display the data as a graphic equaliser and then playing with building a real-time frequency heatmap.

What happened

When I started messing around this evening, I thought I’d start easy and turn the heatmap into a histogram too. So as well as moving the colour from green to red, I increased the size of the bar. I had to fiddle with the numbers a bit to make it look right, but when I got it right, I was quite interested in what the shape of the visualisation was saying.

The results

I’ve taken a screenshot and put some arrows around the various peaks and which instruments they represent. That’s pretty cool to be able to do! It confirms what I know about cymbols which is that they are a very ‘noisy instrument’ – not just in how loud they are, but that the frequencies range is very high. You can see this because high frequencies are spread out and have a much smaller height, so the same noise level is spread out over many of the lines – unlike the tall thin bars for the organ’s chords. I think that the lower bars are chords on the guitar because it is a bit quieter.

I’d like to thank…

This last paragraph doesn’t really fit with the rest of the post, but I wanted to say it anyway. When I read it back to myself, it sounded like an oscars acceptance speech, so I’m sorry about that, but I don’t know how else to write it :P

After being to Seb Lee-Delisle’s Creative JavaScript Training in Brighton, I haven’t been able to put JavaScript down. It opened my eyes to a whole new way to think about coding; creative coding and visual programming. If you’re interested in this kind of stuff, you should check out his podcast, creative coding, which he hosts with Iain Lobb. The best thing about it is that it’s not just about canvas – it’s more about the art of creative coding, so there’s everything from Flash to Processing, Unity3D to Windows Phone 7.