Bipolar is the result of a short experimental journey into visualising sound using computer vision. The initial idea was to capture a mesh of my face in realtime, and warp it using the sound buffer data coming in from the microphone as I speak. Initially I explored ofxFaceTracker but had trouble segmenting the mesh so moved to the Kinect camera. I had a rough idea of how the final result might look but it turned out quite differently.

As this intense spiky effect began to take shape I realised this would be perfect for the chaotic and dark sound of Dubstep. Thankfully I know just the guy to help here. I met the DJ and producer Sam Pool AKA SPL at the Fractal ’11 event in Colombia. He kindly offered to contribute some music to any future projects so I checked out his offerings on SoundCloud and found the perfect track in Lootin ’92 by 12th Planet and SPL. This, of course, meant I would have to perform to the music. Apologies in advance for any offence caused by my “dancing” 🙂

This was build using openFrameworks and Theo Watson’s ofxKinect addon which now offers excellent depth->RGB calibration. I’m building a mesh from this data and calculating all the face and vertex normals. Every second vertex is then extruded in the direction of it’s normal using values taken from the microphone.

The project is still at the prototype stage and needs some refactoring and optimisation. Once it is looking a little better I will release the code.

13 Responses to “Bipolar”

  1. 1 Danilo Figueiredo March 5, 2012 at 3:31 pm

    I bet a lot of artists would love to have such visuals while they perform live. Sweet!

  2. 2 James Alliban March 7, 2012 at 11:45 am

    Thanks Danilo. That’s exactly what I was thinking.

    Any takers 🙂

  3. 3 matt March 8, 2012 at 6:44 am

    alright- here it is- straight up: i see a dope marketable effect.

    but, it seems to be beat connected, not motion controlled as much as motion captured…am i right?
    now, i DO dig the work-in-progress nature. if this is as embryonic as i think it is, james, there are many venues that this could be used in.
    music-wise: check a site called FWD: if you aren’t already aware of it. a whole range of electronic (dancefloor) music ”lives” there, i like to imagine.
    also, re: dubstep- for this effect you are birthing it seems waaaaay too busy.
    look into some minimal/minimalism etc. so your effect is highlighted in an almost mathematically hypnotic manner, not a thunder-beat “machines-they-be-comin’-to-get-me!” kind of chaos sound.
    though, i liked what i heard that mr pool had to offer.

    and, oh yeah, just move, bro. those who can dance to dubstep well have been practicing. the beats and the genres change so fast it seems foolish to learn to dance to music with no set rhythm at ALL. so no need to apologize! dancing is only about that beat. only.

    i obviously like what i see. a couple of your flickr pics are gonna hit my circle and we will see what who likes what.
    thanks for the work.

  4. 4 James Alliban March 8, 2012 at 10:36 am

    Yes this is controlled by the sound data coming in from the Microphone.

    Bipolar is intended to be, as you say, the first in a series of experiments in this area of music/sound visualisation. The idea of the more minimal, ambient approach is interesting. I might just explore this idea further. I understand how you might feel that it is too busy. Once I saw this effect develop I knew that a dark and twisted Dubstep track would, in my mind, perfectly match it.

    Oh and Re: the dancing, I was just joking. My moves are the dopest 🙂

  5. 5 David Bardos April 11, 2012 at 10:50 am

    Hey James, It looks really cool!
    I am working on an installation with kinect,,
    and i am looking for how to build a mesh (triangulation?) from the vertex array that gives you ofxKinect . Do you know if exists any addons that works like this?


  6. 6 James Alliban April 16, 2012 at 12:25 am

    Thanks David

    Check out the ofMesh class. You will need to create the mesh by adding the vertices, and indices. You then need to calculate all the face normals. You then calculate the vertex normals by getting the average from all the surrounding face normals. You then add these to the ofMesh.

    I use the ofShader to texture the mesh with the RGB image.

  7. 7 matt May 31, 2013 at 8:31 pm

    HI James,
    Just coming bac from shoreditch digital exhibition where Bipolar was exhibited.

    Well amazing!!

    Can ask you if you working with any pixel color recognition?
    Do you know how I could get processing to recognise single pixel colour or areas of colours?

    thanks in advance,


  8. 8 James Alliban June 6, 2013 at 9:09 pm

    Thanks Matt. I’m not using colour recognition.

    I did some colour recognition ages ago but I’m not sure my solution was the best. There should be plenty of info on this online.

  9. 9 Mas Nalgan September 21, 2013 at 8:18 pm

    Saw this at the The New Sublime in Brighton.
    Very cool stuff

  10. 11 Daimon Wolf June 30, 2014 at 1:37 pm

    very good work! I am doing something similar. Can you give me some tutorial to follow, or something that can help me, because i am not good on OF but I work with Processing and Max/Msp?

    • 12 James Alliban June 30, 2014 at 2:06 pm

      Thank! I don’t use Max/MSP and haven’t used Processing properly for ages. I’m not sure Processing would be fast enough to do something like this. OF/Cinder are best for this sort of thing. I released an add-on that takes care of a lot of the depth processing recently. There is no example yet but you can see it here –

  1. 1 Bipolar | James Alliban Trackback on October 2, 2013 at 5:32 pm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


Flickr Photos

%d bloggers like this: