Composite for Windows Phone 7

I’m proud to announce that Composite is now available for Windows Phone 7 devices.

My latest project, Cell (a collaboration with Keiichi Matsuda) was supported by Microsoft. Will Coleman, the UK Windows Phone product manager was our main point of contact throughout. After showing him my mobile apps, he expressed an interest in creating a Windows Phone version of Composite. A few weeks later this was confirmed.

Brighton based agency, Matchbox Mobile, who were also involved with Cell were brought in to develop the app and work with me and Juliet to keep to the original concept and style. It was a pleasure to work with these guys again and was interesting to see Composite adapted for the Metro design language. Big thanks to Juliet for redesigning the website (with lightnening speed) and tweaking the branding. Also, thanks to Steven Trew for helping to organise and maintain the partnership.

In terms of the aesthetic output I’m very pleased. We have managed to stay very close to the original app. Here are a few images I’ve made so far:

So install it, let me know what you think, and as ever, I’d love to see what you make with it. Please send your work to submissions [at] composite-app.com to be featured in the Composite gallery.

Now to make the iPhone 4 version. Perfect excuse to get the 4S

Cell

Cell at the Alpha-Ville festival

Cell is an interactive installation commissioned for the Alpha-Ville festival, a collaboration between myself and Keiichi Matsuda. It plays with the notion of the commodification of identity by mirroring the visitors in the form of randomly assigned personalities mined from online profiles. It aims to get the visitors thinking about the way in which we use social media to fabricate our second selves, and how these constructed personae define and enmesh us. As users enter the space they are assigned a random identity. Over time, tags floating in the cloud begin to move towards and stick to the users until they are represented entirely as a tangled web of data seemingly bringing together our physical and digital selves.

I first got in touch with the organisers of the festival, Estella Olivia And Carmen Salas, around May with a view to contributing. They asked if I knew of Keiichi Matsuda, and whether I would be interested in a collaboration. Coincidentally we had met up a month before and had discussed the idea of joining forces as our areas of research are very similar. We come from different fields, he architecture and film making, me new media art and interaction design. This turned out to be a perfect combination. We shared the concept and design, Keiichi focussed on the fabrication, planning the space and putting together the documentary while I happily wrote the software. Even with these distributed roles we found we were often offering suggestions and help to each other throughout the course of the project.

The concept wall

Microsoft have supported the project from the early stages. Keiichi and I were both speaking at an event in June when we met Paul Foster who was promoting the MS Kinect for Windows SDK. We discussed our project which would be utilising the Kinect camera and he was interested in helping out. He introduced us to William Coleman and since then they have supplied all the equipment and funded the studio space (thanks to Tim Williams and Tom Hogan at Lumacuostics for putting us up and all the advice).

In addition to this, Microsoft also introduced us to Simon Hamilton Ritchie who runs Brighton based agency Matchbox Mobile. These guys contributed a great deal to the project, most importantly, ofxMSKinect, an openFrameworks addon for the official Kinect SDK. One of the main advantages of using this over the hacked drivers is the auto user recognition, we no longer need to pull that annoying calibration stance which can be a big barrier in a piece such as Cell. In addition to depth/skeleton tracking, the potential for utilising the voice recognition capabilities is an exciting prospect for the interactive arts community. This will be integrated into ofxMSKinect in the coming months.

Skeletal data from 4 Kinect cameras

So on to the setup, Halfway through the project we realised that we would only be able to track 2 skeletons using a single Kinect camera. While this is fine for gaming, for a large scale interactive experience this would not be enough. So instead of 1 camera we decided to go with 4! We organised 4 Dell XPS 15 laptops each connected to a Kinect camera. The skeletal data from each client is fed to an Alienware M17x laptop through a Local Area Connection (with help from Matchbox) giving us the potential to track the skeletal data of up to 8 users in a space of around 5m x 4m. The software on the Alienware server then calculates and renders the scene which is rear projected onto a large screen using a BenQ SP840 projector.

The screen posed a bit of a challenge. We could either rent one for a ridiculous price or build our own and have complete freedom over the design. This was important to us so Keiichi put his woodwork skills to the test and made a 4.2m x 1.8m screen that can be reduced to 1.5m. Quite an achievement for a rear projection screen with no supporting beams! We used ROSCO grey screen material which was perfect for our requirements.

Keiichi and Iannish preparing the screen at the Alpha-Ville festival

We were very pleased with the reaction to Cell. The feedback from the festival goers was really positive. It was important to us that the participants were both interested in the concept and taken by the experience. Many that we spoke to seemed to engage with the piece on both levels.

If you would like any more information please visit the Cell website. If you would like to contact us regarding this piece, please email – info [at] installcell.com

I’d like to thank the following for their help in realising this piece (in order of appearance):

Carmen Salas, Estela Oliva, Paul Foster, Will Coleman, Simon Hamilton Ritchie, Theo Watson, Kyle McDonald, Arturo Castro, Tim Williams, Tom Hogan, Claire Holdsworth, Vincent Oliver and Iannish Posooa.

Behind the screens: Interview on the AND Project

I’m a little late in blogging about this as I’ve been very busy on a project (more on that very soon).

I was interviewed a while ago by Tyler Flynn of the A.N.D Project about mobile art. The article also included Filip Visnjic of CreativeApplications.net and Scott Snibbe, creator of Bjork’s app/album Biophelia and many other wonderful pieces. It was a pleasure to be included alongside 2 such big names in this area of software art. The article – Behind the Screens: Digital Applications – is pretty extensive but definitely worth a read.

Here it is.

New iPad app: Composite

I take a great deal of inspiration from the abstract art of the 20th century and through my own work I often aim to replicate these aesthetics in a digital context. In particular, I’m pretty enamoured by the neo-dadaist collages of Robert Rauschenberg. These pieces consist of silk screen prints from magazine images overlaid with paint. Using this technique, Rauschenberg found that he could comment on modern society using the very imagery used to shape that society.

Estate, 1963 - Robert Rauschenberg

Buffalo II, 1964 - Robert Rauschenberg

About a year ago I decided to write some software which would allow me to paint using this approach. I wanted to build a mobile app that would give users the ability to paint using a live video stream. Unfortunately the technology to allow me to realise this idea had not yet been released. This could be achieved with a smartphone but I required more screen space. I needed an iPad with a camera. In March this year, the iPad 2 was released with both a front and back facing camera. Bingo! I proclaimed, and snapped one up straight away.

I’m excited to announce the release of Composite, my 3rd mobile app, and my first for the iPad 2. Composite allows you to remix your surroundings to create artistic compositions. Users are given the opportunity to paint pictures using the live video stream from their device’s front and back facing cameras. Simply point your iPad towards your subject and start painting to reveal it. A variety of control over the brush and video stream is offered to allow for a huge range of different aesthetics. These include brightness, contrast, colour offset, alpha, blur and thickness.

More than anything, I’m looking forward to seeing how people use Composite. In addition to being a tool to create finished compositions, I believe it would be excellent as a quick sketch pad for artists and illustrators to combine ideas on the go for future pieces. I also think it would be great in schools to help teach kids about art. As with Fracture and Konstruct I have created a Flickr gallery to showcase the best submitted artwork. I can’t wait to start filling it up with your work.

In terms of the credits, I was responsible for the concept and development. My fiance and in-house designer Juliet Lall took control of the design and all branding, including the Composite website. The app was built using openFrameworks.

So install it, paint some outstanding compositions and please email your best work to submissions [at] composite-app.com to be featured in the Composite gallery. And if you like it, feel free to review it.

Articles

I’ve been busy writing lots of words over the last couple of months. I had my first magazine article published a couple of weeks ago. It was for Web Designer magazine and was on, yep you guessed it, augmented reality

It’s an investigation into AR in terms of the current state of play and where we are headed. I also discuss my approach, and that of others to the technology as a platform for art and storytelling. It’s on the shelves now.

I also wrote a guest post for the Creators Project called Augmenting The Future Of Art. This is focused more on how artists are using AR and the importance of investigating the boundaries and social implications of the technology.

Thanks to Mark Billen (WD) and Kevin Holmes (CP) for the opportunity to contribute to their respective publications.

Fractal’11 – A diary

We’ve just come to the end of what I can quite confidently describe as the best conference I have ever been to, Fractal’11 in Medellín, Colombia. Hernán Oritz, one of the festival organisers (alongside Vivi Trujillo), contacted me around 18 months ago to speak at last year’s event. At the last minute the unpronounceable Icelandic volcano erupted and my flight was cancelled. I was relieved and excited when I was invited again this year. We’ve had an extremely busy but rewarding few days.

DAY 1

During a fun first evening getting to know the other speakers and organisers over crepes and smoothies we discovered we had an interview for Colombian daytime TV at 8am the following morning (the first of several surprises). An early night followed. After a gorgeous breakfast we went to the lobby to meet the film crew. I think we were all a bit apprehensive, but as with most public speaking, once you start talking the nerves tend to evaporate.

Once the interviews where over I was whisked away to do an Augmented Reality Q&A at a local tech company called Ilimitada ran by the charismatic (and slightly mental) Juan José Mejía. After a brief introduction to AR the questions began. The audience seemed very excited by the technology and many interesting ideas came up during the discussion. Next stop, lunch in the jungle.

Lunch in the Jungle

The launch party was at a great venue called DRY. We were treated to Martini cocktails called Fractum created especially for the event before partaking in another TV interview. Halfway through the night I was introduced to an interesting character called Alex Bennett who is the Headmaster of a local school. He informed me of our next surprise, that we were coming to his school the following morning to to judge an innovation competition his students had been involved in. We continued drinking and laughing into the night with Alex and the rest of the guys, all looking forward to our next adventure.

DAY 2

Another early and slightly hazy start. We made our way up the mountain to the heavily guarded Columbus High School and met some of the teachers and students. We selected a category each and began to sift our way through the work. I chose the video and stop motion projects which were, on the whole, very impressive. Once we had selected our top 5 we made our way to the assembly hall where we got on stage in front of 300 students to discuss the categories and announce the winners. My fiancé Juliet had come along and was also on stage. She was handed the microphone a couple of times to address the audience. She was nervous but did a great job.

Columbus High School Assembly

Presenting at Columbus High School

After a quick lunch and a tour of the school we went to the event. The venue was the outdoor theatre at the beautiful botanical gardens of Medellín. This made such a change from the usual stale conference rooms. The surrounding forest and mountains combined with the sounds of the exotic birds and occasional thunder resulted in a dramatic location for such an event.

Medellín Botanical Gardens

After Hernán introduced the event, John Kessel, the award winning Science Fiction author and professor of creative writing at the NC State University began his talk. The heavens opened within moments of his presentation but he didn’t let it phase him. John discussed the way in which science fiction authors consciously and subconsciously include current social issues in their renditions of the future.

Hernán introducing Fractal'11

John Kessel

Sam Pool AKA SPL is a Dubstep DJ and producer. He had been invited to perform a set and do a presentation. Sam spoke with passion about the current state of play in the music industry, the effects of piracy and digital technology and his open source approach to his craft. He ended with a walkthrough of the technology he uses to perform and create.

Sam Pool

Johanna Blakley is deputy director of the Norman Lear Centre, a think tank at the University of Southern California. Her presentation was a discussion of her background in academia and her current research into entertainment, particularly TV drama and it’s influential effect upon the behaviours and attitudes of the viewers.

johanna Blakley

DAY 3

I was one of the first speakers of the day with my Augmented Storytelling presentation. It’s an investigation of AR as a platform for constructing unique interactive stories. The gorgeous surroundings definitely put me at ease and, according to the other half (and my fiercest critic), it was the best presentation I had ever given. Even my demos worked perfectly which is always a bit sketchy in an outdoor environment. Lots of great questions followed and even a little Spanish heckling by a crazy old guy.

James Alliban

My first heckler

In addition to speaking, I was also exhibiting my new AR iPhone app Konstruct – see here for more details. Last year my Fractal’10 AR ident, made especially for the show was exhibited.

Konstruct

Kij Johnson is another award winning Science Fiction author and educator. In her talk, she compared writers to enigma machines in the sense that they use fiction to encode their own ideas into stories. She finished up by reading one of her short stories. Tij also contributed a story called “Myth Girls” to an interdisciplinary book created specially for Fractal’11. This work also involved fashion designer Camilo Álvarez, illustrator Oscar González, graphic designer Carlos Andrés Ortiz, and editors Viviana Trujillo and Hernán Ortiz.

Kij Johnson

Amber Case is a Cyborg Anthropologist and tech consultant. Amber discussed the idea that our reliance on technology is transforming us into cyborgs. A wide variety of technology was discussed with in relation to how it is affecting us. I found the discussions on mobile phones and our online social selves particularly fascinating.

Amber Case

My last responsibility on the list was to take part in a panel called ‘Decoding the present’ alongside all the other speakers. I’m not usually a fan of panels so I can’t say I was looking forward to it. The fact that it was being broadcasted live on local TV didn’t make me feel any better. Within minutes of it starting I knew it was going to be a good one. John was moderating and had put together a thought provoking collection of questions. A stimulation discussion ensued in which everyone had something interesting to say.

The panel - Deconstructing the present

The evening ended with Sam’s stomping 2 hour Dubstep set. Although there were only 50 or so people present, the crowd soon transformed into a crazed mass of dancing maniacs. A Peruvian feast and much merriment followed before heading back to the hotel for the final night.

Sam Pool's Dubstep set

Dinner at the Peruvian Restaurant

I’ve been to some great conferences in the past but this one had it all. Interesting and affable speakers, excellent organisers/hosts, a lovely enthusiastic audience, a stunning setting, excellent catering and entertainment and a full and rewarding schedule. What more could you ask for. A big thanks to Hernán Oritz and Vivi Trujillo for inviting me to such an exciting conference. Event organisers, take note.

Konstruct – AR iPhone app

In the coming years, once Augmented Reality glasses are commonplace, I hope to see apps that allow for creative user generated content to be distributed in virtual space. These tools would give people the opportunity to virtually build sculptures, paint on walls and leave trails using gesture and voice as they wander through cities. Others plugged into the same AR channel would see these digital artefacts seamlessly attached to the places they were created. Apps such as TagDis have explored this area, but, as with all mobile AR, there is a sizable barrier in the act of navigating to and using an app on a handheld device. Ubiquitous AR experienced via a head mounted display could transform whole cities into virtual art galleries.

Konstruct explores this vision by investigating generative art in an Augmented Reality environment. It is a sound reactive AR experience for the iPhone that allows the user to create a virtual sculpture by speaking, whistling or blowing into the device’s microphone. A variety of 3D shapes, colour palettes and settings can be combined to build an endless collection of structures. Compositions can be saved to the device’s image gallery.

Konstruct is a free app available on iPhone 3GS and 4 running iOS 4+. A version for the iPad 2 is planned in the coming months.

In terms of the credits, I came up with the concept and developed the app. Juliet Lall was responsible for designing the user interface and all branding including the Konstruct website.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Users are encouraged to email virtual sculptures to konstruct[at]augmatic.co.uk to be featured in the Flickr gallery.

So on to the technology. I used the new iPhone tracking library String. I’ve been experimenting with the beta version of String for a few months now and it’s surprisingly powerful – I managed to fit over 100,000 polys into a scene without any noticeable dip in frame rate. The level of accuracy in the tracking is also very sturdy. I’m told that that this has improved considerably with the latest iteration of the beta SDK so I’ll be updating the app very soon.

There are currently 2 approaches to developing String apps, Unity, and straight up OpenGL ES. With the Unity option, developers can be up and running within minutes. Being a sucker for punishment I decided to take the OpenGL ES and Objective-C route. This was my first proper experience using OpenGL. Coming from a Flash background, where most of the hard work is wrapped up for you with libraries such as Away3D and PaperVision3D, it was a bit of a learning curve. Ultimately though, it was a great experience to learn about and implement buffer objects, matrices, normals etc. I look forward to learning more about OpenGL and GLSL for future projects.

The latest

OK lots of things to mention so I thought I’d be lazy and lump them into one blog post.

I’ve spent the last month beta testing a new iPhone tracking technology called String. I’ll leave the details for a future blog post, but I will say that it is one of, if not the fastest and most accurate mobile trackers I have seen to date. Also, importantly, there is no license fee for artists and designers wanting to experiment with and publish apps using the tech. It’s about to be released any day now. Definitely one to watch.

I’m currently experimenting with generative art in AR space using OpenGL ES. Should the app be accepted into the store, it will be included in the Square Art exhibition. The theme is Blank Canvas, which ties in freakishly well with what I have in mind.

I just had an article published on the Young Creative Council (AKA YCC London) blog. Every week they get a guest author to write about whatever they like. I used it as an opportunity to write about my work, my inspirations and my thoughts on AR in terms of the current state of play, the future and my approach to it as a platform for art. I don’t tend to use my blog as an outlet for those sorts of ramblings so it was nice to write down my thoughts on these matters for a new audience.

You can read the article here. Big thanks to Laura and Emma for sorting this out.

I’ve been asked to both speak at and be on the advisory board for the AR Summit. This event is penned to be the biggest augmented reality conference in the UK and is taking place in London in June. If you don’t fancy the presentations, there will be an exhibition hall showcasing the latest and greatest in the world of AR. Definitely worth a visit.

It’s a huge honour to be given this opportunity. I’m looking forward to having a say in the organisation of the event. Hopefully I can inject a creative edge into the proceedings.

Look here for other upcoming speaking engagements.

I was featured in this month’s Web Designer magazine. The article is titled “Whatever happened to Augmented Reality?”. I attempt to answer this question and give my thoughts on current trends, mobile AR and the future. I’ve been informed that I also had an interview published in the iPhone special of Computer Arts Projects about my iPhone app Fracture last month. I didn’t actually manage to get a copy so if anyone has the issue and a few spare minutes I’d really appreciate a scan.