AR (Augmented Reality) is starting to become popular and increasingly accessible.
Click Suite has been having serious fun with this technology as we developed a working prototype for one of New Zealand's biggest companies.
Here, Jotham (our senior programmer), gives a reality check on the tools behind making AR happen...
In a previous post Emily summarised Augmented Reality in relation to our business as:
“Augmented Reality is where you have downloaded a piece of software, onto your computer, or more commonly recently, onto your mobile phone. Using the software, and the camera on your phone or computer, you can translate a special (coded) image [Glyph] into a 3-D object you can turn around and interact with.”
For Click Suite, our main concern has always been the value of AR beyond a novelty-item status. It's fun, but usually not useful. Personally I'm adverse to gimmicks, but we came across a use for AR which had the potential to produce a useful AR product rather than just a flashy time-waster.
We were excited by the possibilities and took the concept to a potential client. They went for it and in the past month we built them a working prototype.
In this post I’m going to focus on the process of assembling our own AR application using Adobe Flash and the FLARToolkit.
In an AR application like this there are only two major parts:
- The glyph
- The viewer
The glyph is a monochromatic image specifically crafted to allow the FLARToolkit to find it easily in a frame of web camera data. Once the glyph is found the spatial transformation information is extracted. This transformation information allows us to project our graphics onto the web camera image at the right scale, and rotation. The user typically prints the glyph out from a supplied PDF or similar.
The tasks of the viewer application are to read the web camera data, locate the glyph in the data, then render a composition of our computer graphics and the camera image.
For the viewer we went with a Flash browser applet. Adobe Flash provides us with a convenient interface to people’s web cameras and a number of useful tools (such as FLARToolkit), already exist to support building AR applications.
FLARToolkit provides us with a set of routines to locate and then determine the transformation of the glyph.
Ryan, a graphic designer here, and I did quite a few experiments with different sizes and shapes of glyph, trying to determine what would work best in any given environment.
It turns out the routines in the FLARToolkit are very robust and allowed us a wide range of flexibility in the glyph design. I was impressed at how successful the toolkit is at identifying the correct glyph from a line up of very similar glyphs. This means we could keep very close to our customers brand (using their logo as the glyph) while providing something with utility to the AR experience.
I did find the tools for building and adjusting glyphs to be very minimal and many of the sites (as well as comments in the toolkit source code) are in Japanese. However, there is one elegant little AIR application that lets you capture a glyph from an image file or your web camera. Unfortunately before finding this gem I ended up assembling my own tool in Python to decode and encode the glyph data. From this I discovered the FLARToolkit glyph file format is very simple and convenient to use.
Once we settled on a glyph and had conducted various tests with different cameras, light levels, desktop (physical) environments etc. we were ready to make things look pretty!
After looking at various Flash 3D engines we settled on Papervision3D. PV3D promised the ability to display 3D models and animations made with external tools which meant it was going to fit well into our graphics production pipeline. I hadn’t worked with PV3D before and I found it to be well documented easy to use and it seems to have a friendly user community.
It looked like the best format for all our artwork was going to be Collada. In the studio we usually use Cinema 4D for our 3D work, unfortunately the version of Cinema 4D we use doesn’t provide Collada support (though newer versions do). Looking around it seemed like much of the PV3D community was using a FOSS product called Blender.
If you have worked with 3D you will know learning a new 3D application is no small task.
My boss Rex bravely suggested he was willing to learn Blender for the project. One thing to note about Blender was how convenient its UV mapping tool is, this let us get all of our textures looking very nice. It wasn’t long at all before we had models and animations being projected onto our chosen glyph in the viewer.
Once the user interface was complete and all the elements came together we found the application was feeling great to use.
The nice thing about this solution is that the access barrier for the user is very low. They don't need any special drivers, software or downloads - just Flash and their web browser of choice. I would say as a tool chain we found Flash, FLARToolkit, and Blender to be productive and rewarding.
I hope for more AR projects in the future. As cameras and computers improve this technology looks set to only get better, more useful and more compelling.
