i.document

PointCloud is an iOS App that expands a Webkit Browser with advanced Augmented Reality features. Thus you can stick for example a DIV with video, images or even Processing.js sketches onto images or objects in reality. I simply took a reference image of Wil Wheaton’s t-shirt and stciked an SVG / CSS 3D animation to it. The result is an perspective emitter of stencils.

Take a look at the HTML document to get an impression how to write a simple app in PointCloud:
http://dev.m05.de/pointcloud/wil/
PointCloud’s documentation isn’t complete yet. They are providing some examples, which are too complex for learning and a PDF document with further descriptions. They are still lacking a complete API reference. But i know that documentation is hard work and they just released their software.

I reduced their example to a minimum. It’s only initializing and tracking one image. Therefore you have to link an image in the head of the document. The image has to be on a public server on the internet in order to be downloaded and processed by PointClouds server. A local web server doesn’t work.

The only relevant Javascript function is onAppLoaded(). Here we are activating the tracking target image:

function onAppLoaded() {
viper.activateReferenceImage("p1_wil"); // id of image link
viper.requireRealityMap();
viper.log("requireRealityMap");
}

Your HTML content is put within a hierarchy of DIVs controlled by PointCloud:

<div id="viper.world_node" style="display:none;">
<div id="viper.scene_node">
<div id="p1_mobilehand_node" class="node" style="position:absolute;">
<div class="contentnode"
style="width:300px; height:500px; -webkit-transform:scale(0.37, 0.37) rotateZ(90deg) translateX(-150px) translateY(-200px);">
<!-- Your HTML -->
</div>
</div>
</div>

That’s it. Load your file in PointCloud browser and fine tune the positioning.

You probably have heard of the Recursive Wil Wheaton t-shirt. Paul (of Paul and Storm) sent it to Wil and this photo became quite popular. It was remixed by an unknown genius into an animated gif and “won at the internet”.

Over the weekend i experimented with the new PointCloud Augmented Reality SDK (which is by the way brilliant and simple: 3D tracking and HTML5). I took Recursive Wil and turned the concept around: A Wil Wheaton Particle Emitter.

Scott Meyer from basicinstructions.net sent me the original SVG file of the shirt’s image. My first try was using Processing.js for animation (Yes! Processing.js now works in AR). But SVG and CSS 3D were the better choice to get a perspective effect. And it’s hardware accelerated on iOS.

If you want to try it on your iPad / iPhone:

1. Download and install PointCloud Browser in App Store
2. Open it and point it to this URL:  http://dev.m05.de/pointcloud/wil/
or click this link on your iPad / iPhone 

How does it work?
Here’s a short description and the source code.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Testing Processing JS WordPress Plugin by Keyvan.

Works great as soon as i disable WordPress should correct invalidly nested XHTML automatically in Settings / Writing. Otherwise Wordpres adds i.e. </width> when using i<width in a for loop.

An overview of my interactive Cultural Heritage projects over the last years.

Motion Bank Augmented Reality Postcard
An Augmented Reality visualization of Ros Warby dancing Deborah Hay’s “No Time To Fly”. She was captured by 3 cameras in order to extract her silhouette and calculate her 3D position. The postcard and the brochure are tracked via natural feature tracking with Fraunhofer IGD’s Mobile AR framework.

DATEV Lange Nacht der Wissenschaften in Nürnberg
Posters come to live thorugh Augmented Reality. The iPad app illustrates the history of the Datev Company, data processing and taxation. It was presented 2011 at Nürnberg’s Lange Nacht der Wissenschaften.

dARsein: Augmented Reality Tour through Architectural History at House of Olbrich
The iPhone app visualizes the compelling history of Darmstadt’s unique Jugendstil quarter with Augmented Reality. Jump back in time visually by photos you take with your iPhone: Augmented Reality superimposes information on each picture and visualizes the impressive historical architecture of the Art Nouveau in front of real building.

MovableScreen at Allard Pierson Museum in Amsterdam
During the “A Future for the Past” exhibition of Allard Pierson Museum (http://www.allardpiersonmuseum.nl) in Amsterdam we are presenting two Augmented Reality applications on the MovableScreen: A virtual reconstruction of Satricum and an annotated landscape on a 1855 photograph of Forum Romanum (Rome Reborn).

Augmented Reality Sightseeing
A table with a satellite image of Berlin shows a 3D model of the Berlin Wall and the urban development from 1940 – 2008 are displayed. Therefore urban grain plans showing areas covered with buildings is augmented on the satellite image. The visualization was presented on UMPCs and the iPhone via video seethrough. Furthermore posters are simulating the system working outdoors. Historic photographs are seamless superimposed and showing the development of landmarks.

iTACITUS Reality Filtering
Reality Filtering enables context sensitive overlays of original historic drawings of missing paintings or lost architecture. For a seamless integration we are rendering the reality in the style of the original drawing (here: blach and white). At Reggia Venaria Reale (http://www.lavenaria.it) we are visualizing missing paintings in Diana Hall, different architecture styles of Palazzo Diana and the lost Temple Diana in the gardens of the palace.

Rome Reborn Augmented Reality at SIGGRAPH 2008
Augmented Reality overlays of 3D roman monuments via markerless tracking.

Finally the 3D prints arrived from Shapeways. There’s still no direct 3D model export in the application due to irregular holes in the Kinect depth map. But it works with minor manual fixes with Meshlab.

Here’s the Flickr album with the results.