quakr phrase
quakr phrase
recycle symbol
Frowny

Quakr Where 2.0 30th May, 2007

Is it possible to build a virtual world from user-contributed photographic metadata?






Quakr Home


This document contains Embedded RDF

Quakr where 2.0 30th May, 2007

Where 2.0, The Fairmont, San Jose

Who is Quakr?

Dave, Katie and Peter

This talk

  1. Laying the Foundations
    (The Plan)
  2. Bricks and Mortar
    (What We Built)
  3. What is a "Brick" anyway?
    (SWeb Issues)

Part 1

Laying the Foundations
(The Plan)

The Giants' Approach - Google Earth

Google Earth

Google Earth
google earth 3D © Hector Mila (cc by-nc/2.0)

The Giants' Approach - Google SketchUp

Google SketchUp

Google SketchUp

The Giants' Approach - Microsoft Virtual Earth

Microsoft Virtual Earth

Microsoft Virtual Earth
virtualearth © Antonio Ortiz (cc by-nc/2.0)

Limitations (1)

Q. Where does the 3D data come from?

A. Internally (i.e. within Microsoft or Google)

That's a never ending task!

The Forth Bridge
The Forth Bridge © Steve Makin (cc by-nc-nd/2.0)

Limitations (2)

Q. How flexible are these worlds?

Arc De Triomphe By Night
Eiffel Tower By Night
Arc De Triomphe In The Snow
Eiffel Tower In The Snow
Arc De Triomphe In The Fog
Eiffel Tower In The Fog
Arc De Triomphe At Christmas
Eiffel Tower At Christmas

A. Not very.

Limitations (3)

The installation is taking longer than expected...

Virtual Earth Setup Dialog

An Alternative (1)

Geotagging

Geotagged _DSC6019 © youngrobv (cc by-nc-nd/2.0)

An Alternative (2)

A reasonable question in the year 2000:

"But where are you going to find anyone even remotely interested in doing this?"

An Alternative (3)

Who saw THAT coming!?

Growth of geo:lat
Flickr geo:lat query

An Alternative (4)

So, if you give the people tools to make it easy...

Flickr Map

Flat Mashups (1)

Photos positioned on maps: http://loc.alize.us

loc.alize.us

Flat Mashups (2)

Photos nearby: Zooomr

Zooomr
zooomr © MillyNeT (cc by-nc-sa/2.0)

Quakr

A project to build a 3D world from user-contributed photographs

Metadata for photos in 3D (1)

Altitude

(height above sea level)
Altitude

Metadata for photos in 3D (2)

Rotation about y-axis

(compass bearing)
Compass Bearing

Metadata for photos in 3D (3)

Rotation about z-axis

(tilt)
Tilt

Metadata for photos in 3D (4)

Rotation about x-axis

(roll)
Roll

Metadata for photos in 3D (5)

Quality of the world will improve with:

Quakr constraints

Quakr constraints

  1. Quakr should run in a browser.
  2. Quakr should require little or (preferably) no installation.
  3. Quakr should be a thin client, consuming raw resources.
  4. Quakr should add context information to enhance the 3D experience.
  5. Quakr should adapt to an ever changing semantic web.

Part 2

Bricks and Mortar
(What We Built)

Capturing the Data

We need to capture a pile of meta data...

Nokia (550/Sport?) Phone with "tilt" :
Nokia phone with tilt
Digital Compass and a 3-Axis orientation + compass :
Digital Compass 3 Axis + compass
Basically; we want a camera + a GPS device + a "gizmo" :
A camera plus a GPS device

Homemade - 7D Tiltometer (1)

Basic conclusion - nothing exists... so we built one...

Get a camera and some basic stationery items...

Camera, compass, string and white-tac

Homemade - 7D Tiltometer (2)

Stick protractors to a camera, add string and plumb bobs :

Camera, compass, string and white-tac

Homemade - 7D Tiltometer (3)

Add a compass and a GPS device :

And a GPS device

Homemade - 7D Tiltometer (4)

One person to take the photos, one to log the readings :

7D Tiltometer in use

What and how to tag

The choice of tags for the test data was guided by two principles:

Alternative image APIs

Require bounding box query and ability to add tagged meta data

Possibles include;

A fully tagged image on Flickr

Machine Tagged...

A fully tagged image on Flickr

Flickr API bounding box

Bounding Box API query

"http://api.flickr.com/services/rest/?method=flickr.photos.search
&api_key=XxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXx&min_taken_date=1970-01-01%252000%3A00%3A00
&bbox=-1.3%2C51.754%2C-1.2%2C51.755&tags=quakr&extras=machine_tags"

A Flickr API response

Flickr API single

A second query for each image

http://farm{farm-id}.static.flickr.com/{server-id}/{id}_{secret}_[mstb].jpg =>
http://farm1.static.flickr.com/164/364490580_426a5c9ec2.jpg

Single image returned

Viewr technology options (1)

Version 0.01: Java applet

Java Applet Java Applet - bust

Viewr technology options (2)

Version 0.02: VRML/X3D with a plugin

VRML plugin - bust

Viewr technology options (3)

Version "Just Works" - Flash + SANDY 3D API

Sandy - 3D Flash API

http://www.flashsandy.org/

Adding context

Map tiles for floor

gmap tile

Altitude of floor (http://www.earthtools.org)

Earth Tools API response

There are other services...

Live Demo

greasemonkey script from gmap

Part 3

What is a "Brick" anyway?
(SWeb Issues)

Using Tagging Metadata

"Tagging: describing web content using whatever words seem right" [Beckett06]

milton_keynes_recycling_factory
Milton Keynes Recycling Factory © gavinandrewstewart (CC Attr 2.0)

Philosophy of Data Tolerance



  1. Recycle everything
  2. Tolerate unhygenic results. (enhance? adjust?)

Benefits of Data tolerance

We, the people..

crowd scene
Mexican Wave © ~Prescott (CC Attr-NC 2.0)

Costs of Data Tolerance

Synonyms and Homonyms (1)

Dual (competing) Specifications

screenshot of GEO vocabulary screenshot of KML vocabulary

Synonyms and Homonyms (2)

Different syntax - same symantics?

screenshot of flickr showing usage of geo:dir screenshot of flickr showing usage of ge:heading

Synonyms and Homonyms (3)

Even within the same specification...

screenshot of flickr showing usage of geo:long screenshot of flickr showing usage of geo:lon

Semantic Ambiguity One (1)

"Tilt"

screenshot of flickr showing usage of ge:tilt

Semantic Ambiguity One (2)

What does "Tilt" mean?

Leaning Tower of Pisa
The Leaning Tower of Pisa © *natsu* (cc by-nc-sa/2.0)

Semantic Ambiguity One (3)

A bit tilty?

Leaning Tower of Pisa with angles overlay
The Leaning Tower of Pisa © *natsu* (cc by-nc-sa/2.0)

Semantic Ambiguity One (4)

In Camera terms then...

tilting a camera

Semantic Ambiguity One (5)

KML Spec thinks Tilt is... (and doesn't go > 90)

KML spec showing meaning of tilt
http://code.google.com/apis/kml/documentation/kml_tags_21.html

Semantic Ambiguity Two (1)

"geo:lat" and "geo:long"

http://www.makezine.com/ blog/archive/2005/07/how_to_gps_tag.html
screenshot of MAKE blog discussing how to geotag photographs

Semantic Ambiguity Two (2)

After uploading - should I drag to the place I was?

screenshot of flickr map tool

Semantic Ambiguity Two (3)

Or the place I was taking a photo of?

View from the Eiffel tower
France 1 © SqueakyMarmot (CC Attr 2.0)
View of the Eiffel tower
Evening At the Park © StuckInCustoms (CC Attr-Nc 2.0)

Inaccuracy

Our 7D Tiltometer GPS device (actual readings)...

quakr demonstrating inaccurate geo data

Inaccuracy

The Quakr TAGGR solution!

screenshot of quakr taggr tool

Sparseness

It's good, but not quite good enough...

screenshot from flickr demonstrating sparse metadata set
P1030675 © cesargp (CC By-nc-sa/2.0

Tolerating Sparse Data Sets

Technical approaches:

  1. Defaulting - tilt
    ( ~= 90 = straight ahead)
  2. Enhancement - altitude
    ( ~= ground + 1.5 metres)

Abundance

An Image Jungle

quakr demonstrating image jungle

Spam

Nothing to stop this kind of manipulative image...

quakr demonstrating spam

The Power of Heuristics

Quakr thin client architecture

quakr heuristic layer

Future Heuristic Work

Conclusion 1 - What we Did

  1. Aim: 3D world built from community contributed metadata, which runs in a browser
  2. Geotagging
  3. What metadata would be needed?
  4. What are people actually doing?
  5. Our Flash App
  6. Which is tolerant and recycles

Conclusion 2 - What we are still struggling with

  1. Messy tagging data - "heading"/"dir"
  2. Semantic ambiguity - what does "tilt" mean?
  3. Inaccuracy - GPS
  4. Sparse data (enhancement)
  5. Overabundant data - image jungles
  6. Spam

http://www.quakr.co.uk

Sightseeing in Quakr

quakr sightseeing - cows over the river quakr sightseeing - crossroads in Utrecht quakr sightseeing - Broad Street, Oxford quakr sightseeing - pub by the side of the road