GIS, GPS and Digital Photography
Dutch Contribution to European Research Project

The introduction of digital cameras has led to enormous (online) collections of digital images. The European Tripod project aims to introduce new tools on automated labelling of these images. A Dutch partner, Geodan, wanted to know if GIS could be used for recognizing objects and label images. Dutch Geomatics student Arnoud de Boer took the challenge and came with promising results for the future of digital imaging and GPS.

Edward Verbree (l), Arnoud de Boer (m) en Eduardo Dias (r)

Since the introduction of digital imaging it has become easier to share large photo collections online with other internet users. Describing enormous online photo collections isn’t automated yet. It still takes a lot of time. To change this situation, a European research project called Tripod has been initiated. The purpose of this project is to improve the access of visual media (especially digital images) for multiple groups of users by deve­loping tools that improve the quality of

existing images, labels and automatically place labels of new images. The project couples with technological deve­lopments which integrate photo gear and GPS. This means that alongside storage, also the location of the image is being saved in the metadata. Although useful tools have been developed to save environmental data (such as weather information and geographical names), it’s not yet possible to save information of imaged objects, such as, for example, names of historical buildings.

Geodan

In the future this all will be different: digital cameras for the consumer market will be equipped with GPS and a digital compass, so that imaging position and view angle can be recorded. In Asia these cameras are already being used for pedestrian navigation. Also, the exact scale of the imaged situation is registered. Geomatics student Arnoud de Boer performed research on the use of 3D positioning and orientation of digital imaging to automatically recognize and label imaged objects with the use of GIS. This all happened within the scope of a graduation project at Geodan, the Dutch partner of Tripod. The results of this research turned out to have significant benefit for Tripod.

Figure 1. Extrusion model of the intersection of vectorized elevation model with the building footprints

Extrusion Model

De Boer’s research combines digital imaging, GPS and GIS. Roughly, the research can be divided into three steps: the first step is combining GPS and digital imaging, to be able to localise the images geographically. The second step contains linking the images in a virtual model with a spatial dataset with the use of GIS, in order to recognize the images with a desktop computer. The third step contains the automated labelling of the images, also with the use of GIS. Reference point of this research is recognition of historical buildings, based on the assumption that many people take pictures of histo­rical buildings and use the names of those buildings when labelling them. A second reference point is that a perspective view generator will be used for object recognition. This can be done with GIS software such as ESRI ArcScene, where 3D data can visualised and rendered as defined by the user. Another reference point is to make use of a perspective viewer generator for recognizing objects. A perspective viewer generator a software program such as ESRI ArcScene, that can be used to visualize, render and export 3D data in a user-defined perspective. For carrying out step one, spatial and visual accurate images have been taken at the Grote Markt (market square) in Delft, a town in the west of the Netherlands. The most accurate spatial images of step one have been made with a Topcon imaging total station. Step two demanded most creativity of all three steps: in order to recognize the objects on these 3D georeferenced images, they need to be

localized in a 3D model and linked to a spatial model and data set.

To do justice to the irregular-shaped, unique historical buildings at the market square in Delft, a 3D extrusion model has been built, based on spatial data sets such as 2D map data and Buildings Top10nl (figure 1). With this, the footprints of all individual buildings have been defined and subsequently height data have been added to these footprints, from a national Dutch LiDar elevation mapping initiative called AHN (General Elevation Map in the Netherlands, Algemene Hoogte­kaart Nederland). The advantage of this extrusion model, built in ArcGIS, lies in the fact that the buildings simulate their real heights, and don’t necessarily look like flat blocks like normal extrusion models do.

Figure 2. Object identification results of Topcon image 1010

To identify the names of the imaged objects, the extrusion is linked to the digital images. The reason behind this is to present the 3D extrusion model in 2D by means of a perspective change, to be able to make an overlay of the digital image and the identical virtual scene. The reason for this is that the digital image contains information about the distance and depth, in relation to the real objects. The virtual image, on the other hand, contains information about the names of the objects in the picture, along with geometry of these objects. ESRI ArcScene has been used to make a perspective view of the virtual objects that corresponds to the perspective of the digital images. Each and every object gets a unique colour value, that serves to be able to relate the 2D vector features from the extrusion model with each other. This export happens in a PNG-file, so that no loss of (meta)data occurs. Now, objects in the images can be labelled with the object labels of the virtual scene with an ‘overlay’ command. With replacing the object labels with the object names, the user can place the object names inside the labels in the virtual scene (figure 2).

Figure 3. Screenshot of the demo-application build in ESRI Arcmap using VBA

Evaluation of The Final Results

What is the benefit of De Boer’s research for Tripod? Eduardo Dias (Geodan) explains the evaluation of the end results: “Arnoud showed that GIS techniques can be applied for Tripod. Now, we are convinced to further develop this technique”. When asked if there is a faster and more convenient way of object recognition on digital images, for instance with techniques as applied by a company as Cyclomedia, thesis supervisor Edward Verbree (Delft University) explains the diffe­rent approaches: “Cyclomedia can easily convert pixel coordinates with view direction and view angle accurately to terrain coordinates, because of its image recording system specifics. Arnoud carried out this conversion in a GIS by using available height data of buildings”. Especially this point of approach is what makes this research special, states De Boer himself: “The idea is to present a 3D problem in a 2D environment, by means of a perspective viewer service like ESRI ArcScene. It is expected that as spatial data sets become more detailed because of a higher resolution DTM’s, more accurate object identification can be performed by better visual models”.

Eric van Rees evanrees@geoinformatics.com is editorial manager for GeoInformatics.

For more information, have a look at:

www.projecttripod.org

www.geodan.nl

http://home.tudelft.nl/en/

Edward Verbree e.verbree@tudelft.nl teaches at Delft University and is researcher GIS Technology. Eduardo Dias eduardo.dias@geodan.com is Research Coordinator at Geodan. Arnoud de Boer http://arnouddeboer@cs.uu.nl is PHD-student at Utrecht University.