Augmented reality provides an exciting new tool for heritage restoration, edutainment and exploration
Research Division of Architectural Heritage
CHOI Hyangsun
With the advancement of digital technology, augmented reality (AR) and virtual reality (VR) are being widely adopted in many academic and professional fields. VR technology, for example, provides a realistic visual and immersive environment accessible from anywhere, and AR technology shows 3D virtual images superimposed against the background of real-world images. In this respect, AR technology is expected to be of great use for virtual restoration of ancient ruins.
Marrying Modern Technology and Ancient Heritage in Digital World
Restoring lost architectural heritage is an extremely challenging task, but the demand has been steady because restoration can satisfy a natural human curiosity about our past and help expand the utilization of cultural heritage.
Rebuilding a lost architectural heritage to its original state is virtually impossible. Even when a building is restored, the surviving artifacts, such as foundation stones, platform stones, and several archaeological layers underground, may be damaged during the construction process. Furthermore, even if sufficient protective measures are taken to avoid damage to the existing remains, buried ruins are permanently hidden from view unless the restored building is dismantled and the soil covering removed.
In this case, the issue of authenticity may arise since only the restored building is visible, while real artifacts are hidden. Besides, even if a building is restored as authentically as possible after a sufficient historical verification process, the restoration may later turn out to be inaccurate when new evidence emerges; however an actual building cannot be easily modified once restored.
New technologies do not suffer from these restraints. With an AR-based restoration, an AR model can be conveniently readjusted at any time if new research requires changes. In this regard, AR-based restoration is an effective alternative to real-world restoration as it allows for both the protection and utilization of a cultural heritage item.
Restoring a Lost Temple
Founded in 553 (the 14th year of the reign of King Jinheung of Silla), Hwangnyongsa Temple underwent changes over a long period of time before becoming the greatest temple of Silla. However, in 1238 (the 25th year of King Gojong of Goryeo’s reign), it was completely destroyed during the Mongolian invasion. Only the site (Historic Site No. 6) remains today.
Research on Hwangnyongsa Temple began with the 1976 archaeological excavation, which led to a full-scale research project that started in 2005. From 2012 to 2013, researchers created a small-scale model of the temple to determine the feasibility of restoration. In 2015, the 3D restoration of all buildings, including the Middle Gate, was completed to create an overall picture of the temple. In this process, digital modeling techniques were extensively applied using 3D modeling programs such as Autodesk Inventor. Based on this 3D restoration, researchers began to design the Middle Gate using the building information modeling (BIM) technique. In addition, research on detailed aspects of the structure, such as roof tiles and metal architectural artifacts, was conducted and reflected in the design. In 2018, as an initial task, the Middle Gate was restored, using AR technology, as a two-story building with a hipped roof. Later, from 2019 to 2020, the Middle Gate was restored again as a single-story building with a gable roof, and the Southern Roofed Corridor was also added.
Information and Edutainment
Visitors to the Hwangnyongsa Temple Site can rent a tablet PC on the spot to experience an AR program that allows them to walk around the Middle Gate and the Southern Roofed Corridor. This AR program also comes packaged with diverse “edutainment” content, specifically, a treasure hunt that requires finding artifacts unearthed from the Hwangnyongsa Temple Site, background effects showcasing the temple’s four seasons, and a docent tour led by Gyeongju City’s mascot.
AR program users can take photos using Hwangnyongsa Temple as the background and send them via email. They can also appreciate the Hwangnyongsa Temple Site as it appeared during the 2017 archaeological excavation, a restoration created using 3D scan data of the excavation. Furthermore, they can magnify what they see and enjoy an animation that shows a building’s wooden structural elements being assembled in the order of construction. The AR program also provides explanations about construction materials, allowing visitors to learn about wooden materials used in ancient Korean construction.
 |  |  |
| Treasure Hunt | Experiencing the Four Seasons | Docent Tour |
 |  |  |
| Taking and sending photos | Screenshot of an AR-restored archaeological feature | Animation of wooden structure |
Upgrading the AR Experience
In the past, AR restoration work generally suffered from occlusion and perspective issues where a virtually restored building appears in front of a person even if they are standing in front of the building.
In order to address this problem, our AR restoration team enhanced the sense of perspective and made the experience much more realistic by calculating and factoring in the distance between the user and the virtual buildings. The AR restoration team also implemented time-related changes to shadows and diversified textures to provide the user with the most lifelike experience of roaming around the buildings inside Hwangnyongsa Temple.
At the same time, by using marker recognition and camera-based location tracking, instead of more commonly used GPS technologies, the team was able to ensure the locational accuracy of virtually restored buildings. Moreover, the level of detail (LOD) technique, which enables the adjustment of rendering workload by distance, was applied for the efficient use of large data sets.
  |  |
| Application of LOD technique (3 phases) | Application of LOD technique (3 phases) |
 |  |
| Before adding baked lighting | After adding baked lighting |
 |  |
| Image marker recognition | Photo featuring a perspective-corrected AR image |