Powerful even for thickness inspections of tunnels and dams! On-site verification of structures with AR heat maps

Table of contents

• Introduction

• What is an AR heat map

• Challenges of conventional thickness inspections

• How AR heat maps work and preparation

• Use of AR heat maps for tunnel thickness inspections

• Use of AR heat maps for dam thickness inspections

• Benefits of introducing AR heat maps

• Simple surveying with LRTK

• FAQ

Introduction

In civil engineering and construction, thickness inspections of large structures such as tunnels and dams are an indispensable process for ensuring safety and quality. This work confirms whether concrete lining or the structural body thickness meets the design, but conventional methods have required considerable effort and time from measurement through evaluation. Recently, an advanced technology called an AR heat map has appeared in the field of thickness inspections, attracting attention as a revolutionary method that can visualize the condition of structures on site.

By using AR (Augmented Reality) heat map display, you can instantly see which parts of a tunnel inner wall or dam face are under-thickness or over-thickness. Inspection tasks that traditionally required comparing drawings and numerical data and marking on site can, with AR heat maps, be confirmed by overlaying the visualization on the actual object through a smartphone or tablet screen, enabling intuitive and speedy on-site responses. This article explains the overview and benefits of using AR heat maps for thickness inspections, and concludes by introducing LRTK, a simple surveying solution that makes it easy to adopt such advanced technologies.

What is an AR heat map

An AR heat map is a technology that overlays a “heat map,” which represents measurement data as a color distribution, onto real space. First, a heat map is a chart that visualizes the distribution or differences of numerical data with a color gradient; in civil engineering it is often used to color-code deviations between as-built (completed shape) and design values. For example, if you color areas thinner than the design red and areas thicker than the design blue, you can immediately identify locations that fall outside the allowable range. Overlaying this heat map onto the actual structure using AR is what we call an “AR heat map.”

When you point a tablet or smartphone camera at a tunnel wall, the screen shows the real wall image with the thickness distribution heat map overlaid. Because you can confirm the color-coded results on site while viewing the actual object, there is no need to stare at lists of numbers or plan views to identify locations. Modern mobile devices include high-performance cameras and LiDAR sensors, and AR apps that utilize these make it possible to view heat maps on site that previously could only be seen on a PC.

Challenges of conventional thickness inspections

When thickness inspections of large structures are performed with traditional methods, the following challenges have been common.

• Time-consuming and labor-intensive: Workers measured thickness point by point using total stations or measuring rods and recorded the results on paper. Covering an entire tunnel circumference or a giant dam face takes a great deal of time, and it is not uncommon for the process from measurement to drawing comparison and judgment to take several days.

• Lack of immediacy leads to rework: Even if construction defects such as insufficient thickness exist, it was difficult to detect them on site. Problems were often only noticed after bringing data back for analysis, by which time the concrete may have already hardened or heavy equipment removed, resulting in rework such as additional construction or re-grinding at a later date.

• Dependence on experienced technicians: Judging the adequacy of concrete thickness requires the intuition and skills of experienced surveying technicians; if qualified personnel are lacking, inspections become a bottleneck. With labor shortages and an aging workforce, it is not easy to always have skilled personnel on site.

• Burden of high-place and large-area measurements: Checking thickness on ceilings requires access via high-reach work platforms, so the measurement work itself involves physical burden and safety risks. Also, because measurements are limited to certain points, there is a risk of missing thin areas between measured points.

• Cost and burden of specialized equipment: Accurate thickness measurement requires high-precision laser scanners or GNSS positioning devices, which have high initial costs and pose a hurdle for small and medium contractors. Maintenance costs and theft risk further increase the cost burden.

• Record organization and reporting workload: Creating drawings of thickness distribution and preparing reports from measurement data is also required. The cumbersome steps of marking on site and later drawing cross-sections in CAD place a large burden on field personnel.

How AR heat maps work and preparation

To realize AR heat maps, you need to prepare digital data in advance and arrange on-site measurement and display. The general workflow is as follows.

• Prepare reference data: Prepare 3D reference data for comparison. For new construction, the design-stage 3D model or drawing data such as BIM/CIM serves as the reference. For inspections of existing structures, past as-built data or design drawings are used as reference data.

• On-site 3D measurement: Perform 3D measurement of the structure to be inspected on site. Scan the interior of tunnels or dam surfaces with LiDAR scanners or photogrammetry to obtain point cloud data, or measure positions and shapes at various locations with a smartphone equipped with high-precision GNSS such as LRTK. At this time, acquire sufficient point clouds and measurement points to cover the whole as much as possible.

• Generate heat map data: In the cloud or specialized software, compare the reference data (design shape) with the current point cloud data to create a heat map that color-codes thickness and shape differences. The displacement at each point is numerically calculated and output as a color map—red, blue, etc.—according to thresholds. With high-precision point cloud measurement data, automatic processing can produce precise heat maps in a short time.

• Load into AR device and align position: Load the created heat map into a tablet or smartphone AR app and prepare to overlay it on the actual object on site. Accurate overlay requires matching the data coordinates with real-space coordinates. Either attach positioning coordinates to the measurement data in advance or set markers or known points on site and map them in the app to align positions. Systems that support high-precision positioning with RTK-GNSS can automatically achieve highly accurate overlays based on coordinate information.

• AR display and confirmation on site: Once preparation is complete, point the camera at the actual structure and start the AR display. The heat map is overlaid on the real space on the screen, and under-thickness and over-thickness areas are emphasized in real time. Personnel can view that image and immediately perform additional work (repair thin areas) or corrective work (grind over-thick areas) as needed.

Use of AR heat maps for tunnel thickness inspections

In tunnel construction, the thickness of sprayed concrete (secondary lining) applied to the inner surface after excavation is rigorously inspected to ensure it meets design standards. Traditionally, the finished tunnel cross-section was measured at intervals of several meters and differences from the design cross-section were checked on drawings to find areas of insufficient thickness. Using AR heat maps, a single scan of the tunnel can visualize the thickness distribution of the entire circumference, instantly revealing insufficient areas and excessive bulges.

For example, if the concrete at a certain point on the tunnel ceiling is 5 cm (2.0 in) thinner than the design, that area will be shown in red on the AR heat map. The person in charge can identify the location from the screen and immediately order additional spraying. Conversely, areas that protrude thicker than the design are shown in blue and are immediately recognized as locations needing chiseling (surface grinding). Problems that might previously have been discovered the next day can now be identified on site immediately after construction, making it much easier to perform repair work before the concrete hardens.

Because AR heat maps have high positional accuracy, they eliminate the need to mark walls or re-measure points repeatedly. Everyone can intuitively share which parts need how much correction, preventing missed work. Even in long tunnels, scanning and AR display allow thickness inspections to be completed in a short time, significantly shortening the PDCA cycle of construction management.

Use of AR heat maps for dam thickness inspections

AR heat maps are also effective for large-scale structures such as dams. For concrete dams, it is necessary to check after each placement that the thickness conforms to the design shape. Traditionally, after concrete placement, measurement teams measured thickness at many locations and compared them with the design cross-section to evaluate deficits. Using AR heat maps, you can scan the entire dam surface to obtain as-built data and evaluate thickness distribution all at once.

For example, by 3D scanning the downstream face of a dam and displaying the differences from the design model as a heat map, subtle unevenness or wear that is not obvious at a glance becomes apparent. Areas where the concrete surface has thinned due to long-term wear or areas that have become over-thick after repair material application also become immediately visible by color differences. On site, a person in charge can look up at the dam face through a tablet and confirm red-marked deteriorated areas or blue-marked excess areas. Based on the results, they can quickly plan necessary reinforcement work or prioritize repairs for high-risk areas.

Visualization by AR heat maps is also effective for information sharing with clients such as municipal staff and managers. The structural health that used to be explained through technical reports and drawings can be intuitively understood by anyone when viewing the colorized AR display together. Bringing a large display to the site or handing a tablet to reviewers to check can facilitate on-site consensus building and decision-making.

Benefits of introducing AR heat maps

By incorporating AR heat maps into thickness inspections, you can obtain many benefits such as the following.

• Immediate detection and faster rework: Because problem areas can be visualized on site immediately after construction, under-thickness or excessive bulges can be discovered and addressed at once. Correcting defects on the same day instead of discovering them later greatly reduces rework, shortening schedules and cutting costs.

• Comprehensive checks to prevent oversights: Heat map display based on 3D data such as point clouds enables surface-based inspection of the entire structure. Local thin areas that were previously missed between measurement points emerge as color changes, so nothing is overlooked. This ensures quality throughout large structures such as tunnels and dams.

• Intuitive and easy to understand: By overlaying color-coded results on the actual object, anyone—not just specialists—can understand the condition at a glance. It is easier to explain to junior engineers or clients without relying on veteran experience and intuition, smoothing communication on site. Visual sharing of inspection results makes it easier for all stakeholders to have a common understanding.

• Improved efficiency and safety: Inspection can be completed with a short scan and single-operator AR confirmation, allowing sizable reductions in personnel and equipment. One person can confirm with a smartphone in hand, minimizing the need for high-reach work or prolonged surveying that was previously necessary. This reduces worker burden and improves safety.

• Immediate use of digital data: The 3D measurement data and heat map results obtained on site can be stored and used as electronic data. Through cloud services, office staff can review on-site inspection results remotely, and reports can be auto-generated for electronic delivery. There is no need to rely on paper drawings or handwritten notes, promoting DX (digital transformation) of inspection work.

• Application to maintenance management: Created heat maps and point cloud data are retained as digital records after inspection. This enables the construction of a digital twin of the structure, which is useful for future periodic inspections and monitoring. Comparing aging changes against the as-built data taken at construction makes it possible to accurately grasp the progression of deterioration, benefiting long-term maintenance planning.

Simple surveying with LRTK

You might think, “But doesn’t this require advanced surveying equipment?” However, by using LRTK you can easily perform high-precision on-site surveying that includes AR heat maps. LRTK is a solution that transforms a smartphone into a surveying instrument with centimeter-level accuracy (half-inch accuracy) by attaching a small device to the phone. By mounting a dedicated device with an integrated high-sensitivity GNSS antenna on a smartphone and using real-time satellite positioning correction information (RTK), the smartphone GPS—which normally has meter-level errors—can be improved to centimeter-level accuracy. In other words, your everyday smartphone becomes a “versatile surveying instrument,” enabling precise on-site measurement without specialized equipment or veteran surveyors.

With LRTK, point cloud measurement that previously required a total station or large laser scanner can be completed with just a smartphone. Combined with the LiDAR scanner and high-performance camera built into iPhone or iPad Pro, you can obtain wide-area 3D point cloud data simply by waving the device around, and cloud processing can automatically perform volume calculations and heat map creation immediately after acquisition. Of course, the acquired point clouds and position data can be used directly for AR display, so thickness inspections of tunnels and dams described above can be performed one-stop without special effort.

Furthermore, LRTK is a system that complies with new standards such as the Ministry of Land, Infrastructure, Transport and Tourism’s “3D As-Built Management Guidelines (draft).” It can perform measurements with the positioning accuracy and data format required for public works, and offers features to output as-built reports with heat maps on the cloud with a single click. It supports the entire process from simple surveying to inspection reporting, dramatically improving efficiency from on-site AR heat map confirmation to formal inspection documentation.

FAQ

Q: What is needed to use AR heat maps on site? A: Basically, you need an AR-capable device (an AR-compatible smartphone or tablet) and 3D measurement data of the structure. Measurement equipment such as laser scanners or drone photogrammetry can be used to obtain point cloud data, but recently it has become easy to acquire high-precision point clouds with the LiDAR function of iPhones and iPads or with smartphone-attached devices like LRTK. After generating a heat map from the acquired point cloud data and the design data, prepare a dedicated app (a cloud-linked AR app) to display it, and you can experience AR heat maps on site. The important task is calibration to align data with real-space coordinates, but high-precision GNSS or known reference points can be used to smoothly perform position alignment.

Q: Is the accuracy and reliability of AR heat maps sufficient? A: If measurement and position alignment are performed properly, AR heat maps can be as reliable as or more reliable than conventional survey drawings. The key is using high-precision positioning data and high-density point clouds to ensure the accuracy of the heat map itself. With RTK-GNSS positioning such as LRTK that achieves centimeter-level accuracy, AR display misalignment is kept extremely small. The Ministry of Land, Infrastructure, Transport and Tourism is also promoting as-built management methods using 3D measurement, and inspections using heat maps plus AR meet official guidelines as an advanced method. There are cases where inspections have been completed based on AR heat map confirmations, so the accuracy is sufficient for practical construction management.

Q: Can AR heat maps be used for maintenance management of existing structures? A: Yes. AR heat maps are effective not only for new construction but also as a tool for periodic inspections and health assessments of existing infrastructure. For example, you can scan an in-service tunnel to examine aging changes or periodically capture point cloud measurements of a decades-old dam to monitor the progression of cracks and surface wear. By comparing past inspection point clouds with the latest measurement data and displaying the differences as a heat map, you can intuitively see how much each part has deteriorated. Projecting that on site with AR lets you immediately identify areas in need of repair, enabling efficient and accurate maintenance management.

Q: I’m worried that the cost and barriers to introducing AR heat maps are high. A: Compared with conventional ICT equipment, the barriers to introducing AR heat maps have steadily decreased. You don’t need to assemble expensive specialized surveying equipment; as mentioned, combinations of off-the-shelf smartphones and small devices like LRTK can achieve the solution, greatly reducing initial investment. The operation is also designed to be usable without specialized knowledge: simply launch the app and follow the prompts to complete measurement and display. Using cloud services removes the need for complex on-site processing, and as long as data connectivity is available, anyone can start using it immediately. In sites suffering from an aging workforce and labor shortages, younger staff can often substitute with smartphone operation, making this a cost-effective solution overall.

Q: If I confirm things on site with an AR heat map, can I skip creating reporting drawings and documents? A: While on-site confirmation with an AR heat map can greatly simplify intermediate recording tasks (such as on-site photography or hand-drawn sketches), final inspection deliverables must still be submitted according to contracts and regulations. However, don’t worry: systems that support AR heat maps provide functions to automatically output reports from the obtained point cloud data and heat map results. For example, LRTK’s cloud can generate as-built management charts and forms with heat maps with one click, dramatically reducing the effort needed to prepare inspection documents. In other words, you can build a smart inspection workflow that relies on digital confirmation on site and obtains the necessary reports at the push of a button, virtually eliminating paper-based work.