LiDAR sensors × AR evolving site management: visualizing 3D data on the spot

In construction and civil engineering, as-built quality control involves measuring the shape and dimensions of completed structures and earthworks to confirm whether they were built according to the design drawings. Traditionally, this verification work has involved measuring heights and thicknesses point by point using a total station (TS) or a level, then returning to the office to compare the data with the drawings to decide pass/fail. While this is an indispensable step for quality assurance, bringing measurement point data back from the site before checking often creates a time lag until problems are discovered, causing rework. In addition, such as-built inspections tend to depend on skilled technicians, and with labor shortages and an aging workforce, there is strong demand for improved efficiency.

In recent years, AR (Augmented Reality) technology has attracted attention as a solution to these as-built management challenges. AR is a technology that overlays three-dimensional digital information on real-world images. Once confined to advanced research, AR has become usable in everyday construction management thanks to the improved performance of smartphones and tablets. In particular, the latest smartphones (for example, iPhone or iPad Pro) are equipped not only with high-performance cameras but also LiDAR sensors, and AR apps for these devices now make it possible to intuitively check as-built conditions on site. With industry-wide DX (digital transformation) initiatives such as the Ministry of Land, Infrastructure, Transport and Tourism-led i-Construction, AR is increasingly expected to be a powerful solution that simultaneously improves site efficiency and quality.

Challenges of traditional methods: inefficiency and lack of immediacy

First, let’s summarize the typical issues inherent in traditional as-built management methods. Site personnel will recognize many of these points.

• Work takes time: Setting up surveying equipment and measuring point by point with multiple people can take an entire day on large sites or those with many measurement points. The manpower and time burden is large, hindering productivity improvements.

• Time lag in problem detection: Measurement results must be taken back and compared with drawings, so the quality of the work cannot be judged on the spot. Many cases reveal nonconformities only after data processing, forcing inefficient returns to the site for corrections.

• Dependence on experienced technicians: Surveying and as-built inspection tasks tend to rely on experienced technicians and carry the risk of human error. With shortages of veteran technicians and insufficient experience among younger staff worsening, continuing traditional, person-dependent methods is becoming difficult.

What is a LiDAR sensor?

A LiDAR sensor is a measurement technology that determines distance by emitting laser light toward a target and measuring the reflection. LiDAR stands for “Light Detection and Ranging”; it rapidly emits countless laser pulses and calculates distances to points in space from the time it takes for the pulses to return. The resulting data is called a point cloud. A point cloud is a collection of many points in three-dimensional space, each point having X, Y, Z coordinates (and sometimes RGB color information). When buildings or terrain are converted into point clouds, surface shapes are reproduced as collections of countless points, allowing digital acquisition of 3D models that closely resemble the real thing. From a captured point cloud, you can measure arbitrary dimensions or create cross-sections, making it powerful for site record-keeping and as-built verification. LiDAR sensors are used for surrounding detection in autonomous vehicles and for 3D mapping; recently, their inclusion in smartphones has drawn attention as a technology that enables anyone to perform 3D scans easily.

Differences between smartphone LiDAR and terrestrial LiDAR

There are the following differences between small LiDAR sensors built into smartphones and high-performance tripod-mounted 3D laser scanners (terrestrial LiDAR).

• Smartphone-mounted LiDAR: LiDAR sensors built into the latest iPhones and iPad Pro devices can measure the surrounding environment in 3D in real time just by pointing the device. The measurable distance is roughly limited to a radius of about 5 m (16.4 ft), but by integrating scans from multiple positions you can capture the overall shape of interiors or small structures. The revolutionary ease of being able to scan on the spot while viewing the point cloud on the smartphone screen gives mobility that allows measurement in confined or elevated spaces with just the phone. The low introduction cost and the ability to “take it out of your pocket and measure immediately when needed” are major advantages without requiring expensive dedicated equipment. However, smartphone LiDAR has reduced laser output for safety, so the obtainable point cloud density and accuracy do not match professional equipment. While it lags slightly in reproducing fine details and in noise levels compared to specialized devices, it can measure with errors on the order of a few cm (a few in), which is sufficient for many construction management uses.

• Terrestrial 3D laser scanners: High-precision LiDAR devices used on a tripod can measure distant points tens to hundreds of meters away with millimeter-level accuracy. These devices are large and very expensive, requiring investments of several million to several tens of millions of yen for acquisition. When scanning a wide area, you must measure from multiple positions and later merge the acquired point clouds. Transporting and setting up the equipment and processing the data requires expertise and effort, making application to small sites or routine surveys a high hurdle. On the other hand, for tasks requiring high accuracy and wide ranges—such as detailed documentation of complex plant facilities or large-scale earthwork topography—these tools remain indispensable. In short, “smartphone LiDAR emphasizes convenience, while tripod-mounted LiDAR emphasizes accuracy and range.” Recently, drone-mounted LiDAR and photogrammetry have also appeared, and combining these methods for large-scale 3D site surveys is becoming more common.

What is AR (Augmented Reality)?

AR (Augmented Reality) is a technology that overlays virtual 3D models or information on real landscapes. By using a smartphone or tablet camera image along with the device’s pose sensors and image recognition technologies, virtual objects are composited and displayed at designated positions in real space. In other words, digital information such as design data or guidance is pasted onto the real world as seen through the screen. For example, on a construction site you can overlay a pre-created 3D design model onto the actual scenery to visualize the finished form, making it easy to grasp the intended outcome. Spatial relationships that were hard to understand from paper drawings or renderings become clearer when you see a life-size model in place. AR experiments once centered on specialized AR goggles or markers, but now that high-precision alignment is possible with just a smartphone, AR has entered practical use even on outdoor construction sites. AR technology allows three-dimensional data to be “visualized on the spot” in various situations from on-site consensus building to construction instructions and inspections.

3D point cloud measurement: digitally recording the current state with LiDAR

The first scenario is recording the current state of a site as three-dimensional data. Traditionally, understanding the shape of terrain or structures required measuring many points manually, but with a LiDAR scanner you can capture the entire current state as a point cloud simply by scanning the surroundings. With smartphone-mounted LiDAR, holding the device and walking around allows continuous measurement of the ground and structures, and the 3D point cloud is displayed on the screen in real time. For example, scanning the ground before excavation or the in-progress as-built condition with a smartphone enables later detailed dimensional checks and cross-sectional analysis at any time. LiDAR measurement also improves safety because you can measure hazardous areas from a distance where people cannot enter. Once point cloud data is acquired, analyses such as earthwork volume calculations and area computations can be automated digitally. In practice, sites that have introduced smartphone point-cloud systems have reported dramatic reductions in surveying time—up to about 90% shorter in some cases—greatly improving the efficiency of current-state understanding.

AR display of design models: overlaying 3D drawings onto real space

Next is the scenario of displaying a 3D model created during the design phase directly onto the site scenery. By loading design model data (BIM/CIM models or 3D CAD drawings) into an AR app on a smartphone or tablet, you can display the completed structure in the actual scenery as if it were already there. For example, placing an AR model of a building on a vacant lot lets you intuitively check the building’s height and its relationship with the surroundings. Points that were hard to visualize from drawings or perspectives become easier to understand when everyone can see a life-size completed form on the spot, facilitating common understanding among stakeholders. This smooths on-site consensus building and design review, helping to prevent rework. Overlaying design models on reality also allows the identification of potential clashes during the planning stage and the early detection of aesthetic or landscape issues. AR visualization of design models is effective for explaining projects to clients and local residents; sharing a completed-image that paper materials cannot convey easily helps gain understanding and cooperation for the project.

AR navigation for construction guidance: improving accuracy with digital guidance

AR also works as a navigation tool for construction. Traditionally, layout (staking) work involved surveyors placing stakes or markings on site based on coordinates in drawings, but AR can digitally replace this process. In an AR app linked to high-precision position information, reference lines or outlines from design drawings can be overlaid on live site footage to guide workers to designated positions. For example, target points and lines for construction can be displayed on a tablet’s camera view, and users can follow on-screen guides to place materials or direct machinery. Even on sites with complex curves or elevation changes, following AR instructions enables precise alignment, greatly reducing the burden of traditional surveying and marking. Even less experienced workers can rely on AR to tell them “what to place where next” like a car navigation system, helping reduce human error and standardize work quality. In practice, cases have begun to appear in which smartphone AR has been used to guide pile-driving positions or pipe routes, enabling faster and more reliable construction than before. AR navigation guides reduce the burden on construction managers and improve the team’s overall efficiency and accuracy.

Comparing as-built with point cloud data: on-the-spot quality checks

Finally, consider the scenario of immediately checking quality by comparing the as-built with point cloud data. Traditionally, after completion the heights and dimensions of measured points were compared to drawings to judge pass/fail, but combining AR and point cloud technologies enables this inspection to be performed in real time. Specifically, you scan the completed structure or terrain with LiDAR to obtain point cloud data, then overlay that point cloud onto the design model for AR display on the spot. This lets you visually check whether the as-built matches the design and intuitively recognize any deviations. For example, scanning a paved road and comparing it to the design elevation makes it possible to color-code areas with insufficient thickness. Because differences between the actual construction and the design are immediately visible in AR, any construction errors or unevenness can be discovered on the spot. This enables immediate correction instructions while machinery and workers are still on site, preventing the waste of discovering issues later and redoing work. Point-cloud and AR-based as-built comparison dramatically increases the immediacy of quality control and contributes to “zero rework” (eliminating rework). Moreover, verification results are stored as digital data, streamlining the creation of as-built documentation. The ability to perform real-time inspection and record-keeping on site is another major advantage of LiDAR × AR utilization.

Ensuring positional accuracy by linking with high-precision GNSS (RTK / LRTK)

To realize the scenarios described above in practice, technology that can position devices with high accuracy is essential. Standalone smartphone GPS can have errors of several meters (several ft), which is insufficient for correctly overlaying design models on real space or detecting as-built differences at the centimeter level. The key here is the use of RTK-GNSS (Real-Time Kinematic positioning). RTK uses correction information from a base station to correct GPS and other positioning errors in real time, achieving planar positioning accuracy on the order of 2–3 cm (0.8–1.2 in). Recently, ultra-compact GNSS receivers that support RTK have appeared, allowing smartphone or tablet attachment for easy use of centimeter-level positioning.

For example, a startup originating from Tokyo Institute of Technology developed a device called “LRTK Phone,” a thin GNSS receiver weighing just 125 g that can be attached to an iPhone with one touch, achieving positioning accuracy comparable to traditional tripod-mounted surveying instruments (errors within a few cm (a few in)). It also supports the centimeter-level positioning augmentation service (CLAS) provided by Japan’s Quasi-Zenith Satellite System (QZSS), enabling high accuracy to be maintained even in mountainous or maritime areas where cellular signals are unavailable. By combining such high-precision GNSS devices, a smartphone can essentially become a “universal surveying instrument,” allowing anyone to obtain 3D data with accurate position information anywhere.

The benefits of smartphone + RTK position accuracy are enormous. Since every acquired point cloud can be assigned exact latitude, longitude, and elevation in a public coordinate system, the need to reconcile local coordinates with drawing coordinates is eliminated. Also, knowing your position at cm level accuracy (half-inch accuracy) at all times prevents point cloud distortion even during long scans. In AR display of design models, this allows you to align the models precisely with real-world coordinates. In other words, linking with high-precision GNSS eliminates accuracy concerns in LiDAR measurement and AR utilization, greatly enhancing practical usability on site. Moreover, these GNSS devices are very affordable—only a fraction of the cost of traditional professional equipment—making “one device per person” deployment realistic. If all site staff carry RTK-enabled smartphones, waiting times for surveying would be nearly zero, speeding up the entire construction cycle. Indeed, smartphone-mounted GNSS devices like LRTK are quietly becoming popular among field engineers, who note that “this single device completes site surveying.”

Effects of LiDAR × AR adoption and future outlook

Introducing LiDAR sensors and AR on site will significantly change construction management. The greatest effect is increased productivity. Time required for point cloud measurement and as-built inspection is dramatically reduced (as noted above, surveying time can decrease to a fraction of what it used to), and reduced wait times plus rapid feedback contribute to shorter construction schedules. At the same time, quality improvements are expected. Because construction errors can be discovered and corrected on the spot, variability in as-built results decreases and rework is suppressed. Intuitive AR guidance also raises workmanship accuracy, contributing to stable final product quality.

Improvements in safety should not be overlooked. Hazardous areas that people previously had to enter to measure can now be measured remotely with LiDAR, reducing risk. If the locations of underground buried utilities are recorded as point cloud data and displayed in AR, this can help prevent accidents like accidentally damaging cables during excavation. Digital utilization also helps address labor shortages. With smartphone + AR, even newcomers can easily operate the system and perform tasks with a reliable level of accuracy without relying on veteran intuition. In fact, smartphone surveying apps have received top ratings in the Ministry of Land, Infrastructure, Transport and Tourism’s NETIS (New Technology Information System) for being “easy for less-experienced staff to measure,” and are recognized as technologies that help young workers become immediately effective. LiDAR × AR site adoption can be a solution both for labor-saving and skills transfer.

Looking ahead, these technologies are expected to become increasingly refined and widespread. Beyond improvements in smartphone performance, the combination of 5G communications and cloud processing will build the infrastructure to utilize the enormous 3D data obtained on site in real time. In the future, AR-capable smart glasses that display necessary information in the user’s field of view while both hands are free may become commonplace. Above all, what matters is that a “user-friendly 3D site management tool available to everyone” is already within reach. Cost and usability barriers have never been lower. In fact, by combining the latest smartphones and apps with high-precision GNSS, you can start using 3D data on your site today. Once you experience this new approach, many site engineers will feel that “there’s no going back to the old way.” LiDAR sensor × AR site management is an innovative effort that elevates efficiency, quality, and safety to the next level. The fusion of digital technology and field expertise will significantly evolve construction sites going forward.