Achieving Improved AR Integration Accuracy with High-Precision Point Cloud Acquisition by LRTK and Gaussian Smoothing

Background: Challenges and Potential of AR and Point Cloud Integration

Augmented reality (AR) technology is gaining attention on construction and surveying sites. AR, which can superimpose virtual 3D models and information over a site simply by holding up a smartphone or tablet, is a powerful tool for intuitively sharing blueprints and CIM models. However, a major challenge for practical AR use is the accuracy of alignment with the real space. For example, when projecting a building design model onto the site, if the model and the actual structure are misaligned by just tens of centimeters, users may see the model floating or buried in the ground. Even slight display jitter can interfere with precise verification or work.

To solve this alignment accuracy issue, it is important to measure the real space with high precision and use that as the AR reference. This is where the use of 3D point clouds becomes noteworthy. A point cloud is 3D data composed of countless points acquired by laser scanners or LiDAR, recording the shape of the real world in detail. If a point cloud is used as the foundation for AR, differences between the virtual model and reality could potentially be detected in millimeter-level detail. However, historically acquiring point clouds required expensive equipment and specialist skills, and the obtained point cloud data included noise and errors. Because of these issues, combining point clouds with AR sometimes still produced jitter and positional offsets, preventing the expected accuracy improvements.

A recent solution to these challenges is the combination of high-precision point cloud acquisition using LRTK and applying Gaussian smoothing (Gaussian filter) to that data. LRTK is a system that makes it easy for anyone to acquire high-precision point clouds with absolute coordinates by using RTK-GNSS technology. By applying Gaussian-kernel-based smoothing to these point clouds, the accuracy of AR integration can be dramatically improved. This article explains the mechanism and effects in detail.

What Is an RTK-GNSS-Referenced High-Precision Point Cloud Acquired by LRTK?

LRTK is a point cloud measurement system that combines a smartphone’s (for example, an iPhone’s) built-in LiDAR sensor or camera with high-precision positioning from real-time kinematic GPS (RTK-GNSS). By attaching a dedicated receiver weighing about 165g to a smartphone and using a proprietary app, anyone can easily achieve centimeter-class positioning and 3D scanning that previously required specialized instruments. By utilizing RTK-GNSS correction information, position errors that usually occur on consumer smartphone GPS—on the order of 5–10 m (16.4–32.8 ft)—have been reduced to about ±1–2 cm (±0.4–0.8 in) horizontally and about ±3 cm (±1.2 in) vertically. Each point in the acquired point cloud is assigned absolute coordinates consisting of latitude/longitude and elevation, so the point cloud itself becomes a high-precision 3D map that aligns with the site coordinate system (public coordinates or any local coordinate system).

LRTK can acquire such high-precision point clouds with surprisingly simple procedures. For example, even on a wide road or slope, you can scan point clouds covering tens to hundreds of meters (tens to hundreds of ft) in the surrounding area in a short time—sometimes in about 1–2 minutes—simply by walking while holding the smartphone. Since all acquired point clouds include global position information, there is no need for post-processing alignment. In other words, it is like obtaining a “digital replica” of the entire site measured in situ. These high-precision point clouds from LRTK are extremely effective as the basis for AR integration. If 3D design models (BIM/CIM, etc.) are created in the same coordinate system, simply overlaying the acquired point cloud and the design model will result in perfect positional alignment, eliminating the need for complicated alignment work. Also, securing accuracy in the vertical direction reduces the risk of models sinking into or floating above the ground.

Spatial Noise in Point Clouds and Problems for AR Use

Even though point clouds acquired by LRTK are high-precision, as with any sensor measurements, they are not free from noise or error. Spatial noise in a point cloud refers to variations of points such as unevenness appearing on surfaces that should be flat or shapes that should concentrate at a single location spreading slightly. Causes include LiDAR ranging errors, environmental factors (variations in reflectivity or occlusion), and small device movements. For example, a point cloud scanned from a perfectly flat concrete floor may still show point heights varying by several centimeters (several in), spreading like a cloud. Using such noisy point clouds directly in AR can cause problems.

First, when detecting or recognizing planes or objects in the real world from a point cloud, noise can cause misdetection and fluctuation. AR apps detect planes in the environment to place virtual objects, but if the floor point cloud is bumpy, it will not be recognized as an accurate plane, causing virtual objects to wobble.

Second, if the point cloud itself is used as the background “reality” for AR rendering, the point distribution variation due to noise will appear as AR overlay jitter. Imagine that just by moving the device slightly, the virtual model and the point cloud appear to shift relative to each other. This is caused by local errors in the point cloud making the relative positions between the virtual model and the camera viewpoint vary irregularly.

Third, when comparing the acquired point cloud with a design model to evaluate consistency, noise adversely affects the result. Even where things should match exactly, slight error-induced offsets occur, making it difficult to judge what is construction deviation and what is measurement noise.

As described above, noise and variability in point clouds lower the accuracy of AR integration. Even with high-precision LRTK point clouds, millimeter- to centimeter-scale variability cannot be ignored. In the real world, a few centimeters are visually noticeable in AR as clear offsets or jitter of objects. Therefore, filtering of point cloud data is necessary. Among filters, smoothing—which averages each point’s position with its neighbors to produce a smoother result—is a fundamental and effective method for noise reduction. The next section explains the principle of a representative smoothing method: Gaussian smoothing.

Basics of Gaussian Smoothing: Spatial Weights and Kernel Mechanism

Gaussian smoothing is a process that smooths and corrects values (such as position coordinates) at each point in a point cloud by weighted averaging with neighboring points. The key is the weighting by the Gaussian kernel. The Gaussian kernel is a weighting function based on the normal distribution (the familiar “bell curve”) and gives larger weights to closer points and smaller weights to farther points. Specifically, a neighboring point at a distance d from the target point receives a weight proportional to exp(-d^2/(2σ^2)) (σ is the standard deviation of the Gaussian distribution). Performing such continuous distance-dependent weighting enables spatially smooth smoothing.

An image analogy is placing each point in the point cloud on a rubber sheet and adjusting positions as if they pull on each other with the surrounding points. Extremely isolated points (outliers) are hardly pulled by neighbors and thus standout noise is reduced. Conversely, dense clusters of points correct each other’s positions, resulting in smooth surfaces or curved surfaces overall. The advantage of Gaussian smoothing over a simple average filter is that it can adjust the degree of smoothness according to distance. Close points are largely preserved while the influence of distant points is suppressed, so edges (boundary parts) can be preserved as much as possible while reducing noise. Very sharp corners or fine details may become somewhat rounded, but for AR integration the benefits of noise removal usually outweigh slight shape changes due to smoothing.

Implementation and Parameter Tuning: Effects of Standard Deviation and Neighbor Selection

When applying Gaussian smoothing to point cloud data in practice, several parameters and implementation considerations are necessary. The main points are the standard deviation σ of the Gaussian kernel (the strength of smoothing) and the method for selecting neighboring points used in smoothing.

First, σ directly affects the range and degree of smoothing. The smaller σ is, the more rapidly the Gaussian weight decays, so only very nearby points are averaged. As a result, it removes fine noise while relatively preserving small bumps and edges. Conversely, a larger σ gives non-negligible weight to more distant points, averaging over a wider area. This smooths larger irregularities but may blur fine details (for example, edges of angular structures). The appropriate σ depends on the noise level of the point cloud and the scale of the features of interest. For removing fine noise in typical LiDAR point clouds, set σ on the order of several millimeters to several centimeters (a few tenths of an inch to a few in) and adjust while observing effects. The important point is to strike a balance that removes noise while preserving important shape features, avoiding values that are too large or too small.

Next, neighbor selection determines which surrounding points are considered “neighbors” for smoothing. Typical approaches choose points within a fixed radius or the k nearest points by distance. When selecting by radius, set that radius to be about the same as σ or larger so that enough neighboring points are included. If the radius is too small, few neighbors will be found and smoothing will be unstable. If it is too large, points from unrelated objects may be incorporated and distort the shape. When selecting by k-nearest, k should not be a few points; setting it moderately large (for example, 20 points or 50 points) provides a stable average. However, if point density varies across the point cloud, a fixed k can cause over-smoothing in sparse areas or under-smoothing in dense areas, so combining a radius limit and k can be effective.

Implementation must also consider computational efficiency. Performing neighbor searches and weight computations for all points becomes time-consuming for large point clouds. Using spatial indices such as KD-trees for neighbor search makes the computation efficient. In LRTK system cloud processing, uploaded point clouds can be automatically denoised and smoothed so that users obtain a certain level of effect without worrying about parameters. If processing locally, start with relatively weak smoothing and incrementally increase strength as needed.

Case of Reduced Jitter in AR Overlays Using Smoothed Point Cloud Models

So, how effective is Gaussian smoothing when applied to point clouds for AR display? Consider an example of a paved road point cloud. In the original point cloud acquired by LRTK, the seemingly flat road surface still contained height variations on the order of ± several centimeters (several in). When that point cloud was used as-is to display a virtual design line on the road in AR (for example, a centerline or finish elevation guide), the virtual line was observed to slightly move up and down as the device moved. The virtual line was being dragged by tiny bumps in the road surface point cloud, making it appear unstable with viewpoint changes.

Applying a Gaussian smoothing filter to that road point cloud averaged out the fine bumps. Specifically, smoothing was performed using a kernel with σ around 5 cm (≈2.0 in) over a radius on the order of several tens of cm (several tens of in). The smoothed point cloud showed an almost ideal smooth plane for the road surface, and the original centimeter-scale bumps became inconspicuous. Using this smoothed point cloud as the AR base, the previously observed virtual line became perfectly stable; even when moving the device, no perceptible jitter was felt. Users reported that “the virtual line looked as if it had really been painted on the ground,” indicating a natural overlap of reality and virtual content.

This example shows that Gaussian smoothing removes random error components from point cloud data and reduces jitter in AR overlays. Especially for LRTK point clouds with global coordinates, positional offsets are negligible, so remaining issues are local noise. Averaging out that noise through smoothing makes the appearance of the point cloud stable across consecutive frames, resulting in stable visibility of virtual objects. In field AR use, preventing display instability as users walk or change their gaze is important for safety and trustworthiness. Gaussian smoothing contributes significantly to such practical quality improvements.

Application to Consistency Evaluation between Point Clouds and CIM/3D Design Models

Gaussian smoothing is not only effective for stabilizing AR displays but also helps with consistency evaluation between point cloud data and design models (BIM/CIM). On construction sites, scanned point clouds of completed structures or terrain are compared with original design models to verify how closely the work follows plans. For smooth comparisons, it is desirable that the point cloud not contain excessive noise so that only the genuine differences from the design model stand out.

Raw, unsmoothed point clouds contain the aforementioned centimeter-level variations in many places. When overlaid with a design model, noise causes colored differences (for example, nonzero ± centimeter errors on a distance map) to scatter even in areas that should match, making true construction deviations hard to see amid measurement noise. By applying Gaussian smoothing to the point cloud, random variation components are greatly reduced, and clear trends appear on the difference map. For example, if part of the point cloud is uniformly 3 cm (1.2 in) higher than the model, it becomes easier to judge that the construction was indeed 3 cm high. Conversely, minute differences under ±1 cm (less than ±0.4 in) that were masked by ±3 cm noise prior to smoothing are no longer displayed (effectively ignored), thus removing noise factors from evaluation.

Gaussian kernels are also effective in point cloud–model alignment. ICP (Iterative Closest Point) algorithms, commonly used to align point clouds with each other or with models, compute transformations that minimize distances between corresponding points, but noisy point clouds often lead to incorrect correspondences and unstable convergence. With smoothed point clouds, outlier points have reduced influence through averaging, and alignment based on large-scale shapes becomes easier. In other words, by smoothing local irregularities with a Gaussian kernel, global alignment is facilitated.

Thus, Gaussian smoothing aids the entire comparison and alignment process between point clouds and CIM/3D models, enabling more reliable verification. When overlaying scanned point clouds and design data on site to determine “this area is slightly overbuilt compared to the drawings” or “the slope here is gentler than planned,” smoothing reduces the risk of misjudgment due to noise.

Effects of Introducing Gaussian Smoothing: Stable AR and High-Precision Matching

As discussed so far, combining high-precision point clouds with Gaussian smoothing brings about a qualitative leap in AR–point cloud integration. The introduction effects can be summarized as follows.

• Stabilization of AR overlays: Jitter caused by point cloud noise is suppressed, so virtual models overlaid in AR appear firmly fixed to the real world. There is no need to rely on markers or manual adjustments for alignment, allowing users to use AR without stress.

• High-precision model matching: Smoothed point clouds with global coordinates enable highly accurate overlap between site point cloud data and design models in the same coordinate system. As a result, differences between model and reality can be understood accurately at millimeter to centimeter levels, allowing AR to be used reliably for quality control and as-built inspection.

• Improved consistency of scanned data: Gaussian smoothing reduces inconsistencies and position fluctuations between consecutively acquired point cloud frames. Large point clouds scanned over time can be treated as a single smooth model, simplifying post-processing and analysis.

• Efficiency gains in field work: With point clouds that need no alignment and stable AR, on-site verification and instruction become significantly more efficient. For example, if a design model is displayed in AR during construction, measurement and confirmation tasks are reduced and early detection of mistakes is facilitated.

Thus, smoothed point cloud data greatly enhances the reliability and accuracy of AR use, effectively bridging the digital and real worlds.

Future of Field AR Utilizing LRTK × Gaussian Smoothing

The combination of high-precision point cloud acquisition by LRTK and Gaussian smoothing will likely drive AR utilization on-site to the next stage. Future developments may include:

• Real-time feedback: LRTK can provide cm-level accuracy (half-inch accuracy) position information in real time. Combining this with fast point cloud smoothing could enable building a smooth point cloud model on the fly while holding up a device and immediately reflecting it in AR. If surveying and verification are integrated in real time, instant checks and feedback could be performed throughout the construction process.

• Integration with AR glasses: Currently AR use is mainly on smartphones and tablets, but if AR glasses (smart glasses) become widespread, LRTK positioning data and smoothed point clouds could be linked to glasses so workers can always see high-precision AR displays hands-free. For example, if a BIM model is continuously overlaid on the site through glasses without shifting, tasks that previously relied on skilled workers’ judgment and sight could be digitally supported.

• Cloud sharing and remote support: By sharing point clouds and AR information acquired with LRTK via the cloud, managers or designers in the office can check model alignment on the site point cloud and issue instructions remotely. Smoothed point clouds reduce data size and communication load, making it realistic to support on-site AR from remote locations.

• New quality control standards: If the combination of high-precision point clouds and AR becomes common, standards for construction management and inspection may change. For example, tasks that used to use string lines and chalk lines for layout could be performed with AR-projected guidelines, or as-built inspection could be judged by color-coded deviations in AR. Stable point clouds achieved by smoothing would become an indispensable element.

In these ways, LRTK’s high-precision positioning and Gaussian smoothing technology expand the possibilities for AR utilization on site. AR with assured accuracy and stability will go beyond mere visualization and function as infrastructure for field operations.

First Steps for Practical Use: Start High-Precision Point Cloud Acquisition and Utilization with LRTK

We have introduced the effects of combining advanced point cloud processing and AR technology, but to obtain these benefits in actual projects, acquiring high-precision point cloud data is indispensable. With LRTK, you can easily start point cloud measurement with absolute coordinates that previously required specialized surveying instruments using just a smartphone. Preparation and operation are simple, and technicians without special training can perform 3D scans on site in a short time. Acquired point cloud data can be automatically uploaded to the cloud and used on the spot for measurements such as distance and area or for model overlay displays. If necessary, noise reduction processing by Gaussian smoothing can be applied to further improve accuracy.

The combination of high-precision point clouds and AR is expected to become increasingly important in practical work. As a technology that bridges digital and real, it has broad applications in construction management, infrastructure maintenance, urban planning, and disaster prevention. As a first step, why not start with simple surveying using LRTK? By precisely “visualizing” site conditions and intelligently smoothing and utilizing that data, you can achieve a stable AR experience and improved accuracy control.