How to Record Cultural Heritage in 3D with Smartphone LiDAR｜4 Steps to Avoid Failure

In the field of research and preservation of cultural heritage, there is a growing demand each year to capture shapes as quickly and as accurately as possible, and in a form that can be easily re-verified later. Traditionally, drawings, photographs, and measured values were managed separately, but recently the idea of using 3D data as a core to record shape, dimensions, spatial relationships, and surface condition together has become widespread. In the cultural heritage field, 3D recording is treated as a foundation for conservation, restoration, public display, and chronological comparison, and is also used for cross-section inspection, orthophoto generation, and tracking surface deterioration. Furthermore, internationally, digital recording of cultural heritage emphasizes a workflow that includes not only acquisition but also processing, optimization, visualization, and reuse.

Among these, smartphone LiDAR is one of the means that has made 3D documentation of cultural heritage more accessible. It does not have the versatility of large equipment, but it is easy to handle, allows one to grasp the overall shape of a site in a short time, and is suitable for initial recording, supplementary measurements, and routine record updates. On the other hand, results can be unstable for complex fine details, homogeneous surfaces, highly reflective materials, and objects with many missing parts, and simply thinking "scanning with a smartphone is sufficient" can easily lead to failure. In fact, research in the fields of cultural heritage and archaeology has recognized the effectiveness of smartphone LiDAR while also showing that it is necessary to choose methods according to target shape and surface conditions, to combine it with other techniques, and to devise acquisition procedures on-site.

Table of Contents

• The significance of 3D documentation of cultural heritage using smartphone LiDAR

• Step 1 Decide the purpose of the record and the required accuracy

• Step 2 Confirm on-site conditions and develop an acquisition plan

• Step 3 Combine scans and photos to reduce missing parts

• Step 4 Organize the data and save it in a form that can be reused

• Situations Where Smartphone LiDAR Is Likely to Fail

• A perspective for linking cultural heritage documentation to on-site operations

• Summary

The significance of 3D recording cultural heritage using smartphone LiDAR

The significance of preserving cultural heritage in 3D is not merely to make its appearance three-dimensional. What is important is transforming information that could only be understood by visiting the site into a form that can later be shared, compared, and examined by multiple people. For stone objects, the positional relationships of losses and wear; for buildings, the fitting of components and their tilts; and for archaeological remains, their connections to the surrounding terrain—these are easier to grasp three-dimensionally than from flat photographs. The emphasis on 3D documentation in the cultural heritage field stems from the fact that the records themselves become the basis for preservation and restoration decisions, and that data must be preserved in a form capable of withstanding future re-examination. Because 3D documentation also supports the production of drawings and cross sections, surface aging comparisons, educational use, and public engagement, it should be regarded not as merely a convenient visualization but as a foundation of preservation practice.

The reason smartphone LiDAR is attracting attention is that it makes it easier to nimbly advance this foundational work. While it is not as capable as large dedicated equipment at capturing wide areas or high-density data, the highest precision is not always required in cultural heritage surveys. In situations such as grasping current conditions, comparing before-and-after changes, reducing entry into hazardous areas, or first capturing the overall shape and then examining key areas in detail, the ability to quickly create 3D models is itself of great value. Even for large-scale cultural heritage sites, it is realistic to carry out an agile initial recording of the whole and then use other methods to delve deeper into only the important parts. Research also shows that mobile LiDAR is effective for recording and sharing cultural heritage, and that it is important to balance accuracy and work speed according to the subject and conditions.

Step 1: Decide the purpose of the record and the required accuracy

The first step to avoid failure is to decide in advance why you are doing 3D recording. If you start work with nothing more than the phrase “3D digitization of cultural heritage,” it’s easy on site to either capture a wider area than necessary or, conversely, fail to capture the required details. For example, whether the purpose is updating a conservation register, documenting the pre-restoration condition, comparing cracks and wear, or visualizing for exhibition or education will affect both the level of resolution needed and the areas that should be prioritized. Treating projects that only require grasping the overall shape the same as those that aim to trace fine surface relief or damage boundaries will inevitably cause problems in downstream processes. First, define the deliverables in words and explicitly state how much is required—whole models, partial details, cross-sectional checks, photo-based orthorectification (orthophotos), georeferenced records, etc. 3D documentation of cultural heritage should be considered not only in terms of geometric shape but also in terms of visual reproducibility and how it will be used in subsequent workflows.

The next thing to decide is how to think about the required level of accuracy. What’s important here is not aiming for the highest possible precision from the start, but judging what accuracy is sufficient for the purpose. In cultural heritage research focusing on smartphone LiDAR, studies have shown that, while it is effective as practical measurement documentation for relatively simple, not extremely complex objects, results can vary greatly depending on shape and surface conditions. In other words, although it is suitable for overall documentation and for auxiliary dimensional checks, it is risky to assume that a single device can handle extremely fine ornamentation or faithfully reproduce uniform small details. In Step 1, drawing a clear line on which tasks will be handled by smartphone LiDAR and which will be supplemented with photographs or other measurement methods is the most important point for avoiding rework.

Step 2: Check on-site conditions and develop an acquisition plan

In Step 2, prioritize the on-site conditions over the object itself. Failures in 3D documentation of cultural heritage are often due less to equipment limitations than to misreading the on-site conditions. Dark locations, backlighting, narrow passages, unstable footing, pedestrian traffic, vegetation occlusion, glossy or transparent surfaces, and monotonous surfaces with few patterns all destabilize acquisition quality. Research in the cultural heritage field also shows that environmental conditions, reflectivity, transparency, occlusion, lighting, and weather affect data quality, and when complex shapes or surface conditions are involved, gaps and distortions are especially likely. Rather than starting to scan aimlessly after arriving on site, it is essential to think in advance about which directions to approach from, which surfaces are prone to missing data, and which positions will allow close access.

In acquisition planning, it is practical to consider the subject not as a single block but divided into three layers: the overall, major parts, and details. First capture the overall outline, then move closer to important surfaces and areas with heavy deterioration, and finally fill in the details—this sequence reduces omissions. Also, while continuous close-range acquisition is well suited, moving too far away tends to reduce accuracy and density, so it is important to maintain a reasonable distance to the subject and move while being mindful of overlap. In research, mobile LiDAR and smartphone LiDAR offer high mobility, but final quality is governed by the acquisition conditions and the way scans are assembled. On site, don’t finish after just circling once like a single stroke; you need the mindset to predict surfaces likely to be missing and collect them two or three times from different angles.

Furthermore, for projects that require positional consistency, the approach to coordinates should be decided at this stage. A local 3D model is sufficient if you are only recording the shape of a single cultural property, but if you intend to link it to the entire site, surrounding terrain, existing drawings, and past records, providing a positional reference will make it far easier to handle later. Recent research has shown that by combining high-precision external positioning with smartphone RGB and LiDAR, in many cases it is possible to achieve accuracy close to that of conventional surveying. In other words, in cultural heritage practice it is highly effective to design the acquisition of 3D geometry and the assignment of positional references together during the on-site planning stage rather than treating them separately. MDPI

Step 3 Combine scans and photographs to reduce missing areas

The key point of Step 3 is not to try to complete everything with smartphone LiDAR alone. For 3D documentation of cultural heritage, both the correctness of the shape and the appearance of the surface are important. LiDAR is strong at capturing geometry, but photographic information is often advantageous for surface color, fine texture, and the visibility of degradation boundaries. Conversely, geometry can become unstable with photos alone. For this reason, in the cultural heritage field it has been repeatedly shown that combining LiDAR and photogrammetry, or using multiple methods, is effective for achieving both geometric accuracy and texture reproduction. Even when using smartphone LiDAR, it is better to first capture the shape, simultaneously secure sufficient photographs, and operate on the assumption that missing parts and surface information will be supplemented later—this increases the likelihood of success.

On-site capture should be done at a steady pace—neither rushing nor stopping too much—by moving around the subject. If you go too fast, surfaces can be skipped, and if you stop too much, pose estimation can become unstable. Also, rather than tracing a monotonous surface for a long time, keeping distinctive features such as corners, edges, steps, and damaged areas in view helps stabilize the model. For cultural heritage in particular, shaded parts of sculptures, under eaves, backsides, ground-contact areas, and narrow recesses that are hard to approach are prone to being missed, so it is important not to try to finish with a single frontal pass but to supplement with oblique angles and different heights. Research has also shown that smartphone LiDAR performance varies with the target’s shape and texture conditions, and simply circling once does not always yield practical results.

When using photographs, it is important to be conscious not only of visual beauty but also of taking images that can be used later for explanation. Having a set that includes shots showing the overall positional relationships, close-ups of damaged areas, mid-distance views that reveal relationships with the surroundings, and fixed-point photos that make it easy to compare differences in work dates and weather will supplement information that is difficult to read from a 3D model alone. In cultural heritage documentation, usability is more important than mere production. Considering that restorers, managers, researchers, and commissioning parties will look at the materials later, it is ideal that 3D data and photographs can be cross-referenced. The value of a 3D model is supported not only by the reproduction of geometry but also by the reproduction of appearance and the clarity of its explanations. Sai

Step 4 Organize the data and leave it in a reusable form

Step 4 can justifiably be considered a stage even more important than the fieldwork. Even if you have managed to create a 3D record, if filenames are ambiguous, it is unclear which data is the latest, and the capture date, coverage, or processing conditions are not recorded, the dataset will become an asset that cannot be used after a few months. Cultural heritage documentation is not a one-off activity; it is used for future comparison and re-survey, verification before and after restoration, and handover after personnel changes. From the international perspective on digital heritage, preservation is not something that ends at the point of acquisition but is regarded as a continuous activity that should be designed from the time of creation with management, discoverability, accessibility, and reusability in mind. Data should always be linked to and retain the object name, acquisition date, acquirer, coverage, acquisition method used, whether processing was performed, whether coordinates are present, and the location of related photographs.

The basic principle for preserving materials in a way that makes them easy to reuse is not to keep only a single finished model. It is better to manage separately the original photographs, the original point cloud or raw data, lightweight viewing data, report images, and, if necessary, derived data for cross-sections or dimensional verification so you can accommodate future changes in use. In cultural heritage, data acquired initially as a record of the current condition may later be repurposed for restoration planning, exhibition interpretation, public archives, or comparative research. That is why it is important not to compress everything to suit only the current purpose, but to preserve it in a structure that can be reread later. UNESCO’s recommendations also call for digital heritage to be managed with attention to metadata, standardization, and interoperability, assuming long-term preservation and access. UNESCO

Also, in 3D documentation of cultural heritage, it is important to be aware of where the original source ends and the processed result begins. Noise removal, hole filling, meshing, and color correction make models easier to view, but if overdone they can depart from the original condition. In the context of preservation and research, being able to trace how much processing has been applied can be more important than having an attractive model. To ensure records are trustworthy, keep a processing history and manage original and processed data separately so that, even if the site personnel change, you can explain why the object took that form. 3D documentation of cultural heritage is not merely three-dimensional promotional material; it also has a documentary character that can serve as evidence for future decision-making.

Situations Where Smartphone LiDAR Often Fails

Smartphone LiDAR often fails in situations where detailed reproduction is required but only an overall capture is performed. For example, information such as shallowly carved motifs, weathering boundaries, small losses, and slight steps can be difficult to read from overall model density alone. Even in cultural heritage research, smartphone LiDAR is effective for subjects that are not particularly complex, but results can be unstable for unusual shapes or homogeneous textures. In other words, although it is strong for overall documentation, it is risky to unconditionally rely on it for detailed diagnostics. If the goal of the survey is to capture fine details, you should switch from the outset to an acquisition plan that focuses on specific areas.

The second mistake is underestimating surface conditions. If you proceed thinking that dark areas can be fixed later, or that it’s sufficient to capture only the shape of glossy surfaces, you will actually end up with increased missing data and noise that cannot be salvaged in postprocessing. Cultural heritage objects are made of diverse materials—stone, wood, earth, metal, plaster—and their states of weathering and dirt are not uniform. Reflections, translucency, occlusion, and changes in illumination affect data quality, so rather than forcing a one-time capture, you should change angle or distance for problematic surfaces, supplement with photographs, or, if necessary, defer to another acquisition day.

The third is postponing spatial information. A standalone 3D model of a cultural property may work on its own, but the moment you want to overlay it with site plans, existing drawings, surrounding facilities, or past surveys it becomes problematic. This is especially true for outdoor cultural properties and spatially extensive remains: it is important to record not only the shape but also the location using a consistent reference. Recent reports indicate that combining smartphone-based capture with high-precision external positioning can provide comparable accuracy in many cases. In cultural-heritage documentation, taking one step beyond shape-only 3D and preserving models as 3D tied to location can greatly change the value of downstream processes. MDPI

Perspectives for bridging cultural heritage documentation to field operations

3D recording of cultural heritage does not end with acquisition; it only becomes meaningful when integrated into field operations. For example, if you look ahead to uses such as comparing the same area before and after restoration, mapping damage locations, sharing inaccessible surfaces with stakeholders, using the data as material for exhibition interpretation, or using it to examine surrounding circulation routes and protection zones, the required level of detail becomes apparent. UNESCO’s initiatives on digital cultural heritage also frame recording as a continuous process from acquisition through processing, visualization, and reuse. In other words, the purpose of introducing smartphone LiDAR is not only to make on-site work easier but also to improve the circulation and reuse of cultural heritage information.

In that sense, the integration of 3D records and positional information will become increasingly important. When you want to capture not only the cultural property itself but also the surrounding terrain, movement paths, boundaries, and related structures, simply having a 3D model as a standalone file tends to be insufficient. An effective approach is to link the acquired 3D data to local coordinates and management location information. By using LRTK, an iPhone-mounted GNSS high-precision positioning device, you can more easily proceed not only with recording the cultural property itself but also with confirming positions around it, organizing local coordinates, and laying the groundwork for comparing data from multiple time points. The combination of capturing shape with smartphone LiDAR and ensuring positional consistency with LRTK is a well-matched approach for developing cultural heritage 3D records beyond one-off datasets into information assets that continue to be used on site.

Summary

The way to record cultural heritage in 3D with smartphone LiDAR is not simply pointing the device and scanning, but defining the purpose of the recording, assessing on-site conditions, combining it with photographs, and preserving it in a form that can be reused.

The four essential, fail-safe steps are: first, clearly determine what to record; next, develop a capture plan tailored to the site conditions; then collect data so that shape and surface information complement each other; and finally, organize and store it so it can withstand future comparison and sharing. Smartphone LiDAR is a powerful means of lowering the barrier to documenting cultural heritage, but it is not a panacea. That is why understanding its strengths and, when necessary, combining it with photographs and high-precision positioning is the quickest path to success.

If you want to cultivate 3D records of cultural heritage not as attractive one-off models but as practical data usable for preservation, management, sharing, and re-examination, it is important to consider workflows that preserve shape and position together, incorporating concepts of high-precision positioning like LRTK in addition to smartphone LiDAR.