Is SfM Suitable for Cultural Heritage Documentation? 5 Basics to Know Before You Implement It

Table of Contents

• Understanding what SfM is from the perspective of cultural heritage documentation

• Basic 1: SfM can be used for cultural heritage documentation, but it is not a cure-all

• Basic 2: The quality of the results is largely determined by the image acquisition plan

• Basic 3: Accuracy is determined by coordinate management and the placement of reference points

• Basic 4: Lighting, material properties, and the surrounding environment affect reproducibility

• Basic 5: Deciding the purpose of the record in advance leads to long-term usability

• Summary

Understanding What SfM Is from the Perspective of Cultural Heritage Documentation

SfM is a method that estimates camera positions and the shape of the subject from overlapping photographs and reconstructs three-dimensional forms. In the field of cultural heritage, it has been used to document a wide range of subjects, including buildings, stone structures, archaeological remains, small artifacts, decorative elements, and records of surface condition. The reason SfM is attracting attention in cultural heritage surveys is that because it enables three-dimensional reconstruction based on photographs, it can relatively easily be introduced on site and can flexibly handle subjects from large areas to small objects. In practice, technical guides for cultural heritage recording note that photogrammetry, including SfM, can be applied across scales from landscape to small objects and can serve as basic material for conservation planning, documentation, condition assessment, and analysis.

However, what is important in the field of cultural heritage documentation is not to understand SfM simply as a convenient method for creating three-dimensional models. What really matters is first defining what to record, to what accuracy, and for what purpose, and then designing the photography and processing to match that purpose. In practical work in the cultural heritage sector, imaging should not be an end in itself but positioned as part of a methodology to answer the initial questions. In other words, objectives such as capturing the shape of the object, checking deterioration, comparative verification, conservation decision-making, and public use should come first, and it is necessary to determine whether SfM is truly suitable for those objectives.

With that premise in mind, the answer to the question "Is it possible to document cultural heritage with SfM?" is no longer a simple yes-or-no. The answer is "Yes. However, it will lead to strong results only if the process is designed to account for the nature of the object, the required level of accuracy, the site conditions, and how the data will be used in the future." SfM for cultural heritage documentation should not be chosen merely for its ease of use. The more the person responsible can strategically design how records are created, the more effectively SfM’s strengths can be leveraged.

Basic 1 SfM can be used for cultural heritage documentation but it is not a panacea

One major reason SfM is valued in the field of cultural heritage documentation is that it makes it easy to record both shape and appearance without directly touching the object. If photographs have sufficient overlap, the object's surface has distinguishing features, and shooting conditions are stable, one can produce three-dimensional records of the current condition, orthoimage-equivalent outputs, models for surface observation, and baseline data for comparison. Its high versatility is why it is applied in a wide range of contexts from fieldwork to storage—historic buildings, stone structures, excavation sites, and small artifacts. Technical documents related to cultural heritage also show that SfM is being used not only for archaeological features and buildings but also for stored collection items and small artifacts.

On the other hand, SfM is not a universal method for three-dimensional recording. Because it bases its computations on correspondences between photographs, processing stability tends to decrease for subjects with few surface features, highly reflective subjects, highly transparent subjects, subjects with strong reflections of their surroundings, or environments where moving objects enter the scene during capture. In general, SfM-based reconstructions tend to lose accuracy on polished surfaces, strongly reflective surfaces, and surfaces with little texture, and static scenes and conditions close to diffuse reflection are considered advantageous. In cultural heritage, metallic ornamentation, lacquer-like glossy surfaces, objects displayed behind glass, wet stone, homogeneous white walls, and designs with strong repeating patterns are precisely the kinds of subjects that are prone to this weakness.

Therefore, when introducing SfM you should not judge it solely on whether a model can be produced. Even if it looks like a plausible three-dimensional model, the geometric accuracy and reproducibility required for preservation and research may not be guaranteed. In cultural heritage documentation, looking good and being usable as a record are not the same thing. What the person in charge should check is not the model’s appearance but whether the necessary dimensions can be read, whether it can be treated by the same standards for comparisons, whether it will be consistent if re-photographed, and whether it will still make sense to a different person in the future.

Additionally, in the field of cultural heritage it is important to recognize that the objects are highly valuable and difficult to redo. For ordinary subjects you can often compensate by re-shooting or re-processing, but with cultural properties overlapping constraints—restrictions on public display, access restrictions, lighting limitations, seasonal conditions, and the timing before and after conservation work—mean that if you miss a single recording opportunity you may not be able to reproduce the same conditions. That is why, when using SfM, you must prepare with the mindset of "ensuring a recording opportunity is reliably converted into results" rather than "just trying it out casually." SfM is sufficiently practical, but being practical and being unconditionally unlikely to fail are not the same.

Basic 2: The quality of results depends heavily on the shooting plan

What most strongly determines the success or failure of SfM is not how the processing software is operated, but the planning done before shooting. If you enter the site with ambiguity about what area to record, from which angles to shoot, how much overlap to capture, how close to get, and which parts to prioritize, it becomes difficult to make up for that in post-processing. Practical guidance related to cultural heritage likewise repeatedly states that acquiring multiple overlapping images is a prerequisite for SfM and that acquisition strategies tailored to the subject and purpose are important. Furthermore, recent research in cultural heritage emphasizes the importance of identifying standardized, reproducible shooting strategies matched to the characteristics of the subject.

The first thing to consider in a photography plan is how to reconcile photographs that capture the entire subject with dense, close-up images of important areas. In cultural heritage documentation, having only well-composed overall views is of limited practical value if images of damaged areas or decorative elements are coarse. Conversely, if only the details are captured precisely but their positional relationship to the whole is unclear, the images become difficult to use for comparison or explanation. Therefore, on site it is fundamental to combine circumferential captures that connect the whole with supplementary close-up captures of important parts, structuring the acquisition so as not to break the continuity of the overall model. Depending on the size of the subject, it is important to consciously separate the roles of photos that record the scene and photos that increase information density.

The quality of the connections between images is more important than the number of photos themselves. If overlap is insufficient, corresponding points will be unstable; conversely, when the required overlap is ensured, the estimation of three-dimensional shape tends to be more stable. In the practical guide to conservation techniques, setting the amount of overlap is identified as an important preparatory item even in flight planning, and SfM is described as a method that computes three-dimensional shape from overlapping images. At cultural heritage sites, this same approach is necessary not only for aerial photography but also for ground-based shooting, and the shooting sequence must be planned so that neighboring photos reliably connect.

Furthermore, in cultural heritage documentation it is important to abandon the assumption that everything can be completed in a single shooting session. Before the main capture, perform test shots of part of the subject to confirm whether feature points are easy to pick up, whether shadowing is not extreme, whether surface reflections will not be problematic, and whether the distance yields the necessary level of detail; doing so can greatly reduce failures in later processing. Especially with complex three-dimensional forms, deep carvings, surfaces with many recesses, around display cases, or shooting over scaffolding, things that seem fine on site can actually result in weak connectivity between photos. Because revisiting or re-acquiring cultural properties is costly, planning that includes test shooting will determine the quality of the results.

Basic 3 Accuracy is determined by coordinate management and the placement of reference points

When conducting cultural heritage documentation with SfM, an easily overlooked aspect is coordinate management. Even if a three-dimensional model can be produced, whether it is merely a visual model or a record capable of withstanding comparisons of position and dimensions depends greatly on how reference controls are applied. Simply reconstructing three-dimensional shape from photographs does not automatically determine absolute scale or position in real space. Only by providing scale bars, known points, or high-precision coordinate information does the result become practically usable. Public technical guidance likewise shows that photogrammetric models require scaling and georeferencing, and that scale bars, known points, or high-precision positional information determine the meaning of the model.

Especially in cultural heritage documentation, whether future comparisons will be possible becomes a major determinant. For example, if you consider comparisons before and after conservation work, checking changes over time, tracking deformation or delamination, organizing relationships with surrounding topography, or overlaying other survey results, the management of coordinate systems and reference points should be included in the design from the outset. In cultural heritage research, a framework commonly used is to orient models with ground control points and assess 3D accuracy at separate locations, and guides for conservation practice also emphasize ground control points as elements that link models to real-world or local coordinates.

What is important here is to adapt the design of the reference to the required level of accuracy. If the primary purpose is 3D viewing for public distribution, it has a certain value even without strict coordinate control. However, if you are considering drafting, displacement checks, partial comparisons, or the continued use of survey records, you need to think about where to set the origin, which points will serve as reusable benchmarks, and how to reproduce them on site. Because cultural heritage data is often used for a long time, models that are easier to reconnect later have more value than one-off models.

Also, checking accuracy only once after processing is insufficient. You need to consider whether the reference standards assumed at the acquisition stage are correctly reflected in the model, whether distortions have occurred in important parts of the subject, and whether results can be reconciled if re-acquired on a different day. In cultural heritage preservation and investigation, it can be important not only to look at simple mean errors but also to determine which parts, in which directions, and to what extent deviations occur. Therefore, when verifying deliverables, separating the assessment into three aspects—overall consistency, local consistency of important parts, and reproducibility of reference points—makes practical decision-making easier.

Basic 4: Light, Materials, and the Surrounding Environment Influence Reproducibility

When using SfM for cultural heritage recording, the on-site conditions that have particularly large effects are lighting, material properties, and the surrounding environment. Because SfM constructs shape by tracking common features in images, extreme changes in light and shadow or reflections that make appearances vary from photo to photo make it difficult to recognize the same place as the same object. General SfM research also cites highly reflective surfaces and feature-poor surfaces as factors that reduce accuracy, and it is considered important to ensure a stable appearance in a static scene.

This problem is very common on-site at cultural heritage locations. Even stone changes its reflectivity when wet, and metal or lacquer-like finishes produce reflections that differ from photo to photo. In exhibition spaces glass surfaces and lighting reflections appear, and outdoors shadows can shift dramatically depending on the time of day. Tree shadows, grass swaying in the wind, visitors’ movements, and the presence of scaffolding or protective coverings also undermine consistency between photographs. Because the object itself often cannot be altered in cultural heritage documentation, the photographing team must adjust lighting conditions and shooting order to minimize these changes as much as possible.

Furthermore, it is important to understand which materials are suitable or unsuitable. Stone, wood, and soil-based surfaces with fine irregularities or color variations can relatively easily yield feature points. On the other hand, homogeneous painted surfaces, monotonous walls, decorations that are almost mirror-like, and materials that involve transparency or translucency may not provide stable feature points. The important point here is not to treat the subject as a single lump. For example, even within a single building, conditions differ for wall surfaces, openings, decorative elements, roofing materials, metal fittings, and the plinth. Recording plans should not be applied uniformly to the entire subject but should be varied and captured according to the difficulty of each part.

Consideration for the surrounding environment is also essential. At cultural heritage sites, access routes may be limited and it may not be possible to maintain sufficient distance. In high or recessed locations, there will inevitably be more biased photographs taken from above or below, which can make shape estimation unstable. In outdoor remains, terrain and vegetation can obstruct the view, and in cramped storage environments, the background may be too close, making it difficult to separate the subject. In situations with such constraints, rather than attempting from the outset to "photograph everything uniformly well," it is more practical to prioritize important parts, clearly document areas that cannot be captured, and reflect those limitations in the deliverables. Recording is not about filling in everything perfectly; it is also an act of making clear how thoroughly something has been reliably captured.

Basic 5: Deciding the purpose of records in advance leads to long-term use

When introducing SfM into cultural heritage documentation, the greatest practical differences stem from whether the purpose for which the data are being preserved is clearly defined from the outset. Even with the same SfM, if the purpose changes, capture methods, accuracy requirements, and necessary deliverables can change dramatically. If the primary purpose is viewing or exhibition use, visually appealing presentation and an overall understanding are emphasized. If conservation and repair or longitudinal comparison are prioritized, positional reproducibility, scale consistency, and comparability at re-photography become more important. As a principle of cultural heritage imaging, it is therefore important to choose appropriate methods according to the initial question and, where necessary, combine them with other techniques.

If this perspective is missing, you may be able to produce deliverables but have trouble with their operation. For example, something that initially looked sufficient as a 3D model for publicity may later be unusable for comparing deformations or verifying before-and-after repairs. Conversely, you might produce extremely heavy data intended for surveys, but because sharing and viewing it is cumbersome, no one on site ends up using it. In cultural heritage documentation, who can use the data and how they can use it months or years later is more important than the immediate satisfaction of acquisition. Agreeing in advance on the required level of deliverables among preservation, survey, exhibition, and management staff prevents duplicated work and re-shooting.

Moreover, when planning for long-term use, not only the model itself but also records of imaging conditions and reference information become assets. If it is documented when, over what area, under what conditions, and according to which standards the data were acquired, future reacquisition and comparison will be much easier. In cultural heritage, because there are many situations where continuous observation of the same object is meaningful, data preserved as reproducible procedures is more valuable than data produced as a one-off. In recent cultural heritage research, the ability of on-site staff to implement reproducible acquisition strategies has been treated as an important issue.

And finally, what you should keep in mind is not to think of SfM in isolation. SfM is a very powerful option for cultural heritage documentation, but it is not a tool that solves every problem by itself. By combining it, as needed, with other measurements, existing drawings, positional information, and on-site observation records, the reliability of the documentation is greatly increased. In practice, digital documentation of cultural heritage is far more about reconciling multiple pieces of information and making them reusable later than about producing a single flashy artifact. SfM can be at the core of that, but to make it the core you must design its connections with peripheral information.

Summary

SfM is fully capable of recording cultural heritage. Moreover, if it can be designed to suit the subject and site conditions, it can be a powerful tool for understanding current conditions, three-dimensional recording, comparative verification, and creating shared materials. However, its value is not determined solely by the fact that “a 3D model can be made from photographs.” Whether there is a shooting plan suited to the subject, whether the necessary overlap can be secured, how coordinates and reference points are established, how weaknesses in lighting and materials are addressed, and whether future uses have been anticipated—all of these make the difference in outcomes. It is important to regard SfM in cultural heritage recording not as a convenient shortcut but as a technique for designing accuracy and reproducibility according to the purpose.

In particular, for records of outdoor cultural properties, ruins, stone structures, and areas around historic sites, it is not enough to have a three-dimensional model alone; linking the acquired data to on-site coordinates directly supports subsequent comparison and management. In such situations, combining measures that streamline on-site coordinate verification and control-point management and that make it easier to assign positional reference to photo-derived records further increases the practical value of SfM. If you want records of cultural properties to become data usable for future conservation and research rather than merely visualization, it is essential to design how location information will be handled alongside 3D modeling.

In that sense, if you want to carry out on-site reference point checks and coordinate referencing as nimbly as possible, adopting smartphone-mounted high-precision GNSS positioning devices such as LRTK can be an effective option. By adding a locally usable geolocation foundation to cultural heritage records obtained by SfM, the reusability of those records is greatly improved. In work to preserve cultural heritage, data that can be compared, explained, and used in the field years later is more valuable than data that is only useful on the day it was captured. If you are considering adopting SfM, the first step to avoid failure is to treat it not merely as creating three-dimensional models but as an operational plan that includes coordinate management.