What is SfM processing? Basics of creating 3D models even beginners can understand

In recent years, the technique of creating three-dimensional models from photographs, known as SfM processing, has been attracting attention. In construction, surveying, and civil engineering, cases of using drones and digital cameras to record sites three-dimensionally have been increasing. This article explains in an easy-to-understand way for beginners what SfM (Structure from Motion) is, how it works, the equipment required, the basic procedure, real-world use cases, and precautions. Finally, we also introduce LRTK, a latest simple surveying solution that supports SfM.

What is SfM processing?

SfM (Structure from Motion) is a type of computer vision technique that reconstructs the three-dimensional structure (3D model) of an object or site from multiple photographic images. It is a form of photogrammetry and is characterized by the ability to obtain detailed 3D information using only photos taken with a digital camera, without the need for expensive measurement equipment such as special laser scanners. For example, by taking dozens of photos of a building or terrain from various angles and processing that set of photos with dedicated software, the shape of the subject can be reproduced in three dimensions as point cloud data or polygon meshes.

Traditionally, 3D measurement of terrain and structures relied on manual surveying work or the use of laser scanners (LiDAR). However, with the advent of SfM technology, 3D models can be created using inexpensive equipment and automated image processing. Recently, improvements in computer performance and the widespread availability of drones and digital cameras have made it possible to quickly capture and process large numbers of photos, greatly enhancing the convenience of SfM processing. For these reasons, SfM processing is beginning to be widely used in various fields, especially in construction and civil engineering. In particular, the Ministry of Land, Infrastructure, Transport and Tourism’s promotion of “i-Construction” and CIM (Construction Information Modeling) has highlighted the acquisition of point cloud data using SfM, greatly contributing to site digitization and efficiency.

Equipment required for SfM processing

To perform SfM processing, the following equipment and environment are mainly required:

• Camera (imaging equipment): A high-resolution digital camera is desirable, but recently smartphone cameras can capture images with sufficient quality. To shoot the subject or site from various angles, preparing a wide-angle lens or a tripod can help obtain stable photos.

• Drone (for aerial photography): Drones are effective when photographing wide-area terrain or large structures. By automatically flying overhead and taking many photos, you can acquire data for the entire site in a short time. Drone aerial photography is especially useful in surveying for construction and civil engineering.

• PC and SfM software: A computer and software are required to process the captured photo data and convert it into a 3D model. The greater the number of photos, the higher the processing load, so prepare a high-performance PC (sufficient CPU power and memory, and a GPU if possible). Use dedicated applications that perform SfM analysis and point cloud generation. There are options ranging from advanced commercial software to free open-source tools, which can be selected according to purpose and budget.

• Surveying tools (optional): To assign real-world scale (dimensions) or coordinates to the created 3D model, it is ideal to have reference lengths or position information measured on site. For example, you can measure reference distances with a tape measure or install known coordinate points (targets) on-site and measure them with surveying instruments or GNSS. For projects requiring strict surveying accuracy, installing such control points (GCPs) is recommended, but it may be challenging for beginners to implement easily.

Steps of SfM processing

SfM processing generally proceeds through the following steps:

• Planning and preparation: First, create a shooting plan according to the subject or site conditions. Determine shooting positions and angles that can cover the entire subject and plan so that there is sufficient overlap between photos. If necessary, plan the drone flight, perform safety checks, and prepare equipment.

• Photography: Photograph on-site according to the plan. Take many photos from all directions of the subject with an overlap rate of around 60–80% or more. Fix camera settings and take care to avoid blur to obtain sharp, in-focus photos. For outdoor shooting, pay attention to weather and the direction of sunlight to maintain stable exposure.

• Photo alignment (SfM analysis): Import the captured photos into software and execute the SfM process. The software detects feature points (distinctive patterns or corners) across multiple photos and matches these common points to automatically calculate camera positions and orientations. As a result, shooting positions and poses for each photo and a coarse point cloud model of the subject (sparse point cloud) are generated.

• Point cloud generation and 3D modeling: Next, using the camera positions and initial point cloud obtained by SfM, generate a denser point cloud. This process is also called multi-view stereo (MVS) processing, in which corresponding 3D points are calculated for pixels across all photos to generate detailed point clouds and mesh models. The resulting point cloud can contain millions of points; by converting this into a polygon mesh and applying textures, a photorealistic 3D model is completed.

• Output and utilization of results: Finally, output and utilize the generated 3D model data according to your purpose. Point cloud data and mesh models can be displayed in dedicated software to measure site dimensions or imported into CAD software for design and construction review. You can also create orthophotos of the entire terrain or extract cross-sections to calculate earthwork volumes. As needed, correct the model scale using known lengths or verify the alignment accuracy before using the results.

Use cases of SfM

3D model creation by SfM is used in various situations. Here are some representative examples:

• Surveying construction sites: Create 3D terrain models (point clouds and orthophotos) from drone aerial photos to grasp site conditions before construction. Whereas traditional total station surveys captured terrain information only as points, SfM point clouds capture surfaces, improving the accuracy of earthwork calculations and site development planning. SfM is also being used for pre-construction land surveys in i-Construction.

• Progress management and as-built verification in civil works: Used to record intermediate progress and final shapes of works. By periodically photographing with a drone and comparing SfM-generated 3D models over time, the progress of excavation and embankment can be visualized. As-built shapes can be captured as point cloud data and compared with design data to inspect quality.

• Infrastructure inspection and maintenance management: SfM helps in maintenance of infrastructure such as bridges, tunnels, and dams. Even at heights where close visual inspection is difficult, photographing and creating 3D models enable safe inspection of details. Cracks and deformations can be measured and recorded on the model, enabling future change monitoring.

• Disaster investigation and map creation: SfM is used at disaster sites such as landslides and floods to quickly grasp current conditions. Data can be collected by drone even in hazardous areas immediately after a disaster, and the resulting point clouds can be used to analyze the extent and volume of collapse or to create topographic maps useful for restoration planning. SfM is also applied to wide-area mapping such as mountain terrain surveys and urban 3D mapping.

• Cultural heritage recording and 3D content production: There are increasing cases of precisely recording sculptures and structures from photos and preserving and publishing them as 3D models. SfM, being non-contact, is effective for preserving valuable ruins. It is also being used in entertainment to capture real buildings or people as 3D models for games and VR content.

Precautions when performing SfM processing

While SfM processing is convenient, there are several points to be aware of to ensure success.

• Sufficient number of photos and overlap: To obtain high-quality models with SfM, capture enough photos to cover the subject and ensure appropriate overlap between photos. Generally, adjacent photos should overlap by 60% or more, and ideally around 80%, to achieve stable reconstruction. Too few photos or coverage gaps can result in parts not being reconstructed or distortions in the model shape.

• Quality of photographs: The quality of the captured photos also affects the results. Obtain sharp, blur-free images with good focus. Fix autofocus and exposure settings during shooting to avoid inconsistencies. Also, if the subject or surrounding objects move during shooting, SfM may not function correctly, so photograph with the subject as still as possible.

• Distinctive features of the subject: SfM tracks feature points on photos to reconstruct 3D, so it struggles with surfaces that lack features or reflect light. For example, glass, water surfaces, and mirrors do not yield good point clouds. Plain white walls or single-color floors also do not provide detectable points, making modeling difficult. In such cases, adding markings in advance or combining with laser scanning may be necessary.

• Processing time and data management: The more photos you capture, the longer processing will take and the larger the generated point cloud data will be. Handling hundreds of photos can make SfM analysis take several hours to more than ten hours. Ensure sufficient PC storage and memory and avoid interruptions during processing. Also, because the resulting point cloud and model files can be very large, proper storage and backups are important.

• Ensuring accuracy and scale: Models obtained by SfM processed only from photos have arbitrary scale. To match accurate dimensions or coordinate systems, you must assign scale using known on-site measurements or reference points. For example, calibrate the model using a measured distance between two points, or georeference the model by setting coordinates of pre-placed targets. Skipping these accuracy-assurance steps can lead to errors in distances or volumes derived from the model.

• Regulations and safety considerations: Especially when using drones, it is essential to comply with aviation laws and other regulations and to operate with safety in mind. Check for no-fly zones, implement sufficient safety measures, and ensure that third parties are not harmed or their privacy violated. Also, consider people and the environment during shooting. When working at height, take precautions to prevent falls and do not neglect basic safety measures.

For beginners, rather than trying a vast site at once, it is recommended to start practicing SfM on familiar objects or small areas. Gaining experience from small model creation helps you learn shooting and processing techniques, enabling higher-accuracy results.

Simple surveying solution supporting SfM using LRTK

While SfM processing makes it easy to obtain 3D models, tasks such as installing control points for accuracy improvement and learning photography tips can be hurdles for beginners. A new technology attracting attention to support these tasks is a simple surveying solution using LRTK.

LRTK is a system that combines a small high-precision GNSS receiver with a smartphone app to easily acquire centimeter-level position information (cm level accuracy (half-inch accuracy)). It leverages satellite positioning technology called RTK-GNSS, enabling high-precision positioning that previously required specialized equipment to be achieved affordably with simple devices and apps. With a single button on-site, current position can be measured, and positioning completes with a horizontal error on the order of several cm (several in).

By incorporating LRTK into the SfM workflow, beginners can efficiently create high-accuracy 3D models. For example, if you take photos with a smartphone equipped with an LRTK device, each photo can be tagged with high-precision position coordinates. Importing that data into SfM software allows the model to be automatically aligned to real-world coordinate systems and scaled. This can eliminate the need to install GCPs or perform later scale calibration, enabling accuracy-controlled 3D models without specialized surveying knowledge.

Moreover, by mounting an LRTK receiver and camera on a helmet and walking the site, you can collect surrounding 3D data without installing reference points. Compared to traditional manual surveying, the site workflow is dramatically simplified, greatly reducing the effort and time required for surveying.

Also, using LRTK makes it possible to proceed with surveying while confirming drone or ground photo data in real time. With cloud integration, position information and point clouds acquired on-site can be shared and analyzed immediately, enabling faster as-built verification and daily reporting than before. This simple surveying solution that combines SfM and GNSS will make 3D measurement work that once relied on specialists more accessible. For beginners interested in SfM processing, the era when high-precision surveying can be easily achieved by utilizing LRTK has arrived.

Thus, the simple surveying solution enabled by LRTK is turning three-dimensional measurement work that once required specialist skills into something anyone can do. Combined with the convenience of SfM processing, the use of 3D data on construction and surveying sites is likely to accelerate further. With the combination of SfM and LRTK, a time when everyone can handle high-precision 3D data is coming into full swing. Take this opportunity to try SfM processing and experience the digital transformation of your site.